{"id":532,"date":"2018-10-16T08:24:21","date_gmt":"2018-10-16T06:24:21","guid":{"rendered":"http:\/\/35.180.88.53\/?p=532"},"modified":"2018-10-16T08:24:24","modified_gmt":"2018-10-16T06:24:24","slug":"what-i-learned-yesterday-19-pandas-time-series","status":"publish","type":"post","link":"https:\/\/www.sergilehkyi.com\/uk\/2018\/10\/what-i-learned-yesterday-19-pandas-time-series\/","title":{"rendered":"What I Learned Yesterday #19 (pandas Time Series)"},"content":{"rendered":"\n<p>Recently I learned that the best way to digest information, assimilate it is a two-step algorithm:<\/p>\n\n\n\n<ul><li>Share new information within 12-24 hours with 2 different persons.<\/li><li>Apply new knowledge &#8211; practice it.<\/li><\/ul>\n\n\n\n<p>And that&#8217;s it. Nothing complicated. So I decided to use my blog for the first part.<\/p>\n\n\n\n<p>I constantly learn new data science techniques, so here I want to share what was the most recent.\u00a0<\/p>\n\n\n\n<p>(technical text begins here) In the last lesson I learned more about pandas time series and how to work with indexes that contain this type of data.<\/p>\n\n\n\n<p>First and very awesome characteristic of time series index is partial datetime string selection:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"># Select sales data for the 5th of February, 2015<br\/>sales.loc['2015-02-05']\nsales.loc['February 5, 2015']\nsales.loc['2015-Feb-5']\n\n# Whole month\nsales.loc['2015-2']\n\n# whole year\nsales.loc['2015']\n<\/pre>\n\n\n\n<p>Be careful with data types. If your index consists of strings the tricks above won&#8217;t work. To convert a string to a datetime we can use pandas <em>to_datetime()<\/em> method. Specifying format parameter helps with the formatting. Default format is ISO 8601 (&#8216;yyyy-mm-dd hh:mm:ss&#8217;)<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">pd.to_datetime(['2015-2-16', '2015-2-20'], format='%Y-%m-%d %H%M%S)\n<\/pre>\n\n\n\n<p>Other cool feature of time series index is resampling. There are two types of it: downsampling and upsampling. Former is when we have 9 rows of data for 9 hours each row representing each hour. We can downsample it and get a summary for 3 hour groups. Example:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><strong>>>> <\/strong>index = pd.date_range('1\/1\/2000', periods=9, freq='H')\n<strong>>>> <\/strong>series = pd.Series(range(9), index=index)\n<strong>>>> <\/strong>series\n2000-01-01 00:00:00    0\n2000-01-01 01:00:00    1\n2000-01-01 02:00:00    2\n2000-01-01 03:00:00    3\n2000-01-01 04:00:00    4\n2000-01-01 05:00:00    5\n2000-01-01 06:00:00    6\n2000-01-01 07:00:00    7\n2000-01-01 08:00:00    8\nFreq: T, dtype: int64<br\/><br\/><strong>>>> <\/strong>series.resample('3H').sum()\n2000-01-01 00:00:00     3\n2000-01-01 03:00:00    12\n2000-01-01 06:00:00    21\nFreq: 3T, dtype: int64<br\/><\/pre>\n\n\n\n<p>Upsampling is an operation in opposite direction. Example,\u00a0\u00a0upsample the series into 30 second bins. <\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><strong>>>> <\/strong>series.resample('30S').asfreq()[0:5] <em>#select first 5 rows<\/em>\n2000-01-01 00:00:00   0.0\n2000-01-01 00:00:30   NaN\n2000-01-01 00:01:00   1.0\n2000-01-01 00:01:30   NaN\n2000-01-01 00:02:00   2.0\nFreq: 30S, dtype: float64<\/pre>\n\n\n\n<p>Upsample the series into 30 minute bins and fill the\u00a0<code>NaN<\/code>\u00a0values using the\u00a0<code>pad<\/code>\u00a0method.\u00a0\u00a0<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><strong>>>> <\/strong>series.resample('30T').pad()[0:5]\n2000-01-01 00:00:00    0\n2000-01-01 00:30:00    0\n2000-01-01 01:00:00    1\n2000-01-01 01:30:00    1\n2000-01-01 02:00:00    2\nFreq: 30S, dtype: int64\n\n<\/pre>\n\n\n\n<figure class=\"wp-block-image\"><img loading=\"lazy\" decoding=\"async\" width=\"754\" height=\"538\" src=\"http:\/\/35.180.88.53\/wp-content\/uploads\/2018\/10\/image.png\" alt=\"\" class=\"wp-image-533\" srcset=\"https:\/\/www.sergilehkyi.com\/wp-content\/uploads\/2018\/10\/image.png 754w, https:\/\/www.sergilehkyi.com\/wp-content\/uploads\/2018\/10\/image-300x214.png 300w\" sizes=\"(max-width: 754px) 100vw, 754px\" \/><figcaption>And a chetlist for frequencies<\/figcaption><\/figure>\n\n\n\n<h4 class=\"wp-block-heading\">Data Visualization<\/h4>\n\n\n\n<p>We have a variety of option to customize our plots using python. We can change color, marker and line type. Below is a little summary of available options<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img loading=\"lazy\" decoding=\"async\" width=\"714\" height=\"522\" src=\"http:\/\/35.180.88.53\/wp-content\/uploads\/2018\/10\/image-1.png\" alt=\"\" class=\"wp-image-534\" srcset=\"https:\/\/www.sergilehkyi.com\/wp-content\/uploads\/2018\/10\/image-1.png 714w, https:\/\/www.sergilehkyi.com\/wp-content\/uploads\/2018\/10\/image-1-300x219.png 300w\" sizes=\"(max-width: 714px) 100vw, 714px\" \/><\/figure>\n\n\n\n<p>That&#8217;s all for today. I will learn something new and share it here soon. Have an awesome day!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Recently I learned that the best way to digest information, assimilate it is a two-step algorithm: Share new information within&hellip;<\/p>\n","protected":false},"author":1,"featured_media":536,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4,5,7],"tags":[],"translation":{"provider":"WPGlobus","version":"3.0.0","language":"uk","enabled_languages":["gb","es","uk"],"languages":{"gb":{"title":true,"content":true,"excerpt":false},"es":{"title":false,"content":false,"excerpt":false},"uk":{"title":false,"content":false,"excerpt":false}}},"_links":{"self":[{"href":"https:\/\/www.sergilehkyi.com\/uk\/wp-json\/wp\/v2\/posts\/532"}],"collection":[{"href":"https:\/\/www.sergilehkyi.com\/uk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.sergilehkyi.com\/uk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.sergilehkyi.com\/uk\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.sergilehkyi.com\/uk\/wp-json\/wp\/v2\/comments?post=532"}],"version-history":[{"count":1,"href":"https:\/\/www.sergilehkyi.com\/uk\/wp-json\/wp\/v2\/posts\/532\/revisions"}],"predecessor-version":[{"id":537,"href":"https:\/\/www.sergilehkyi.com\/uk\/wp-json\/wp\/v2\/posts\/532\/revisions\/537"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.sergilehkyi.com\/uk\/wp-json\/wp\/v2\/media\/536"}],"wp:attachment":[{"href":"https:\/\/www.sergilehkyi.com\/uk\/wp-json\/wp\/v2\/media?parent=532"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.sergilehkyi.com\/uk\/wp-json\/wp\/v2\/categories?post=532"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.sergilehkyi.com\/uk\/wp-json\/wp\/v2\/tags?post=532"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}