「 Python 」 一覧

Description Pandas is powerful method to deal with the data and here we are showing how to aggregate by this method with some examples. Data for test We are preparing test data like following.以下のようなテスト用の時系列データフレームを用意します。 import pandas as pd from pandas import Series datetime = pd.date_range(start='2021-12-31',end='2022-01-02', freq='30s') df = pd.DataFrame(range(len(datetime)), index=date, columns = ['val']) df.head() df.tail() Method１ : resample The resumple function is actually for downsampling but it’s also useful to aggregate by grouping datetime. You have to remember to set datatime column as index in advance. df['datetime'] = pd.to_datetime(df['datetime']) df.set_index('datetime', inplace=True) And then you set the character which represents year, …

Description When you need to input your ID and Password for certification by program, It’s not good idea to write them as plaintexts in your source code because of security reason. It’s safer to let human to input them at each time. In python, you can use input function for it, but there is a problem the password characters you are inpurting is showed at the screen. Alternatively you can use getpass function hide the characters and now we show the way to use it. Example of input function Firstly, we are showing the example of input function. when input …

Overview In this article, there is a explanation of Multiple Regression Analysis by using statsmodels in python. We focus not analyzing but understanding how to use this library. Environments Python 3.8.6statsmodels 0.13.2 Preparetion of datase This time we will use Red Wine Quality dataset. This dataset is published as OpenDatabase License in kaggle site. We download it and save the csv file in any directory. Red Wine Quality | Kaggle Installing statsmodels By using pip command, we will install the statsmodels library. pip install statsmodels Loading dataset Next, we will load the dataset you save in any directory by pandas …

Overview Pandas can get data from a database with read_sql easily.Here we can show how to convert dataframe to list type for only one row from database. Example table in databse a,b,c are columns of this test_table. abc110100220200330300440400test_table Let’s get only “a” column from this table with pandas.These are sample code. import pandas as pd import psycopg2 connection = psycopg2.connect(host=’host’, dbname=’database’, user=’username’, password=’password’) df = pd.read_sql("SELECT a FROM test_table", connection) df.head() Result If you want to convert this result from dataframe to list type, you may think like [1, 2, 3, 4]. There is a simple method to convert dataframe …

Overview When Scraping with Beautiful Soup a problem occurred like not getting all rows of the table but a few of them. This example shows how to fix it Environment Python 3.7.3 Problem occurred example This is a example table you want to scrape. NumberName1Sato2Kato3Ito4Goto I saw the html code in a web browser by pushing F12 key. <table class="test_table"> <thead> <tr> <th>Number</th> <th>Name</th> </tr> </thead> <tbody> <tr> <td>1</td> <td>Sato</td> </tr> </tbody> <tbody> <tr> <td>2</td> <td>Kato</td> </tr> </tbody> <tbody> <tr> <td>3</td> <td>Ito</td> </tr> </tbody> <tbody> <tr> <td>4</td> <td>Goto</td> </tr> </tbody> </table> When I use the following code to scrape the …

Overview There is an example code to insert data in bulk to Oracle database with python cx_Oracle Environments python 3.7.3Oracle 18c Express Edition Step1 : Creating the table Creating the table ‘oracle_insert’ in the schema ‘USER01’. CREATE TABLE USER01.ORACLE_INSERT( col1 int ,col2 int ,col3 int ) Step2 : Insert in bulk with executemany dataset is the dataset to be insertedMaking a connection to database with cx_OracleInserting in bulk with using cur.executemany() import cx_Oracle dataset =[ [1,2,3] ,[4,5,6] ,[7,8,9] ,[10,11,12] ,[13,14,15] ,[16,17,18] ,[19,20,21] ] HOST = ‘localhost’ PORT = ‘1521’ DB_NAME = ‘xepd1’ USER = ‘user01’ PASSWORD = ‘password01’ SERVICE_NAME = …

Original Data This code is to make sample dataframe. import pandas as pd import datetime as dt df = pd.DataFrame([dt.datetime(2020,6,1,0,0,0),dt.datetime(2020,6,2,10,0,0),dt.datetime(2020,6,3,15,0,0)],columns=[’datetimes’]) 　　　　datetimes 0 2020-06-01 00:00:00 1 2020-06-02 10:00:00 2 2020-06-03 15:00:00 Transforming dataframe to date and time. It can realize to use lambda & apply functions like following. df[’dates’] = df[’datetimes’].apply(lambda x : dt.date(x.year,x.month,x.day)) df[’times’] = df[’datetimes’].apply(lambda x : dt.time(x.hour,x.minute,x.second)) 　　　　 datetimes 　　　　 dates　　 times 0 2020-06-01 00:00:00 2020-06-01 00:00:00 1 2020-06-02 10:00:00 2020-06-02 10:00:00 2 2020-06-03 15:00:00 2020-06-03 15:00:00 If you want to get with string, these code is better. df[’dates’] = df[’datetimes’].apply(lambda x : x.strftime(‘%Y/%m/%d’)) df[’times’] = df[’datetimes’].apply(lambda x …

Overview Dataframe type in python is so useful to data processing and it’s possible to insert data as dataframe into MySQL . There is a sample of that. Environments Python 3.7.3MySQL 5.5.62 Step1 : Making the table Defining a table like the following. > CREATE DATABASE testdb; > CREATE TABLE testdb.mysql_table( col1 int ,col2 int ,col3 int ); Step2 : Making data Making data for insert from python. the data should be the same type as the table you will insert it. The column name of dataframe is also same as the table if they are different you will get …

Overview There are some samples to connect PostgreSQL, Oracle, MySQL from Python. How to connect PostgreSQL Package installation pip install psycopg2 Example import psycopg2 HOST = ‘your_host’ PORT = ‘5432’ DB_NAME = ‘your_db_name’ USER = ‘your_user_name’ PASSWORD = ‘your_password’ conn = psycopg2.connect("host=" + HOST + " port=" + PORT + " dbname=" + DB_NAME + " user=" + USER + " password=" + PASSWORD ) cur = conn.cursor() cur.execute("select version()") rows = cur.fetchall() cur.close() conn.close() print(rows) Connect to MySQL Package installation pip install mysqlclient Example import MySQLdb HOST = ‘your_host’ PORT = 3306 #Not str but int DB_NAME = ‘your_db_name’ …

Overview There is a good library in python to make GUI, that is tkinter. It can display images too. Here, we are trying to change images periodically with tkinter. Python library to be used Following three libraries are used. tkinter ‘tkinter’ is used to make GUI. PIL(pillow) ‘PIL’ is used to deal with images in the python. threading ‘threading’ is used to change images automatically. Detail about that will be explained later. By the way, if you are using anaconda which contains these three libraries already , you don’t have to install them with pip command. Step1 : Display a …