I am trying to retrieve data from a MySQL database by sending a MySQL query using Python. When I send the MySQL Query in MySQL workbench, it runs perfectly fine. When I try the same using Python (in a Jupyter Notebook), it returns an error. Python Code: Error: If I remove the variable in the MySQL Query it ru…
Tag: python
Django: Really slow RunPython migration on related models
I am trying to optimise the speed of a migration on a large table (250K objects). The goal is to add to each row a user field based on the user of an object related to this row: I tried using an F expression but sadly django won’t allow relations in them. Note that I am mostly a beginner in
Calculate TimeDiff in Pandas based on a column values
Having a dataframe like that: Desirable result is to get aggregated IDs with time diffs between Start and End looking like that: Tried simple groupings and diffs but it does not work: How this task can be done in pandas? Thanks! Answer A possible solution is to join the table on itself like this: Output:
Any suggestion for optimizing BCP export from SQL Server using Python Subprocess
I’m studying BCP for export large amount of data (one time for initial and plan to do it in day-to-day job). Source data are in SQL Server tables, which consist of some small tables to larger ones (…
Create Redshift table with new Geometry type through psycopg2
After Redshift announced support for Geometry types and spatial functions, I’d like to create a table with polygons for all countries. I’m failing to do the INSERT and would appreciate help. Here is …
installing h3 python library on AWS Redshift for usage in an UDF
I was trying to install the python library from ubers H3 as a custom library for aws redshift by using this for the installation. I followed the process and created the following function: When executing it with: I receive the following error: OSError: /rdsdbdata/user_lib/0/0/1334190.zip/h3/out/libh3.so.1: ca…
Search for many rows with many distinct values
I am looking to implement a cache in MySQL. I’m doing this because the app I’m developing needs a cache but has no persistence. I am keeping a collection of SHA-1 checksums for files. The user can …
Improve SQL query to find range between start and end date
I’m working with a database called international_education from the world_bank_intl_education dataset of bigquery-public-data. My aim is to plot a line graph with countries who have had the biggest and smallest change in Population growth (annual %) (one of the indicator_name values). I have done this b…
sqlalchemy concat with more than 2 elements on Oracle DB
considering the following table definition I create a select-statement using the sqlalchemy.sql.functions.concat with 3 statements using the query is generated. However, when I run this, the exception ORA-00909: invalid number of arguments is thrown. This is because CONCAT (https://docs.oracle.com/cd/B19306_0…
Python SQL script inserts only the last row of the dataframe to SQL Server [pyodbc]
I am trying to import a pandas dataframe to Microsoft SQL Server. Even though, the command is correct and executed successfully, when I select the first 1000 rows of the table in SQL server only the …