I have 10000 jsons with different ids each has 10000 names. How to flatten nested arrays by merging values by int or str in pyspark? EDIT: I have added column name_10000_xvz to explain better data structure. I have updated Notes, Input df, required output df and input json files as well. Notes: Input dataframe has more than 10000 columns name_1_a,
Tag: python
Set a databricks python variable with a %sql statement
The code in cell 1 works just fine. The problem I just don’t know the correct keywords to search to see how I can make the code in cells 2 & 3 work. Basically, in a %sql cell, can I select into a variable that can be later used in a python cell? I realize this may not even be
How to write a single query to run some retrospective aggregation when time window is different for every row?
I am writing some SQL queries to create a dataset for customer churn predictions based on historical service data. Some of the services date back years ago. Small percentage of them churned at some time in the past while others ended up getting renewed. Some of the attributes are based on aggregation of the services that were active when each
How to pass parameter in PostgresOperator Airflow using for loop
I am using PostgresOperator and I want to pass table name suffix to my SQL query so when it queries the data, it reads dynamically from the for loop iteration So as you can see I have passed .format(country) in task_id. I want to do similar stuff by passing country name like in the below SQL statement but seems like
AttributeError: ‘NoneType’ object has no attribute ’email’. Flask-sqlalchemy
i’ve a strange problem… i do this: and gives me this problem: Who can help me?? thanks <3 Answer it means it didn’t find any users matching your filter
Python: cx_Oracle does not like how I am entering date
I am trying to do a simple select all query in python using the Cx_oracle module. When I do a select all for the first ten rows in a table I am able to print our the output. However when I do a select all for the first ten rows for a specific date in the table all that gets
Retrive data of last month using Python with source as MYSQL
I am getting an error while retrieving data of the last month using python. What is the best approach for the given piece of code? I am not able to get solution, What changes can I do. I am using MYSQL as my source database and Python 3.7.4 as my programming language. Since error is big, posting it in two
Database Error: NameError: name ‘db’ is not defined. What am I doing wrong?
On the web I found this illustration to create a database with gui with Tkinter. Everything ok, except when I enter the data and click on the Add button. I’m getting: NameError: name ‘db’ is not defined I think what I am wrong is nonsense in this part of the code. What am I doing wrong? Here is my code
How to update a specific value of a object present in array of object within Postgres JSON Field
Here is my JSON field value. which is stored in the PostgreSQL table. I want to search and update the specific user_name in the user key whose name is Devang to Dev using Django JSONField for example I have tried the RAWQuery for the find. This is the query. It will return like this I have also tried JSON_SET to
time data ‘(datetime.date(2021, 7, 30), )’ does not match format ‘%Y/%m/%d’
I am accessing date from database using below query, in my jupyterLab notebook: it is giving this ValueError: time data ‘(datetime.date(2021, 7, 30), )’ does not match format ‘%Y/%m/%d’ can anyone guide, the correct way pls? Answer Seems like c_date is already a datetime.date object. You don’t need to cDate = str(c_date). try: