Questions tagged [azure-notebooks]
The azure-notebooks tag has no usage guidance.
azure-notebooks
113
questions
0
votes
1
answer
61
views
parse SQL script via python to a table for field, name, and table it comes from
I am trying to create a helpful table because we have a lot of raw tables and views with business definitions.
for example the design for my v_ods_ACT_Master would be
SELECT MBACCT AS [Account Number],...
- 1
-1
votes
0
answers
32
views
How do I add a SandDance visualization to my notebook in Azure Data Studio?
I do not know how to include a SandDance Visualization in my notebook in Azure Data Studio.
I created a Sundance visualization in Azure Data Studio. I also used the Snapshot feature. But I have no way ...
0
votes
1
answer
69
views
Unable to Drop a View in Serverless SQL Pool from Synapse Notebook
I am trying to drop a view from my synpase Notebook using a connection to a Serverless SQL Pool. I followed the instructions from this post: Access our built-in serverless SQL pool from our notebook ...
- 83
1
vote
1
answer
106
views
Access our built-in serverless SQL pool from our notebook
We are currently trying to access our built-in serverless SQL pool from our notebook. Our goal is to be able to drop views from the notebook, which is something we need to put into our pipeline as per ...
- 83
1
vote
1
answer
55
views
How to Connect Synapse Notebook to SQL Serverless Pool
I'm working with Azure Synapse Analytics and I need to connect my Synapse Notebook to my SQL Serverless Pool to drop and create views directly within the notebook. However, when I try to run the DROP ...
- 83
0
votes
1
answer
56
views
Parquet file not overwriting in azure synapse notebooks
I am working on azure synapsse analytics notebooks and I want to I load Parquet files into DataFrames, perform some transformations, and then overwrite the original files with the transformed data. ...
- 83
0
votes
1
answer
57
views
ADF Web Activity Error: The request content is not valid and could not be deserialized: 'After parsing a value an unexpected character was encountered
I am getting the error message in web Activity of ADF:
The request content is not valid and could not be deserialized: 'After
parsing a value an unexpected character was encountered:
Further looking ...
- 1,657
0
votes
1
answer
106
views
How to Copy Files into Individual Folders in Synapse Analytics
I originally had files in my silver folder. I was using notebooks to make transformations to these files and then overwriting the original files with the transformed ones. After speaking to a Synapse ...
- 83
0
votes
1
answer
95
views
How to Overwrite a Parquet File in the Same Location Using PySpark
I'm working with PySpark within Synapse notebooks and I need to load a Parquet file into a DataFrame, apply some transformations (e.g., renaming columns), and then save the modified DataFrame back to ...
- 83
0
votes
1
answer
54
views
DataFrame to filter rows having Special Characters
I have a DataFrame having around 50K to 100K rows
This DataFrame has around 4 columns.
We need to filter the DataFrame to discard any special character rows
199 Central Avenue
1664 O'block Road
1630 ...
- 51
0
votes
1
answer
80
views
Databricks SQL Query for finding rows having special characters and discard those rows
As part of Databricks, We would like to filter rows having special characters in the columns.
Let's say we have a Table with data like:
Table1 has
Col1
199 Central Avenue
1664 O'block Road
1630 Hahn'...
- 51
1
vote
1
answer
57
views
Using spaCy in Microsoft Fabric
Our organisation is trailing MS Fabric, and i'm trying out Notebooks. I've managed to create an environment and brought in NLTK and spaCy libraries. However, typically when you use spaCy you also ...
- 217
0
votes
1
answer
36
views
Storing a View in a Dataframe in Azure Synapse Notebook
I'm attempting to store the top 100 rows of a view from an SQL database to a dataframe using Synapse Notebook. I have tried using this code:
%%pyspark
df = spark.sql("SELECT DataType, TableName, ...
- 83
2
votes
1
answer
189
views
How to use UAMI authentication in Azure synapse spark notebook
We can authenticate ADLS Gen2 source using User-Assigned Managed Identity (UAMI) to authenticate to an ADLS Gen2 source successfully in an Azure Synapse Pipeline. However, we are unable to connect it ...
- 21
0
votes
2
answers
60
views
ModuleNotFoundError: No module named 'azure.mgmt.eventhub' on synapse notebook even though it is in requirements.txt
We have the azure-management library defined in the requirements.txt:
azure-mgmt-eventhub==11.0.0
But when trying to import the library it is not found:
This has been intermittent for other ...
- 61.9k