Edward Weber accepted your invitation to join Slack βΒ take a second to say hello.
hey can you come into the integrations cave for a minute. Josh has a question for storing env variables into a jupyter notebook
Hey @Ahsan sorry I went into lunch when you sent this. You good or still need help?
Me and Dustin have a question for the current benchmark file. If you have a moment we would appreciate your help.
kk i'll come thru
Just for coding reference here is the bigquery table that my part of the code does:
These are the two functions that runs the code: The bottom one adds the .xlsx files into storage and the top code will take that data and insert / update the bigquery tables
Also these are the two pub/subs to run them too. They match the name of the cloud functions:
The only part that isn't automated is I have to run your guy's code first and email it to myself which is the Flatirons Benchmark.xlsx From there storagebucket code knows to extra the latest ones and insert them into the cloud storage bucket and the BigQueryTables function will take those .csv and make it into a dataframe and insert it into BigQuery (the two tables). And that's it
Yea we will fix that portion in the refactor to be automated, or should I say you will fix that hehe
Zach Wilson data engineer
Data engineering up
Thanks ! :catroombaexceptionally_fast:
Also when you run the benchmark in the am, drop it in the main channel for all
Also probably donβt tell James you will start being pulled in for the work from engineering. Rather not tip off ward. Also told McFadden to pull you as my backfill.
I told them earlier ago that I will still be helping as I transfer it over to Dustin
So once it is fully built as one function then I wouldn't need to step in (unless needed)
This weekend I was learning about power query, power pivot, dax, etc for BI skills and I can tell why you like excel compared to google sheets now lol.
Being doing sheets wrong all this time π€£
Well not wrong but not to it's full protentional
Sheets can do the same thing but you need to know the functions like the back of your hand, PQ is much more drag and drop so novice users become power users quickly
Here is the powerquery for it just in case you want it as well
in github do you have admin rights for the benchmark script repo?
My guess I do not but not 100% sure
I will make my way over to the cave
Okay let me know when you are available ^
That works with me :meow_party:
here is the table
SELECT **
FROM `shield-legal-bi.sl_dashboard.FlatIrons_Financials_AT`;
Thanks ! Okay I shall see if I can find that table and connect to it. If I can then I'll bring the laptop over so we can build the dashboard
Ah I looked it up. The reason we couldn't see any of the "bigquery tables" even though we got looker enterprise connected to bigquery database is since I need to look up how to write the LookML first. Then it will add it to a place called "explorer" which then we can use that to pull the data for our visuals.
I will let you know once I got the LookML set up so that we can then create the visuals / tables, etc
Updates for huddle
Enroute, if I donβt make it there in time. Updates, Edward did financials for fl. Emailed nancy. Working on the fl financials dashboard for looker. Working on the modeling
example of multiple case when statement
``-- Create the temporary table
CREATE TEMP TABLE temp_standardized AS
SELECT
ColumnB,
CASE
WHEN ColumnB = 'toyota' OR ColumnB = 'Toyota' THEN 'Toyota'
WHEN ColumnB = 'nissan' OR ColumnB = 'Nissan' THEN 'Nissan'
ELSE 'Other'
END AS Standardized_Make
FROMyourdataset.yourtable`;
-- Use the temp table SELECT StandardizedMake, COUNT(**) AS Count FROM tempstandardized GROUP BY Standardized_Make;```
Hey you stated for me to not use the V1 however ther eis nothing populating for Flatirons CL Query
cl wont have anything populate because camp Lejeune is closed
I mean there isn't a query for me to run
It likes you since mine doesn't show anything
I shall just type it in and resave it since my end doesn't display lol
Meet is canceled but may you pop by at my desk for some flatirons stuff
Going to record some loom SOPs and kinda want to pick your brain again
@Ahsan Question for you. After running the rewrite.py file do you normally run the pivot table as a last step or is that before doing the financials. I only ask since I am looking at the SOP and it is in the middle so I wonder for your process if you do that pivot table in the middle or last step
Benchmark -> financial injection -> rewrite.py file
Thanks for checking. I have been working on Ryan's financials stuff so far today sadly lol
Once I finish his stuff then I am going to jump back into flatirons financials
Sadly I just did Ryan's financials today and tomorrow I have to continue doing it. So probably Monday will be my next run with flatirons financials
But how is it with you over there π
Just setting up monthly reporting, once this clears I should be good for the rest of the month
Then you get to kick back and enjoy the rewards huh
Good morning Ahsan. We shall see. I shall ping you if I have any questions but I think I should be good
I still have to finish Ryan's stuff today (Today is apparently last day)
Worked this weekend doing Flatirons stuff and his stuff
Good thing I am still hourly (for last weekend)
Yeah it is because James deleted the table I used to get my talk time so I had to rebuild the SQL code test things, then also work on Flatirons reconciliation and create a email report for cam, and then do another task for ryan, etc all in one day lol
I am going to see if I can "train" someone to do Ryan's stuff because Ryan doesn't want to have to retrain someone to do his stuff but once James finishing building his new table I think that new person should do it or him since that means I will have to rebuild my code again / do different processes which I think he should do it since it is his table lol
I will. Or me and James can take turns or something more fair. Since they didn't want to give him that task because "it will be too much for his plate" but at the same time is it not a lot of work for my plate too? lol
That's why I am creating those SOP videos on how to do Ryan's stuff and from there James can just follow it to the T and if he needs help he can slack Ryan lol