Must Have Skills :

Azure Data Lake design and implementation experience

Experience modeling and building out star schema on Cloud Data Lake for BI use cases.

Databricks Data Lake experience

Spark

SQL

Team Coaching skills.

Strong architect who can hold their own and suggest solutions.

AWS/Azure or GCP data lake experience

Streaming data pipelines with kafka for real time needs

Polished consulting skills

Callie Dump Notes From Call:

They have teams that currently work with Azure and data so not completely in the dark here.

The place they need help with is with a data warehouse that is pulling data from a lot of different areas across the organization. 5-6 different analytics pipeline. Feed these to one data warehouse that will be used as a single source of truth.

Data analytics that will have to be aligned.

How will members align in different data pipelines and then transfer to data warehouse.

Azure type structure. Not relational. Utilize star scheema to do rollup queries.

Need help on Data warehousing approach on azure then drive maintenance of star scheema in that environment.

Doing lots of fits and starts and going down false trails. Lots of trial and error.

Wants someone with previous experience in this area. Need super strong resource.

Using looking at synaps on azure or possibly something else.

Medallion type architecture. Gold/silver/bronze.

Not looking at using sql clusters in synapse. This doesn't work for them.

Using spark based ETL. Serverless compute running against deltalake.

Databricks?? TBD they are looking into this. Would love someone to be able to recommend yes or no.

Need someone with strong data warehouse background.

How does everything fit into reporting and data warehouse structure. Wants to collaborate with this person to help decide someo of these things.

Needs someone to speak up and figure out problems and suggest path forward. Be able to explain why and why not.

Data is coming in as parka... some coming in from MicroStrategy. Need landing zone somehow.

Microservice oriented architecture. Reports need to be real team. Publishing out to kafka from microservices that will feed into datawarehouse. Daily basis of this. Not all file based or parka.

Clinical encounters are going to be very important to react to quickly.

They already have kafka but they would like to help ramp this up. Need help with how the kafka is used with datawarehouse. Make sure it lands in warehosue without losing it.

How much data is historic vs day to day? Some monthly some daily. Gigabites of data.

250 terabytes of data by the time this is done. It will be expanding in scope as well.

100 million members in data store.

Updates?? Everything is pretty much new updates. For the most part they do the data work in the intake process and what is a delta?

There will be a silver layer in the warehouse where there will not be any differentiators. Historical data at this level.

Bronze:

Some tracking of history but wants to expand this to be able to do snapshots over time.

Lots of reporting is tableau based but its very manual so they want to move towards powerbi here. They will likely use both for reporting engines.

Data that needs to be isolated by payer as much as possible. Need to be able to show this data trail. Cant have payer data crossover. 100's of payers. They focus mainly on large payers now and then into midsized.

3-4 million providers. 80 million total though. Trying to narrow this down.

Healthcare experience and star scheema experience is really important. Need help with process and strategy.

Data engineering SME that can build out work.

Will start with proof of concept.

Required Skills : 1. Azure Data Lake design and implementation experience 2. Experience modeling and building out star schema on Cloud Data Lake for BI use cases. 3. Databricks Data Lake experience 4. Spark 5. SQL 6. Team Coaching skills. 7. Strong architect who can hold their own and suggest solutions. 8. AWS/Azure or GCP data lake experience 9. Streaming data pipelines with kafka for real time needs 10. Polished consulting skills
Basic Qualification :
Additional Skills :
Background Check :Yes
Drug Screen :Yes
Notes :
Selling points for candidate :
Project Verification Info :
Candidate must be your W2 Employee :Yes
Exclusive to Apex :No
Face to face interview required :No
Candidate must be local :No
Candidate must be authorized to work without sponsorship ::No
Interview times set : :No
Type of project :Architecture/Design
Master Job Title :Architect: Apps (Other)
Branch Code :Minneapolis