summary skills jessica claire 100 montgomery st 10th floor 555 4321000 resumesampleexamplecom 7 years experience big data cloud engineer expertise designing data intensive applications using hadoop ecosystem big data cloud data engineering data warehouse data mart data visualization reporting experience implementing e2e solutions big data using hadoop framework executed designed big data solutions multiple distribution systems like cloudera cdh3 cdh4 hortonworks expertise big data processing using hadoop hadoop ecosystem map reduce pig spark scala hive sqoop flume hbase cassandra mongo db kafka framework zookeeper oozie storm implementation maintenance etl big data analysis operations adept programming languages like scala apache spark including big data technologies like hadoop hive hbase sqoop oozie zookeeper worked analyzing data using sql hive spark sql data mining data cleansing extensively used python libraries pyspark pytest pymongo oracle pyexcel boto3 psycopg embedpy numpy pandas star base tabulae matplotlib beautiful soup experience developing data pipelines using aws services including ec2 s3 redshift glue lambda functions step functions cloudwatch sns dynamodb sqs designed developed implemented etl processes using iics data integration experience collecting realtime streaming data creating pipeline row data different sources using kafka storing data hdfs nosql using spark good knowledge experience google cloud platform gcp extensive experience sql designing database objects tables functions procedures triggers views ctes experience migrating sql database azure data lake azure data lake analytics azure sql database databricks azure sql data warehouse controlling granting database access migrating onpremises databases azure data lake store using azure data factory experience creating impala views hive tables fast access data experienced erwin data modeling iterations schematic logical physical forward reverseengineering processes proficient knowledge handson experience writing shell scripts linux experience developing mapreduce jobs data cleaning data manipulation required business used hive extensively perform various data analytics required business teams good experience tableau data visualization analysis large data sets drawing various conclusions experience writing code python manipulate data data loads extracts statistical analysis modeling data munging utilized kubernetes docker runtime environment cicd system build test deploy experience working creating running docker images multiple microservices experience etl tools like informatica datastage snowflake wellexperienced normalization denormalization techniques optimum performance relational dimensional database environments created snowflake schemas normalizing dimension tables appropriate creating sub dimension named demographic subset customer dimension cloud ecosystem office suite data analytics warehouse models san technologies sql transactional replications aws azure gcp languages python scala spark sql databases mysql oracle cassandra mongodb tools ide git deep learning quality analysis data mining critical thinking team building team management data management data mining data visualization presentations data analysis amazon web services project management data storage retrieval big data technologies hadoop programming intellij visual studio code jupyter notebook pycharm experience data engineer verizon communications boston 022021 current participated phases software development including requirements gathering business analysis designed developed complex data pipelines using sqoop spark hive developed python scala code data processing analytics using inbuilt libraries used various spark transformations including maptopair filter flatmap groupbykey sortbykey join cogroup union repetition coalesce distinct intersection map partitions map partitions index actions cleansing input data designed data models aws lambda applications analytical reports created scripts read csv json parquet files s3 buckets python run sql operations load aws s3 dynamodb snowflake used aws glue crawler designed staging operational data storage ods environment enterprise data warehouse snowflake including dimension fact table design using kimballs star schema approach unit tested data redshift snowflake developed data transition programs dynamodb aws redshift etl process using aws lambda creating functions python certain events based use cases performed review system specifications related datastage etl related applications created functions aws lambda eventdriven processing built fullservice catalog system using elasticsearch logstash kibana kinesis cloudwatch effective use mapreduce developed airflow dags orchestrate sequential parallel etl jobs experience moving data gcp azure using azure data factor installed configured apache airflow workflow management created workflows python worked pyspark script data encryption using hashing algorithms concepts clientspecified columns implemented continuous delivery pipeline docker github devised plsql stored procedures functions triggers views packages made use indexing aggregation materialized views optimize query performance developed er diagrams modeled transactional databases data warehouses using er studio power designer developed databricks notebooks generate hive create statements data load data table experience developing spark applications using sparksql databricks data extraction transformation aggregation multiple file formats analyzing transforming data uncover insights customer usage patterns build data pipelines airflow gcp etlrelated jobs using different airflow operators developed pyspark code used compare data hdfs s3 wrote reports using tableau desktop extract data analysis using filters based business use case environment spark scala aws etl gcp hadoop hive tabulae python snowflake hdfs hive mapreduce pyspark pig docker github apache spark teradata json databricks postgresql mongodb sql agile windows daca engineer verizon communications denver co 092018 012021 utilized aws services focus big data architect analyticsenterprise data warehouse business intelligence solutions ensure optimal architecture scalability flexibility availability performance provide meaningful valuable information better decisionmaking developed scala scripts udfs using data framessql rdd spark data aggregation queries writing back s3 bucket experience data cleansing data mining wrote compiled executed programs necessary using apache spark scala perform etl jobs ingested data wrote spark applications programs data validation cleansing transformation custom aggregation used spark engine spark sql data analysis provided data scientists analysis prepared scripts automate ingestion process using python scala needed various sources api aws s3 teradata snowflake designed developed spark workflows using scala data pull aws s3 bucket snowflake applying transformations designed implemented etl pipelines various relational data bases data warehouse using apache airflow developed custom etl solution batch processing realtime data ingestion pipeline move data hadoop using python shell script experience gcp dataproc gcs cloud functions big query worked data extraction aggregations consolidation adobe data within aws glue using pyspark implemented spark rdd transformations map business analysis applied actions top transformations installed configured apache airflow automated resulting scripts ensure daily execution production created dag use email operator bash operator spark livy operator execute ec2 created scripts read csv json parquet files s3 buckets python load aws s3 dynamodb snowflake saved hdfs hive gcp realtime data source ingested file streams spark streaming platform data implemented aws lambda functions run scripts response events amazon dynamodb table s3 bucket http requests using amazon api gateway worked snowflake schemas data warehousing processed batch streaming data load pipeline using snow pipe matillion data lake confidential aws s3 bucket google cloud platform gcp google cloud storage big query big table cloud sql pubsub lead sql data profile structured unstructured semistructured data across various sources identify patterns data implement data quality metrics using necessary queries python scripts based source environment aws mapreduce gcp snowflake etl tabulae pig spark scala hive kafka testing framework cucumber python airflow json parquet csv code cloud git daca engineer 012017 082018 verizon communications san jose ca created lot complex stored procedures triggers functions indexes views tsql statements sql server management studio ssms involved designing managing schema objects tables views indexes stored procedures triggers maintained referential integrity using sql server management studio designed etl strategy transfer data source landing staging destination data warehouse using ssis dts data transformation service cleansing messaging data done local database experience handling python spark context writing pyspark programs etl developed ssis packages export data excelaccess sql server automated ssis packages monitored errors using sql server agent job interacted subject matter experts understanding business logic implemented complex business requirements backend using efficient stored procedures flexible functions deploying azure resource manager json templates powershell worked azure suite azure sql database azure data lake azure data factory azure sql data warehouse azure analysis service architect implement medium largescale bi solutions azure using azure data platform services azure data lake data factory data lake analytics stream analytics azure sql dw hdinsight databricks nosql prepared reports using ssrs sql server reporting service based cube local data warehouse declare discrepancies user expectations service efforts involved scheduling subscription reports via subscription report wizard designed developed etl jobs extract data salesforce replica load data mart redshift design set maintain administrator azure analysis service azure sql data warehouse azure data factory azure sql data warehouse deployed manage mlflow libraries azure databricks workspace tracked servers outside databricks worked sql server integration services ssis ssas skills stored procedures triggers used various sources pull data power bi sql server excel oracle sql azure etc also created visualizations using dashboards migrate data traditional database systems azure sql databases propose architectures consider costspend azure develop recommendations rightsize azure data infrastructure environment ms sql server management studio ssms sql server integration service ssis sql server reporting services ssrs ms visual studio 2012 sql server profiler pyohon developer cumming llc chicago il 042014 122016 involved phases sdlc including requirement gathering design analysis testing customer specifications development deployment application developed entire frontend backend modules using python django web framework along views templates djangos view controller templating language create userfriendly interface using mvc architecture responsible creating visually appealing web pages bootstrap html5 used serverside authentication cookiebased session management secure information allow persistence data used python django interface jquery ui manage storage deletion content involved python ood code quality logging monitoring debugging code optimization file handling client data files like txt xlxs file data retrieval storing purpose graphical data representation using python modules like numpy scipy pandas pygal installed using pip command toolkit used matlab implementing algorithms creation user interfaces used pandas api put data time series tabular format east timestamp data manipulation retrieval worked development applications unix environment responsible debugging troubleshooting web application creating restful web services django mvt mysql implemented multithreading module complex networking operations like traceroute smtp mail server web server using python collaborated within team using agile development workflow widely accepted collaboration practices using git made use jira manage workflow project progress time environment python django javascript mysql numpy scipy pandas api pep pip jenkins json git javascript ajax restful webservice mysql pyunit education training master science computer engineering university missouri kansas city kansas city mo 3580 gpa 122022