jessica claire 100 montgomery st 10th floor 555 4321000 resumesampleexamplecom summary dynamic motivated professional around 7 years experience big data engineer expertise designing data intensive applications using hadoop ecosystem big data analytical cloud data engineering data warehouse data mart data visualization reporting data quality solutions indepth knowledge hadoop architecture components like yarn hdfs name node data node job tracker application master resource manager task tracker map reduce programming paradigm extensive experience hadoop led development enterprise level solutions utilizing hadoop components apache spark mapreduce hdfs sqoop pig hive hbase oozie flume nifi kafka zookeeper yarn profound experience performing data ingestion data processing transformations enrichment aggregations strong knowledge architecture distributed systems parallel processing indepth understanding mapreduce programming paradigm spark execution framework experienced spark improving performance optimization existing algorithms hadoop using spark context sparksql dataframe api spark streaming mllib pair rdds worked explicitly pyspark scala handled ingestion data different data sources hdfs using sqoop flume perform transformations using hive map reduce loading data hdfs managed sqoop jobs incremental load populate hive external tables experience importing streaming data hdfs using flume sources flume sinks transforming data using flume interceptors experience oozie workflow scheduler manage hadoop jobs direct acyclic graph dag actions control flows implemented security requirements hadoop integrating kerberos authentication infrastructure kdc server setup creating realm domain managing experience partitions bucketing concepts hive designed managed external tables hive optimize performance experience different file formats like avro parquet orc json xml expertise creating debugging scheduling monitoring jobs using airflow oozie experienced using common operators airflow python operator bash operator google cloud storage download operator google cloud storage object sensor handson experience handling database issues connections sql nosql databases mongodb hbase cassandra sql server postgresql created java apps handle data mongodb hbase used phoenix create sql layer hbase experience designing creating rdbms tables views user created data types indexes stored procedures cursors triggers transactions expert designing etl data flows using creating mappingsworkflows extract data sql server data migration transformation oracleaccessexcel sheets using sql server ssis expert designing parallel jobs using various stages like join merge lookup remove duplicates filter dataset lookup file set complex flat file modify aggregator xml handson experience amazon ec2 amazon 53 amazon rds vpc iam amazon elastic load balancing auto scaling cloudwatch sns ses sqs lambda emr services aws family created configured new batch job denodo scheduler email notification capabilities implemented cluster setting multiple denodo node created load balance improving performance activity instantiated created maintained cicd continuous integration deployment pipelines apply automation environments applications worked various automation tools like git terraform ansible experienced fact dimensional modeling star schema snowflake schema transactional modeling scd slowly changing dimension experienced json based restful web services xmlqml based soap web services also worked various applications using python integrated ides like sublime text pycharm efficient cloud engineer years experience assembling cloud infrastructure utilizes strong managerial skills negotiating vendors coordinating tasks team members implements best practices create cloud functions applications databases skills big data technologies hadoop mapreduce hdfs sqoop pig hive hbase oozie flume nifi kafka zookeeper yarn apache spark mahout sparklib databases oracle mysql sql server mongodb cassandra dynamodb postgresql teradata cosmos operating systems windows 78xp20082012 ubuntu linux macos network security kerberos database modelling dimension modeling er modeling star schema modeling snowflake modeling programming python pyspark scala java c c shell script monitoring tool apache airflow perl script sql cloud technologies aws microsoft azure frameworks django rest framework mvc hortonworks tools pycharm eclipse visual studio sqlplus sql developer toad sql navigator query analyzer sql server management studio sql assistance eclipse postman versioning tools svn git github experience aws data engineer 012022 022022 deloitte ann arbor mi visualizationreporting tableau ggplot2 matplotlib ssrs power bi machine learning techniques linear logistic regression classification regression trees random forest associative rules nlp clustering designed setup enterprise data lake provide support various uses cases including analytics processing storing reporting voluminous rapidly changing data responsible maintaining quality reference data source performing operations cleaning transformation ensuring integrity relational environment working closely stakeholders solution architect designed developed security framework provide fine grained access objects aws s3 using aws lambda dynamodb set worked kerberos authentication principals establish secure network communication cluster testing hdfs hive pig mapreduce access cluster new users performed endtoend architecture implementation assessment various aws services like amazon emr redshift 53 implemented machine learning algorithms using python predict quantity user might want order specific item automatically suggest using kinesis firehose 53 data lake used aws emr transform move large amounts data aws data stores databases amazon simple storage service amazon 53 amazon dynamodb used spark sql scala amp python interface automatically converts rdd case classes schema rdd import data different sources like hdfshbase spark rdd perform computations using pyspark generate output response creating lambda functions boto3 deregister unused amis application regions reduce cost ec2 resources importing exporting database using sql server integrations services ssis data transformation services dts packages coded teradata bteq scripts load transform data fix defects like scd 2 date chaining cleaning duplicates developed reusable framework leveraged future migrations automates etl rdbms systems data lake utilizing spark data sources hive data objects conducted data blending data preparation using alteryx sql tableau consumption publishing data sources tableau server developed kibana dashboards based log stash data integrated different source target systems elasticsearch near real time log analysis monitoring end end transactions implemented aws step functions automate orchestrate amazon sagemaker related tasks publishing data 53 training ml model deploying prediction integrated apache airflow aws monitor multistage ml workflows tasks running amazon sagemaker environment aws emr 53 rds redshift lambda boto3 dynamodb amazon sagemaker apache spark hbase apache kafka hive sqoop map reduce snowflake apache pig python ssrs tableau assessed organization technology infrastructure managed cloud migration process configured computing networking security systems within cloud environment implemented cloud policies managed technology requests maintained service availability data engineer 012016 112019 honeywell huntsville al worked azure data factory integrate data onprem sql cassandra cloud blob storage azure sql db applied transformations load back azure synapse managed configured scheduled resources across cluster using azure kubernetes service monitored spark cluster using log analytics ambari web ui transitioned log storage cassandra azure sql datawarehouse improved query performance involved developing data ingestion pipelines azure hdinsight spark cluster using azure data factory spark sql also worked cosmos db sql api mongo api develop dashboards visualizations help business users analyze data well providing data insight upper management focus microsoft products like sql server reporting services ssrs power bi performed migration large data sets databricks spark create administer cluster load data configure data pipelines loading data adls gen2 databricks using adf pipelines created various pipelines load data azure data lake staging sqldb followed azure sql db created databrick notebooks streamline curate data various business use cases also mounted blob storage databrick utilized azure logic apps build workflows schedule automate batch jobs integrating apps adf pipelines services like http requests email triggers etc worked extensively azure data factory including data transformations integration runtimes azure key vaults triggers migrating data factory pipelines higher environments using arm templates ingested data minibatches performs rdd transformations minibatches data using spark streaming perform streaming analytics data bricks environment azure sql dw databrick azure synapse cosmos db adf ssrs power bi azure data lake arm azure hdinsight blob storage apache spark adept troubleshooting identifying current issues providing effective solutions managed performance monitoring tuning identifying repairing issues within database realm identified key use cases associated reference architectures market segments industry verticals designed surveys opinion polls assessment tools collect data tested validated reformulated models foster accurate prediction outcomes created graphs charts detailing data analysis results recommended data analysis tools address business issues developed new functions applications conduct analyses cleaned manipulated raw data collaborated solution architects define database analytics engagement strategies operational territories key accounts big data engineer hadoop developer 102013 122015 infosys ltd irwindale ca ansibledenododenodocloudwatchavropysparkpysparkpysparkmllibdataframenifinifi interacted business partners business analysts product owner understand requirements build scalable distributed data solutions using hadoop ecosystem developed spark streaming programs process near real time data kafka process data stateless state full transformations worked hive data warehouse infrastructurecreating tables data distribution implementing partitioning bucketing writing optimizing hql queries built implemented automated procedures split large files smaller batches data facilitate ftp transfer reduced 60 execution time worked developing etl processes data stage open studio load data multiple data sources hdfs using flume sqoop performed structural modifications using map reduce hive developing spark scripts udfs using spark dsl spark sql query data aggregation querying writing data back rdbms sqoop written multiple mapreduce jobs using java api pig hive data extraction transformation aggregationavrom multiple file formats including parquet avro xml json csv orcfile compressed file formats codecs like gzip snappy lzo strong understanding partitioning bucketing concepts hive designed managed external tables hive optimize performance developed pig udfs manipulating data according business requirements also worked developing custom pig loaders developing etl pipelines data warehouse using combination python snowflakes snowsql writing sql queries snowflake experience report writing using sql server reporting services ssrs creating various types reports like drill parameterized cascading conditional table matrix chart sub reports used datastax spark connector used store data cassandra database get data cassandra database wrote oozie scripts setting workflow using apache oozie workflow engine managing scheduling hadoop jobs worked implementation log producer scala watches application logs transform incremental log sends kafka zookeeper based log collection platform used hive analyze data ingested hbase using hivehbase integration compute various metrics reporting dashboard transformed tpyspark using aws glue dynamic frames pyspark cataloged transformed data using crawlers scheduled job crawler using workflow feature worked installing cluster commissioning decommissioning data node name node recovery capacity planning slots configuration developed data pipeline programs spark scala apis data aggregations hive formatting data json visualization generatipyspark environment aws cassandra pyspark apache spark hbase apache kafka hive sqoop flume apache oozie zookeeper etl udf map reduce snowflake apache pig python java ssrs onfidential developed implemented hadoop code observing coding standards optimized tuned hadoop environments modified hardware meet prescribed performance thresholds developed new functions applications conduct analyses created graphs charts detailing data analysis results tested validated reformulated models foster accurate prediction outcomes python developer 092012 102013 fiserv city state aws 53 ec2 lambda ebs iam datadog cloudtrail cli ansible mysql python git jenkins dynamodb cloud watch docker kubernetes leveraged open communication collective decisionmaking thorough reviews create performant scalable systems worked serverside frontend technologies leveraged common design patterns code dynamic userfriendly systems implemented new api routes architected new orm structures refactored code boost application performance harnessed version control tools coordinate project development individual code submissions introduced cloudbased technologies python development expand onpremise deployment options wrote clear clean code use projects resolved customer issues establishing workarounds solutions debug create defect fixes education training master science data engineering 052023 george mason university fairfax va bachelor engineering computer networking telecommunications 052019 university mumbai india