Role: Pyspark Architect
Location: Jersey City, NJ, Plano, TX(Hybrid)
Full Time
Job Description
In the assigned Job Role of Architecture Consultant 2, your Area Of Responsibility will be as below:
⦁ Participate in customer discussions to gather and analyze system requirements for performance, security and scalability.
⦁ Provide suggestions on solution architecture, contribute to tech framework development, support enterprise architecture definition, and drive design creation
⦁ Identify suitable technologies and tools available in the market, and perform a comparative analysis in line with client's business requirement, thereby identifying optimal architectural solution options
⦁ Participate in developing integration strategies based on validation of proof of concept of proposed architectural solutions
⦁ Facilitate a smooth transition by ensuring knowledge transfer and adherence to quality standards, while contributing to the build and deployment automation strategy
⦁ Collaborate with development teams to resolve complex technical issues using solution deployment and roll out methods, during build, deployment and support phases
⦁ Analyze system performance basis architectural solutions implemented and support in identifying potential risks in the system landscape
⦁ Create artifacts of architectural frameworks and technical solutions, thereby enriching team's knowledge repository
Your contribution to the team:
⦁ The ability to analyse, design, and develop scalable and secure architectures.
⦁ A knack for assessing multiple technologies, tools and frameworks to recommend optimal solutions.
⦁ Understanding of integration methodologies, proof-of-concept validation and deployment strategies.
⦁ The ability to document architectural frameworks and solutions.
⦁ Proactive communication with internal teams and clients, ensuring alignment with business objectives.
⦁ A knack to keep with up technical advancements to ensure relevant and effective recommendations.
⦁ The ability to maintain standards and ensure smooth transition through knowledge transfer
Required Skill and Experience
• Experience in technology consulting, enterprise and solutions architecture and architectural frameworks
• Experience in Cloudera Hadoop ecosystem, i.e. Hadoop, Hive, Scala, SPARK, Sqoop, Flume, Kafka, Python, NoSQL database technologies
• Understanding and knowledge of Hadoop architecture and various components such as high availability architecture, experience in workload management, scalability and distributed platform architecture
• Experience with design and implementation of ETL/ELT framework for complex warehouses/marts. Knowledge of large data sets and experience with performance tuning and troubleshooting
• Background in all aspects of software engineering with strong skills in parallel data processing, data flows, REST APIs, JSON, XML, and micro service architecture
Preferred Skill and Experience
• Experience in working in any hyperscalers, AWS, Azure, GCP, Databricks and Snowflake
• Experience in data virtualization/data federation for creating data product (e.g. in Starburst) in heterogenous environment (Hadoop, RDBMS, NoSQL etc.)
• Hands-on development mentality, with a willingness to troubleshoot and solve complex problems
EEO/About Us
Benefits
Along with competitive pay, as a full-time employee you are also eligible for the following benefits:
- Medical/Dental/Vision/Life Insurance
- Long-term/Short-term Disability
- Health and Dependent Care Reimbursement Accounts
- Insurance (Accident, Critical Illness , Hospital Indemnity, Legal)
- 401(k) plan and contributions dependent on salary level
- Paid holidays plus Paid Time Off