Hello, I'm Diego Gomes Data Engineer & Software Developer

Specialized in building scalable data pipelines, fraud detection systems, and cloud architectures. Passionate about AI, Data Science, and Engineering Excellence.

Get In Touch View GitHub
data_pipeline.py 
from pyspark.sql import SparkSession
import pandas as pd

def process_fraud_data():
    # Real-time fraud detection
    spark = SparkSession.builder \
        .appName("FraudDetection") \
        .getOrCreate()
    
    return spark.sql("""
        SELECT transaction_id, risk_score
        FROM transactions 
        WHERE risk_score > 0.8
    """)

About Me

I'm a passionate Data Engineer and Software Developer with expertise in building large-scale data processing systems and fraud detection platforms. Currently working at TCS/Itaú, where I develop real-time transaction monitoring systems that process millions of transactions daily.

From crafting elegant web interfaces to architecting robust data pipelines that process millions of transactions in real-time, my journey has been driven by an insatiable curiosity for solving complex problems. Specializing in Python, PySpark, and cloud-native architectures, I transform raw data into actionable intelligence that powers critical business decisions.

Armed with a Bachelor's in Computer and Information Systems Security (Universidade Cruzeiro do Sul) and an MBA in Full Cycle Architecture (Full Cycle), I bridge the gap between cutting-edge technology and strategic business outcomes. Currently expanding my expertise through advanced studies in Applied AI at UFPR and Data Engineering at PUC Minas, I'm at the forefront of the AI revolution.

As a Full Cycle Architecture specialist, I orchestrate the entire development ecosystem—from conceptual design to production deployment. My approach combines deep technical mastery with entrepreneurial vision, having founded and led DG Tech Solutions for 7 years, delivering scalable, secure, and innovative solutions that drive digital transformation.

18+ Years Experience
50+ Projects Completed
4 Academic Degrees & Specializations
Diego Gomes

Professional Experience

Senior Data Engineer

TCS/Itaú December 2022 - Present

Leading fraud investigation team for banking platform, processing real-time transaction data and cross-referencing databases to detect fraud indicators. Developed and optimized PySpark jobs on Databricks to process large-scale data, enhancing visibility into fraudulent activities. Built Flask-based project with interactive Dashboard for analysts. Extensively used PySpark for data processing and built data pipelines orchestrated with Oozie. Integrated and maintained data solutions using MS SQL Server, Snowflake, AWS Redshift, Athena, and Data Lake.

PySpark Databricks Flask Django Vue.js AWS Redshift Snowflake Power BI Tableau Docker Kubernetes

CEO & Founder

DG Tech Solutions January 2015 - December 2022

Founded and managed technology consulting company for 7 years, specializing in data engineering solutions and web development. Led strategic planning, business development, and technical architecture decisions. Managed client relationships, project delivery, and team coordination. Developed custom data processing solutions, web applications, and business intelligence systems for various industries. Successfully grew the company from startup to established consulting firm with multiple enterprise clients.

Python Django Data Engineering Business Intelligence Project Management Strategic Planning Team Leadership Client Relations

Lead Developer & Data Engineer

NT Consult December 2021 - December 2022

Led API integration across project platforms and designed scalable data pipelines using PySpark and Spark SQL within Databricks environment. Developed and maintained new features to enhance system performance. Managed development team, performed systems analysis, and participated in client meetings. Utilized Oozie for workflow orchestration to automate ETL processes running on Hadoop clusters.

PySpark Databricks Spark SQL Django Flask FastAPI Kubernetes Docker GitLab CI/CD PostgreSQL GCP Hadoop

API Developer & Data Engineer

Medway October 2020 - October 2022

API Developer focused on development of Content and Student Management Platform. Supported data engineering tasks by developing PySpark scripts for data transformation and aggregation. Responsibilities included developing new products, adding features, and maintaining the system. Conducted system and database performance analysis. Managed new product development, performed systems analysis, and collaborated within Scrum.

Python PySpark Django PostgreSQL Scrum

FullStack Developer

Bedin January 2018 - December 2019

FullStack Developer responsible for managing internal infrastructure and implementing network enhancements. Provided support for both local and remote users. Administered HFSQL databases and maintained CakePHP application alongside MySQL database. Developed internal web applications focused on company control systems, including cash management, financial flow, sales tracking, and BI solutions using Python and Django.

Python Django CakePHP MySQL HFSQL Business Intelligence

Python Developer

Visionnaire January 2018 - December 2018

Python Developer hired as consultant to finalize essential modules for innovative product in aviation market. Focused on BackEnd development using Django 1.7 and PostgreSQL, ensuring robust and scalable solutions. Worked with AppEngine to deploy and manage application in cloud environment.

Django Python PostgreSQL Google AppEngine

Featured Projects

Functional Front-End Layout

Functional Front-End Layout

Modern responsive front-end layout with advanced CSS Grid and Flexbox techniques, featuring dark theme and smooth animations.

HTML5 CSS3 JavaScript Responsive Design
NYC Taxi Data Analysis Trip Patterns & ML Predictions

NYC Taxi Data Analysis

Big data analysis of NYC taxi trips using PySpark and machine learning for pattern recognition and predictive modeling.

PySpark Python Pandas Jupyter
Eventex Platform

Eventex - Event Platform

Django-based event registration platform with payment integration, user management, and real-time notifications.

Django Python PostgreSQL Heroku
Bee Behavior Analysis ML Pattern Recognition

Bee Behavior Analysis

Data science project analyzing bee behavior patterns using Python, statistical analysis, and machine learning algorithms.

Python Pandas Scikit-learn Matplotlib
Real-time Fraud Detection

Real-time Fraud Detection

Advanced fraud detection system processing millions of transactions daily using machine learning and real-time analytics.

PySpark Kafka AWS Machine Learning
Cloud Data Pipeline AWS & Databricks

Cloud Data Pipeline

Scalable data pipeline architecture on AWS with Databricks, processing terabytes of data for business intelligence.

AWS Databricks Terraform Apache Spark

Technical Skills

Programming Languages

Python JavaScript Node.js Java TypeScript Rust C++ SQL PL/SQL Shell Script

Data Engineering & Analytics

PySpark Apache Spark Databricks Apache Hadoop Apache Oozie Pandas NumPy Scikit-learn TensorFlow Power BI Tableau QuickSight

Cloud & Infrastructure

AWS AWS Redshift AWS Athena Data Lake GCP Docker Kubernetes Terraform GitLab CI/CD Git

Databases

PostgreSQL MySQL MS SQL Server Snowflake HFSQL Oracle MongoDB

Web Development

Django Flask FastAPI Spring Boot React Redux Vue.js Node.js HTML5 CSS3 CakePHP

AI & Machine Learning

Artificial Intelligence Machine Learning Data Analysis Statistical Modeling Predictive Analytics Business Intelligence

Let's Connect

Ready to collaborate?

I'm always interested in discussing new opportunities, innovative projects, and challenging problems in data engineering and software development.

diego.gomes87@gmail.com LinkedIn Profile GitHub Portfolio
diego@portfolio:~$
diego@portfolio:~$ whoami
Data Engineer & Software Developer
diego@portfolio:~$ cat interests.txt
• AI & Machine Learning
• Data Engineering
• Cloud Architecture
• Science Fiction
• Mathematics
diego@portfolio:~$ _