Data Pipelines

BigQuery as a Data Warehouse for Cloudflare Firewall Events

Big Data, BigQuery, Cloud Firewalls, Cloud Run, CloudFlare, Cybersecurity, Data Pipelines, Google Cloud, Microservices, SecOps, Web Application Firewall

If you have created some custom rules for your zone in the Cloudflare WAF, you can easily see the IP addresses and URI paths which have triggered those rules by simply selecting the rule from the Cloudflare dashboard. A key…

Deploy n8n with Docker Compose for Automating AI Workflows

AI Infrastructure

AI as a Service, Container Orchestration, Data Pipelines, Docker, Locally Hosted AI, n8n, Open Source AI, Retrieval Augmented Generation, Self Hosted AI, Vector Databases

n8n is a data integration & workflow automation tool that can be self hosted on your own infrastructure. Compared to cloud-based automation platforms such as Zapier, Integromat, Make, or the cloud version of n8n, self hosting has two major benefits.…

AI Document Data Pipelines with S3 or Azure Blob Storage

AI Infrastructure

Airbyte, AWS, Azure, BionicGPT, Data Pipelines, Generative AI, NextCloud, Object Storage, Open Source AI, Retrieval Augmented Generation, Vector Databases

Document data pipelines allow for the creation of automated workflows where an embedding model extracts text from documents added to a share and converts them into vector values that AI/ML applications can use for semantic search and as context for…

ELT Data Pipelines with Airbyte & BionicGPT for AI RAG

AI Infrastructure

BionicGPT, Data Pipelines, Generative AI, Kubernetes, Open Source AI, PostgreSQL, RabbitMQ, Retrieval Augmented Generation, Vector Databases

BionicGPT is an open source AI chat platform with features for teams to upload and share datasets for retrieval augmented generation using large language models. With role based access control, it is designed with a hierarchical user structure where team…