Cloud & Enterprise Nodes

Integrate with AWS, Azure, GCP, and enterprise platforms

Cloud & Enterprise Nodes

Integrate with major cloud providers to build scalable, enterprise-grade automation. These nodes connect you to AWS, Azure, GCP, and orchestrate complex enterprise workflows.

Cloud Integration Node

When to use: When you need to interact with AWS, Azure, or GCP services—upload files, invoke functions, query cloud databases, etc.

The Cloud Integration Node provides access to cloud platform services. Configure the provider and service, then perform the operation you need.

Configuration

Configuration:
  provider: aws | azure | gcp
  service: s3 | lambda | ec2 | blob | functions | storage
  operation: service-specific

Example 1: Upload File to AWS S3

When to use: Store files in the cloud for archiving, backups, or distribution.

Incoming data:

{
  "file_content": "base64-encoded-file-data",
  "filename": "invoice_2025_02_10.pdf",
  "bucket": "company-documents"
}

Cloud Integration Configuration:

provider: aws
service: s3
operation: put_object
bucket: "{{ input.bucket }}"
key: "invoices/{{ input.filename }}"
body: "{{ input.file_content }}"
content_type: "application/pdf"

Workflow:

Generate or receive a file
Cloud Integration uploads to S3
Respond with download URL: https://s3.amazonaws.com/company-documents/invoices/invoice_2025_02_10.pdf

Tip: Use S3 for long-term storage, backups, and file distribution. Much cheaper than storing files in a database!

Example 2: Invoke AWS Lambda Function

When to use: Run serverless code—image processing, document analysis, complex calculations, etc.

Incoming data:

{
  "image_url": "https://example.com/photo.jpg",
  "resize_width": 300
}

Cloud Integration Configuration:

provider: aws
service: lambda
operation: invoke
function_name: "image-resizer"
payload: |
  {
    "image_url": "{{ input.image_url }}",
    "width": {{ input.resize_width }},
    "format": "webp"
  }
invocation_type: RequestResponse

Response from Lambda:

{
  "resized_image_url": "https://s3.amazonaws.com/processed-images/photo-300w.webp",
  "original_size": 2457600,
  "resized_size": 145920,
  "compression_ratio": 0.94
}

Example 3: Read Files from Cloud Storage

When to use: Process files stored in the cloud.

Cloud Integration Configuration (Azure):

provider: azure
service: blob
operation: get_blob
container: "input-files"
blob_name: "{{ input.filename }}"

Returns the file content, which you can then:

Parse with Data Parser
Process with AI
Insert into database
Transform and re-upload

Example 4: GCP Cloud Functions

When to use: Run Google Cloud functions for complex processing.

Cloud Integration Configuration:

provider: gcp
service: functions
operation: call
function_name: "process-document"
region: "us-central1"
payload:
  document_url: "{{ input.doc_url }}"
  output_format: "json"

Integration Automation Node

When to use: When you need to coordinate between multiple enterprise systems—ETL (extract, transform, load), orchestration, event-driven integration.

The Integration Automation Node handles complex multi-system workflows—syncing data between systems, triggering workflows based on events, etc.

Configuration

Configuration:
  type: etl | orchestration | events
  source: system-a
  target: system-b

Example 1: ETL (Extract, Transform, Load)

Scenario: Daily sync of customer data from Salesforce → Data Warehouse

Workflow:

Start (scheduled daily)
  ↓
Integration Automation (Extract)
  ├─ Query Salesforce for new/updated customers
  │  Return: 500 customer records
  │
  ↓
Data Transform (Transform)
  ├─ Normalize fields
  ├─ Validate data
  ├─ Enrich with additional fields
  │
  ↓
Integration Automation (Load)
  └─ Insert/update in data warehouse
     Return: 500 records processed

Configuration:

type: etl
source: salesforce
target: data_warehouse
operation: sync_customers

Example 2: Event-Driven Integration

Scenario: New Slack message → Log to database → Notify team

Workflow:

Integration Automation (Listen for Slack events)
  ├─ Event: message_posted
  │
  ↓
Extract data (channel, user, message, timestamp)
  │
  ↓
Database Insert (log message)
  │
  ↓
AI Agent (check if urgent)
  │
  ├─ If urgent → Send email alert
  └─ If normal → Just log

Example 3: Multi-System Orchestration

Scenario: Customer signup → Create account in 3 systems

Workflow:

Start (new customer signup)
  ├─ Split into 3 parallel paths
  │
  ├─ Path 1: Create in Salesforce
  │
  ├─ Path 2: Create in billing system
  │
  ├─ Path 3: Send welcome email
  │
  Merge (wait for all 3 to complete)
  │
  Log: "Customer onboarding complete"

Distributed Execution Node

When to use: When you need to process large workloads in parallel or distribute tasks across multiple workers.

The Distributed Execution Node breaks large jobs into smaller pieces, processes them in parallel, and aggregates results.

Configuration

Configuration:
  distribution_strategy: map-reduce | parallel | batch
  num_workers: 4
  timeout: 300000

Example 1: Distributed Image Processing

Scenario: Process 10,000 images in parallel

Incoming data:

{
  "image_urls": ["url1", "url2", "url3", ... "url10000"],
  "operation": "resize_and_compress"
}

Distributed Execution Configuration:

distribution_strategy: map-reduce
num_workers: 8
timeout: 600000

map_task: |
  For each image:
  - Download
  - Resize to 300x300
  - Compress to JPEG
  - Upload to S3

reduce_task: |
  Aggregate all results
  - Count total processed
  - Report any errors
  - Return summary

Result:

{
  "total_images": 10000,
  "processed": 9998,
  "failed": 2,
  "total_time_minutes": 12,
  "avg_time_per_image_ms": 72
}

Example 2: Distributed Data Analysis

distribution_strategy: parallel
num_workers: 4

task: |
  Analyze customer data in chunks:
  - Worker 1: Analyze customers 1-25000
  - Worker 2: Analyze customers 25001-50000
  - Worker 3: Analyze customers 50001-75000
  - Worker 4: Analyze customers 75001-100000

  Each worker computes:
  - Average lifetime value
  - Churn risk score
  - Product preferences

aggregate: |
  Combine results from all workers:
  - Global statistics
  - Top customer segments
  - Churn risk distribution

Common Cloud Patterns

Pattern 1: File Upload to Cloud Storage

Start (file received)
  ↓
Cloud Integration (upload to S3/GCS/Azure Blob)
  │
  ├─ Success → Generate download link
  │           ↓
  │           Email (send link to user)
  │
  └─ Error → Log error, notify admin

Pattern 2: Serverless Processing Pipeline

Webhook Trigger (image upload)
  ↓
Cloud Integration (trigger Lambda)
  │
  ├─ Lambda processes image
  │
  ↓
Cloud Integration (store result in S3)
  ↓
Database (log processing)
  ↓
Email (notify user)

Pattern 3: Multi-Cloud Data Sync

Start (daily)
  ├─ Path 1: Extract from AWS
  ├─ Path 2: Extract from Azure
  └─ Path 3: Extract from GCP
     ↓
     Merge (combine data)
     ↓
     Data Transform (normalize)
     ↓
     Database (store in data warehouse)

Pattern 4: Parallel Processing with Aggregation

Start (large job)
  ↓
Distributed Execution (split and process)
  │
  ├─ Worker 1 processes chunk 1
  ├─ Worker 2 processes chunk 2
  ├─ Worker 3 processes chunk 3
  └─ Worker 4 processes chunk 4
     ↓
     Merge (collect all results)
     ↓
     Aggregate (combine stats)
     ↓
     Report

Cloud Cost Optimization

AWS

S3: Use lifecycle policies to move old files to Glacier (saves 90%)
Lambda: Set appropriate timeout/memory. More memory = faster = cheaper
EC2: Use spot instances for non-critical workloads

Azure

Blob Storage: Use hot/cool/archive tiers based on access patterns
Functions: Reserve capacity during peak hours

GCP

Cloud Storage: Use class-based storage (standard/nearline/coldline)
Cloud Functions: Optimize runtime and memory allocation

Tip: Always set resource limits and timeouts. Runaway processes cost money!

Enterprise Considerations

Security

Use VPC/Private networks for sensitive workflows
Enable encryption for data in transit and at rest
Regularly rotate credentials stored in DeepChain vault

Compliance

Log all data access for audit trails
Ensure data residency meets legal requirements
Use encryption keys compliant with HIPAA/PCI-DSS if needed

Scalability

Use Distributed Execution for large datasets
Enable caching to reduce API calls
Monitor performance metrics

Next Steps

Transform cloud data? See Data Processing Nodes
Need AI analysis? Check AI & Intelligence Nodes
Store processed data? Visit Database Nodes
Make decisions based on cloud data? Use Control Flow Nodes
Notify teams? Try Communication Nodes