Trigger Cloud Functions Only for Specific GCS Folders

Problem Statement

When building media pipelines on Google Cloud Storage (GCS), a common pattern is:

1. User uploads a video to:

gs://bucket/videos/video.mp4

2. A Cloud Function (or Cloud Run service) generates a thumbnail

3. Thumbnail gets stored at:

gs://bucket/thumbnails/video_thumbnail.png

At first glance, this looks straightforward using a Cloud Storage “object finalized” trigger.

However — this approach introduces a recursive execution loop.

Why Recursive Loops Happen

A typical trigger configuration looks like this:

1
type = google.cloud.storage.object.v1.finalized
2
bucket = bucket-name

This means:

Trigger on ANY object finalized inside the bucket.

So when:

videos/video.mp4

is uploaded → function runs ✅

Then function creates:

thumbnails/video_thumbnail.png

which is ALSO a finalized object → function runs again ❌

This causes:

• duplicate execution
• wasted compute
• unnecessary logging
• potential infinite processing loops

Traditional Workaround (Code-Level Filtering)

Most developers solve this by filtering inside the function:

Example:

1
if (objectName.startsWith("thumbnails/")) {
2
  return
3
}

Or extension filtering:

1
if (!objectName.endsWith(".mp4")) {
2
  return
3
}

Limitations of this approach

Although this prevents recursion logic-wise:

• Function still executes twice
• Cold starts still occur
• Logging still happens
• Invocation billing still applies

So this solution reduces impact but does not eliminate the problem.

Better Solution: Eventarc Path Pattern Filtering via Cloud Audit Logs

Instead of using:

1
google.cloud.storage.object.v1.finalized

Use:

1
google.cloud.audit.log.v1.written

with path pattern filtering.

This allows triggering functions ONLY when objects are created under:

1
videos/

and NOT elsewhere.

Example filter:

1
projects/_/buckets/bucket-name/objects/videos/*

Now the thumbnail creation event does NOT retrigger execution.

Problem solved at the trigger level.

Architecture Overview

graph TD
    User(["**User**<br/>uploads video"])
 
    GCS_Videos[("**GCS Bucket**<br/>/videos")]
 
    AuditLogs["**Cloud Audit Logs**<br/>storage.objects.create"]
 
    Eventarc{"**Eventarc Trigger**<br/>path filter: /videos/**"}
 
    Compute["**Cloud Run / Cloud Function**<br/>generates thumbnail"]
 
    Thumb(["**Thumbnail**<br/>ready"])
 
    GCS_Thumbs[("**GCS Bucket**<br/>/thumbnails")]
 
    User -->|"① HTTPS upload"| GCS_Videos
    GCS_Videos -.->|"② emits audit log event"| AuditLogs
    AuditLogs -->|"③ forwards event"| Eventarc
    Eventarc -->|"④ matches /videos/** only — /thumbnails writes ignored"| Compute
    Compute -->|"⑤ processes frame"| Thumb
    Thumb -->|"⑥ stores output"| GCS_Thumbs
 
    style GCS_Videos  fill:#1A73E8,stroke:#0D47A1,color:#fff
    style GCS_Thumbs  fill:#1A73E8,stroke:#0D47A1,color:#fff
    style AuditLogs   fill:#EA4335,stroke:#B31412,color:#fff
    style Eventarc    fill:#F9AB00,stroke:#E37400,color:#1a1a1a
    style Compute     fill:#34A853,stroke:#1E7E34,color:#fff
    style Thumb       fill:#34A853,stroke:#1E7E34,color:#fff
    style User        fill:#E8F0FE,stroke:#4285F4,color:#1a1a1a

No recursive loop.

Single invocation per upload.

Step 1: Enable Cloud Storage Data Write Audit Logs

Open:

1
IAM & Admin → Audit Logs

Select:

1
Cloud Storage

Enable:

1
Data Write

Save.

This allows Eventarc to receive object creation events.

Step 2: Create Eventarc Trigger (CLI)

Example:

gcloud eventarc triggers create videos-folder-trigger \
  --location=${REGION} \
  --destination-run-service=thumbnail-service \
  --destination-run-region=${REGION} \
  --event-filters="type=google.cloud.audit.log.v1.written" \
  --event-filters="serviceName=storage.googleapis.com" \
  --event-filters="methodName=storage.objects.create" \
  --event-filters-path-pattern="resourceName=projects/_/buckets/bucket-name/objects/videos/*" \
  --service-account=SERVICE_ACCOUNT

Now only objects inside:

1
/videos/

trigger execution.

Step 3: Create Eventarc Trigger (Console UI)

Navigate to:

1
Eventarc → Triggers → Create Trigger

Choose:

1
Event Provider: Cloud Audit Logs
2
Event Type: Written
3
Service: storage.googleapis.com
4
Method: storage.objects.create

Then configure path pattern:

1
resourceName = projects/_/buckets/bucket-name/objects/videos/*

Select destination service.

Save trigger.

Step 4: Minimal Debugging Function Example

Example Node.js handler:

1
const functions = require('@google-cloud/functions-framework');
2

3
functions.cloudEvent('thumbnailTrigger', (event) => {
4
  const payload = event.data?.protoPayload || {};
5

6
  console.log("eventId:", event.id);
7
  console.log("method:", payload.methodName);
8
  console.log("resource:", payload.resourceName);
9
});

Useful for verifying correct trigger behavior.

Supporting Multiple Video Formats

Recommended trigger pattern:

1
projects/_/buckets/bucket-name/objects/videos/*

Then filter formats inside code:

1
const allowed = ['.mp4', '.mkv', '.mov', '.webm'];
2

3
if (!allowed.some(ext => objectName.endsWith(ext))) {
4
  return
5
}

This avoids needing multiple triggers.

Why This Approach Is Better Than Storage Finalized Trigger

Cost Considerations

Cloud Audit Logs provides:

1
50 GB free logging per month

Typical storage object creation event size:

1
~1–3 KB

Example workload:

1
100 uploads/month

Estimated usage:

1
< 1 MB/month

Effectively free.

Production Best Practices

Recommended trigger pattern:

1
projects/_/buckets/bucket-name/objects/videos/**

Recommended safeguards inside function:

1
ignore folder placeholders
2
ignore unsupported extensions
3
log object metadata
4
handle retries safely

Example:

1
if (objectName.endsWith('/')) return

Final Result

Using Eventarc Audit Log path filtering:

• ✔ prevents recursive execution loops
• ✔ reduces unnecessary invocations
• ✔ supports folder-level triggers
• ✔ scales cleanly for media pipelines
• ✔ works across multiple file formats
• ✔ production-ready architecture

This is the recommended strategy for building thumbnail generation pipelines on Google Cloud Storage.