kfarr/vid2scene
Video to 3D Gaussian splat (.ply). Thin Cog wrapper around the samuelm2/vid2scene standalone reconstruction pipeline (frame extraction -> hloc+glomap SfM -> gsplat training).
Run kfarr/vid2scene with an API
Use one of our client libraries to get started quickly. Clicking on a library will take you to the Playground tab where you can tweak different inputs, see the results, and copy the corresponding code to use in your own project.
Input schema
The fields you can use to run this model with an API. If you don't give a value for a field its default value will be used.
| Field | Type | Default value | Description |
|---|---|---|---|
| video |
string
|
Source video. A slow, steady orbit around a static subject works best.
|
|
| reconstruction_method |
None
|
glomap
|
Structure-from-Motion method. glomap (default): fast global SfM, recommended. colmap: slower incremental, sometimes more robust on difficult captures.
|
| target_framecount |
integer
|
600
Min: 30 Max: 2000 |
Target number of frames to sample from the video.
|
| training_num_steps |
integer
|
30000
Min: 2000 Max: 30000 |
gsplat training steps. Fewer is faster but lower quality.
|
| training_max_num_gaussians |
integer
|
500000
Min: 100000 Max: 10000000 |
Cap on the number of Gaussians: the primary control on output detail and .ply size (~164 bytes/Gaussian, so 5M is ~820 MB). Higher means more detail and a larger file. The 500k default is a sensible mid-range; raise it for large or complex scenes. Warning: a high cap increases training time and may cause the job to time out depending on machine capacity.
|
| equirectangular |
boolean
|
False
|
Treat the input as 360/equirectangular video.
|
| use_background_sphere |
boolean
|
True
|
Add a background sphere for distant/sky content. Helps outdoor or 360 captures where the background is far away.
|
| remove_background |
boolean
|
False
|
Remove the background from each frame before reconstruction (InSPyReNet). Good for isolating a single object; leave off for scenes/environments where you want the surroundings.
|
| apriltag_size_meters |
number
|
0
Max: 10 |
Optional. If an AprilTag of known physical size (in meters) is visible in the video, set it to scale the reconstruction to real-world units. Leave at 0 to skip AprilTag detection entirely.
|
| resolution |
integer
|
1920
Min: 512 Max: 3840 |
Maximum long-edge resolution (pixels) for extracted frames; the video is downscaled to this, never upscaled. Lower is faster and cheaper at reduced detail.
|
| debug |
boolean
|
False
|
Log container memory/disk usage every 15s plus resource limits at startup. For diagnosing failed or killed jobs; adds noise to logs.
|
{
"type": "object",
"title": "Input",
"required": [
"video"
],
"properties": {
"debug": {
"type": "boolean",
"title": "Debug",
"default": false,
"x-order": 10,
"description": "Log container memory/disk usage every 15s plus resource limits at startup. For diagnosing failed or killed jobs; adds noise to logs."
},
"video": {
"type": "string",
"title": "Video",
"format": "uri",
"x-order": 0,
"description": "Source video. A slow, steady orbit around a static subject works best."
},
"resolution": {
"type": "integer",
"title": "Resolution",
"default": 1920,
"maximum": 3840,
"minimum": 512,
"x-order": 9,
"description": "Maximum long-edge resolution (pixels) for extracted frames; the video is downscaled to this, never upscaled. Lower is faster and cheaper at reduced detail."
},
"equirectangular": {
"type": "boolean",
"title": "Equirectangular",
"default": false,
"x-order": 5,
"description": "Treat the input as 360/equirectangular video."
},
"remove_background": {
"type": "boolean",
"title": "Remove Background",
"default": false,
"x-order": 7,
"description": "Remove the background from each frame before reconstruction (InSPyReNet). Good for isolating a single object; leave off for scenes/environments where you want the surroundings."
},
"target_framecount": {
"type": "integer",
"title": "Target Framecount",
"default": 600,
"maximum": 2000,
"minimum": 30,
"x-order": 2,
"description": "Target number of frames to sample from the video."
},
"training_num_steps": {
"type": "integer",
"title": "Training Num Steps",
"default": 30000,
"maximum": 30000,
"minimum": 2000,
"x-order": 3,
"description": "gsplat training steps. Fewer is faster but lower quality."
},
"apriltag_size_meters": {
"type": "number",
"title": "Apriltag Size Meters",
"default": 0,
"maximum": 10,
"minimum": 0,
"x-order": 8,
"description": "Optional. If an AprilTag of known physical size (in meters) is visible in the video, set it to scale the reconstruction to real-world units. Leave at 0 to skip AprilTag detection entirely."
},
"reconstruction_method": {
"enum": [
"glomap",
"colmap"
],
"type": "string",
"title": "reconstruction_method",
"description": "Structure-from-Motion method. glomap (default): fast global SfM, recommended. colmap: slower incremental, sometimes more robust on difficult captures.",
"default": "glomap",
"x-order": 1
},
"use_background_sphere": {
"type": "boolean",
"title": "Use Background Sphere",
"default": true,
"x-order": 6,
"description": "Add a background sphere for distant/sky content. Helps outdoor or 360 captures where the background is far away."
},
"training_max_num_gaussians": {
"type": "integer",
"title": "Training Max Num Gaussians",
"default": 500000,
"maximum": 10000000,
"minimum": 100000,
"x-order": 4,
"description": "Cap on the number of Gaussians: the primary control on output detail and .ply size (~164 bytes/Gaussian, so 5M is ~820 MB). Higher means more detail and a larger file. The 500k default is a sensible mid-range; raise it for large or complex scenes. Warning: a high cap increases training time and may cause the job to time out depending on machine capacity."
}
}
}
Output schema
The shape of the response you’ll get when you run this model with an API.
Schema
{
"type": "string",
"title": "Output",
"format": "uri"
}