How to Use Vector: The Beginner's Guide (First Install, the Sources-Transforms-Sinks Lesson, the VRL Arc, and the First Pipeline)

CodexFuse Team · 2026-11-07T06:31:30.974Z · 23 min read min read

How Do I Use Vector?

Learning how to use vector takes seven moves: install it on your system, write one config file, start it, watch your first events print to the console, reshape them with VRL, run it as a service, and scale it from agent to aggregator. A vector log pipeline is a single Rust process, Vector.dev, that collects logs and metrics from sources, reshapes them with transforms, and routes them to sinks, and this vector.dev tutorial follows the official Vector documentation and the project release pages, with every fact checked as of November 2026 and pinned to the current release, v0.61.0. This guide is about Vector.dev, the observability pipeline maintained by Datadog, not a vector database, not the Anki robot, and not the Vector messaging app.

CodexFuse offers no Vector tool, so this guide teaches the real program on your own machine, not a web version of it. No Facebook promotion applies to this guide. Vector is the observability-pipeline lane opener in our dev-tools cluster, and the container route is covered in our Docker beginner guide, which you should read first if Docker is new to you.

1. Install Vector with the one-line script or your package manager.
2. Write a vector.yaml config with one source, one transform, and one sink.
3. Start Vector in the foreground and watch events print to the console.
4. Reshape a field with a remap transform written in VRL.
5. Move the config to /etc/vector/vector.yaml and run Vector as a service.
6. Add a disk buffer so a sink outage never loses events.
7. Decide whether each host runs an agent or forwards to an aggregator.

What Is Vector, Exactly?

Vector is a lightweight, ultra-fast tool for building observability pipelines, in the words of its own homepage, and it collects, transforms, and routes logs and metrics with one simple tool. It is written in Rust, packaged as a single static binary with no runtime, and maintained by the Community Open Source Engineering team at Datadog. The homepage lists logs and metrics as the signals it covers, and the project README lists traces as coming soon, so this guide never claims full traces support.

The repository, vectordotdev/vector on GitHub, was created on 27 August 2018, which makes it 2,994 days old, or 8.2 years, and it holds 22,811 stars against 2,372 forks as of 7 November 2026. That works out to 9.62 stars per fork, and the 2,573 open issues work out to 11.28 per 100 stars, although the open issues count includes pull requests, because the GitHub API counts the two together. The project is licensed under MPL-2.0, its last push landed on 7 November 2026, and the current release is v0.61.0, published on 29 October 2026, 9 days ago.

The 2026 release line ran from v0.53.0 on 27 January to v0.61.0 on 29 October, which is nine releases in 275 days, or one every 34.38 days on average. The two most recent gaps were 13 days between v0.59.0 and v0.60.0 and 10 days between v0.60.0 and v0.61.0, while the July-to-August gap ran 43 days. Vector is pre-1.0, so the release page advises readers to step through minor versions on upgrade, because each minor release can carry breaking changes.

How Do I Install Vector?

A vector install starts with the one-line script from the Vector homepage, and the route after that depends on your operating system. The official package-manager list covers APT, dpkg, Helm, Homebrew, MSI, Nix, pacman, RPM, and YUM, and the project also ships a single static binary with libc as its only dependency on most systems. The machine that runs the pipeline needs a stable home, and the guide to that machine is our what is VPS hosting page, which explains the server choice before you install anything on it.

The one-line script is the fastest route on Linux, and the package managers are the cleanest route when you want updates through your system. The Docker Hub image still carries the timberio name, and timberio is the legacy org name, while the current canonical registry is ghcr.io/vectordotdev/vector, so new setups should pull from there. The static musl builds exist for machines without glibc, but the docs warn they run noticeably slower on multi-core machines, so use the glibc build when it is available.

curl --proto '=https' --tlsv1.2 -sSfL https://sh.vector.dev | bash
sudo apt update
sudo apt install vector
sudo pacman -Syu vector
brew trust vectordotdev/brew && brew tap vectordotdev/brew && brew install vector
docker run --rm -v $(pwd)/vector/vector.yaml:/etc/vector/vector.yaml:ro ghcr.io/vectordotdev/vector:latest --config /etc/vector/vector.yaml

Check the install with vector --version, which prints the release you just installed, and compare it with v0.61.0 before you go further. If the number is older, the package mirror is behind the release page, and the one-line script is the fastest way to catch up. Building from source is documented but not recommended for beginners, because it needs a full Rust toolchain and a long compile for no benefit on a first install.

What Are Sources, Transforms, and Sinks?

The vector sources transforms sinks model is a directed acyclic graph with three kinds of component, and every pipeline is built from them. A source is where data comes in, a transform reshapes each event on the way through, and a sink is where data goes, and each component has a unique id that the next component names in its inputs list. The vector config file lives at /etc/vector/vector.yaml on most Linux installs, it supports YAML, TOML, and JSON, and YAML is the recommended format.

TOML is not deprecated, and existing TOML and JSON configs still work, but YAML became the default format in v0.33.0, so new readers should learn YAML first. The old habit of relying on /etc/vector/vector.toml loading by itself ended in v0.34.0, so always pass the path with the --config flag. You can also pass several files, a glob, or a whole directory with --config-dir, and Vector merges them into one topology.

Components link by id, so a sink that names parse_logs in its inputs list receives exactly the events that transform produced. Wildcards work in component ids, so an inputs list of app* matches every component whose id starts with app, which keeps multi-service configs short. Vector validates the whole graph at startup and refuses to run a config with a missing input, which is why a typo in an id fails loudly instead of silently dropping events.

How Do I Build My First Pipeline?

The first pipeline reads one log file, parses each line as JSON, tags it, and prints it to the console, and it needs only three components. The file source tails /var/log/logify/app.log, the remap transform runs one line of VRL, and the console sink prints each event as JSON, which lets you see exactly what Vector did to every line. Save the config as vector.yaml in a working directory, and run it with vector --config vector.yaml, or mount it into the Docker image read-only if you installed through Docker.

sources:
  app_logs:
    type: "file"
    include:
      - "/var/log/logify/app.log"
transforms:
  parse_logs:
    type: "remap"
    inputs:
      - "app_logs"
    source: |
      . = parse_json!(.message)
      .environment = "dev"
sinks:
  print:
    type: "console"
    inputs:
      - "parse_logs"
    encoding:
      codec: "json"

The file source adds file, host, message, source_type, and timestamp to every event it reads, so your transform can trust those fields to exist. One spelling detail matters: the main example in the official configuration reference uses include, while the wildcards example on the same page spelled includes in one example, so if Vector warns about an unknown field at startup, check that spelling first. The console sink is only for learning, and a real pipeline swaps it for a storage sink such as Elasticsearch, S3, or Loki once the events look right.

Start Vector in the foreground for the first run, because the startup log tells you immediately whether the config loaded or failed. A clean start prints a line that Vector has started, with the version number beside it, and every new line in the log file appears on your console within a second. If nothing prints, check that the log file path exists and that Vector can read it, because a silent pipeline is almost always a path problem.

What Is VRL, and How Do I Use It?

The vector VRL lesson is the one that turns a working pipeline into a useful one, because VRL is the language that reshapes every event. VRL stands for Vector Remap Language, an expression-oriented language for transforming observability data, and you write it inside the source field of a remap transform. A VRL program acts on one event at a time, the dot means the whole event, and paths such as .kubernetes.pod_id reach into nested fields.

. = parse_json!(.message)
parsed_timestamp, err = parse_timestamp(.timestamp, format: "%Y-%m-%dT%H:%M:%S.%fZ")
if err == null {
  .timestamp = to_unix_timestamp(parsed_timestamp)
} else {
  .timestamp = to_unix_timestamp(now())
}
.timestamp_str = from_unix_timestamp!(.timestamp)
del(.username)
.message = downcase(string!(.message))

The rule that surprises every beginner is fallibility: some VRL functions can fail, and a program does not compile unless every possible error is handled. The two handling styles are the pair assignment, which returns the result and the error together, and the bang mark, which aborts the event when the function fails. The docs example parses the message as JSON with parse_json with a bang, then parses the timestamp with a pair assignment and falls back to the current time when the parse fails.

The VRL bundled with v0.61.0 is version 0.38.0, and it changed one small rule: to_unix_timestamp with the nanoseconds unit is now correctly marked fallible, so old code without a bang or a fallback no longer compiles. The fix is one character, a bang after the function name, or a double question mark with a default value. Test new VRL in the vector vrl command, which starts a read-eval-print loop that takes a JSON file of events, before you wire it into a config.

How Do I Run Vector as a Service?

A foreground process stops when your terminal closes, so the next step is a service that survives logouts and restarts. Package installs ship the service files, and on Linux you manage Vector with systemctl, while Homebrew users use brew services and Kubernetes users use kubectl. The same config file runs under the service, so validate it first and then start the service, and the startup log is the first place to look when something fails.

sudo systemctl start vector
sudo systemctl status vector
sudo systemctl enable vector
sudo journalctl -u vector -f

Vector ships a live dashboard called vector top, which shows throughput per component like a task manager for your pipeline, but the API is disabled by default on bare installs, so the command fails until you set api.enabled to true in the config. Helm deployments enable the API by default, which is one less surprise on Kubernetes, and the 0.55.0 release moved the API from GraphQL to gRPC, so old dashboard scripts that hit the GraphQL endpoint no longer work. A Grafana beginner guide shows how to turn those same numbers into a dashboard that watches the pipeline while you sleep.

Buffers are the durability lesson: every sink holds a small in-memory queue, and when the sink falls behind, Vector either blocks or drops events depending on the buffer setting. A disk buffer turns that queue into a write-ahead log that survives restarts, and Vector picks up where it left off after a crash. The official guidance is to use disk buffers for the sink that holds your system of record, and to watch the free space in data_dir, because Vector stops itself when a disk flush fails.

Agent or Aggregator: Which Role Do I Run?

Vector can serve as both an agent and an aggregator, and the role is a deployment choice, not a different program. An agent is a daemon that collects everything on one host, or a sidecar coupled to one service, and an aggregator is a central process that receives from many agents and does the heavy processing. The official guidance is to keep agents simple forwarders, shift processing and durability to aggregators, and run several aggregators per network boundary.

Size an aggregator at no less than 4 vCPUs and 8 GiB of memory per instance, and use DNS or service discovery so agents find the aggregators without hardcoded addresses. Agents talk to aggregators through the vector source and sink pair, which is the native Vector-to-Vector link, and the scheme-less form of that link changed in v0.61.0, because scheme-less vector sink endpoints now default to https, so a plaintext link needs an explicit http address. Metrics from the pipeline itself can feed a time-series store, and the setup for one is covered in our Prometheus beginner guide, which pairs naturally with Vector on the same host.

When an aggregator consumes from a queue such as Kafka, the official advice is to use memory buffers with when_full set to block, so backpressure flows upstream instead of dropping events. That single setting is the difference between a pipeline that slows down gracefully and one that loses logs under load. Topologies have three names in the docs, distributed, centralized, and stream-based, and most small teams start centralized and add aggregators only when one host stops being enough.

How Do I Run Vector on Kubernetes?

A vector kubernetes deployment runs Vector as a DaemonSet on every node, or as a central aggregator, and Helm is the supported route. Add the Vector Helm repository, update it, and install the unified vector chart with the role set to Agent for the DaemonSet, and the kubernetes_logs source then collects logs from every pod on the node. The cluster guide you need beside this one is our Kubernetes beginner guide, which explains the DaemonSet and StatefulSet concepts the chart uses.

helm repo add vector https://helm.vector.dev
helm repo update
helm install vector vector/vector --set role=Agent
kubectl rollout restart --namespace vector daemonset/vector-agent

The chart version is not the Vector version, and the Artifact Hub page showed chart 0.45.0 while the application was already at 0.61.0, so always read the two numbers with their labels. The old separate vector-agent and vector-aggregator charts are deprecated in favour of the single unified chart with its role parameter. The chart enables the API by default, which means vector top works on a Helm install without the config change a bare install needs.

The chart takes your raw Vector config through the customConfig value, which replaces the generated config entirely, so the YAML you tested locally is the YAML the cluster runs. The chart passes --config-dir to Vector, so the config lands as files under /etc/vector inside the pod. A DaemonSet restart rolls the new config onto every node, and kubectl rollout restart is the command that triggers it.

Vector vs Fluentd: Which Log Pipeline Should I Pick?

Vector vs fluentd is the comparison most readers ask about, because the two tools solve the same problem with different shapes. Vector config is one YAML, TOML, or JSON file with sources, transforms, and sinks linked by id, while Fluentd config is a directive file with source, filter, and match blocks, plugins chosen with an @type parameter, and routing by tag. Fluentd is the older project by 2,626 days, or 7.19 years, and Vector holds 9,200 more GitHub stars, or 1.68 times as many, yet both tools are open source, both run as agents and aggregators, and the choice comes down to config ergonomics and the ecosystem you already run.

The honest gap in this comparison is speed: no measured speed benchmark exists from a verifiable source, so this guide makes no throughput claim for either tool. Old posts quote a vendor-repo benchmark and a single-team migration story, but both are single-source figures from 2022, and neither was independently verified. One genuine migration gotcha is documented by Red Hat: a syslog consumer broke after moving to Vector, because Vector quoted the message value while Fluentd did not, so message-format differences are real.

Choose Vector when you want one binary, a YAML config, and VRL for reshaping, and choose Fluentd when the plugin you need already exists in its ecosystem and your team knows its directive config. Either tool can collect, transform, and route the same logs, so the wrong choice is a config you cannot read six months later. The next step after choosing is to build the first pipeline above, which runs the same ideas with Vector's syntax.

from datetime import date
today = date(2026, 11, 7)
created = date(2018, 8, 27)
v0610 = date(2026, 10, 29)
v0600 = date(2026, 10, 19)
v0590 = date(2026, 10, 6)
v0580 = date(2026, 8, 26)
v0570 = date(2026, 7, 14)
v0530 = date(2026, 1, 27)
betterstack_log = date(2025, 5, 29)
amireslampanah = date(2022, 11, 11)
devto_loki = date(2025, 2, 12)
devto_scenario = date(2026, 11, 1)
fluentd_created = date(2011, 6, 19)
stars = 22811
forks = 2372
issues = 2573
fluentd_stars = 13611
print("repo age days:", (today - created).days)
print("repo age years:", round((today - created).days / 365.25, 2))
print("v0.61.0 age days:", (today - v0610).days)
print("days between v0.60.0 and v0.61.0:", (v0610 - v0600).days)
print("days between v0.59.0 and v0.60.0:", (v0600 - v0590).days)
print("days between v0.57.0 and v0.58.0:", (v0580 - v0570).days)
print("days from v0.53.0 to v0.61.0:", (v0610 - v0530).days)
print("average days between 2026 releases:", round((v0610 - v0530).days / 8, 2))
print("stars per fork:", round(stars / forks, 2))
print("open issues per 100 stars:", round(issues / stars * 100, 2))
print("betterstack log age days:", (today - betterstack_log).days)
print("amireslampanah age days:", (today - amireslampanah).days)
print("devto loki age days:", (today - devto_loki).days)
print("devto scenario age days:", (today - devto_scenario).days)
print("fluentd age days:", (today - fluentd_created).days)
print("fluentd age years:", round((today - fluentd_created).days / 365.25, 2))
print("vector stars minus fluentd stars:", stars - fluentd_stars)
print("vector stars divided by fluentd stars:", round(stars / fluentd_stars, 2))
print("fluentd repo older than vector repo days:", (created - fluentd_created).days)
print("fluentd repo older than vector repo years:", round((created - fluentd_created).days / 365.25, 2))

Sources

The facts in this guide come from the official Vector documentation on vector.dev, including the installation, configuration, VRL, deployment-roles, buffering-model, and management pages, all fetched on 7 November 2026, plus the release pages for v0.61.0 and the 2026 release index, which pin the current version and its breaking changes. The repository figures come from the GitHub API for vectordotdev/vector and fluent/fluentd, fetched the same day, and they are snapshots as of 7 November 2026, not permanent totals. The file-source spelling check was verified against the raw configuration reference in the vector repository on GitHub, fetched the same day, which shows include in the main example and includes only in the wildcards example.

The field guides are the BetterStack guide, whose captured startup log is dated 29 May 2025, 527 days ago, and shows Vector 0.47.0, the Parseable guide from about 250 days ago, which covers the Kubernetes DaemonSet arc, the amireslampanah tutorial from 11 November 2022, 1,457 days ago, which is the oldest complete tutorial found, and the dev.to pieces, a DigitalOcean scenario published 6 days ago and a Vector, Loki, and Grafana setup from 12 February 2025, 633 days ago. The comparison framing draws on the Fluentd configuration docs and the modern-datatools review updated on 19 October 2026, which supplies the 18 transforms and 61 sinks figures, and on the Red Hat knowledgebase note about the syslog quoting difference after migration. Every number in the tables and the computed block was produced by running the Python script above on 7 November 2026, and every version, date, and command was checked against the live sources this run.

No commands in this guide were executed in a live testbed, so every command shown is documented syntax from the official docs, and no benchmark number appears anywhere, because no measured comparison of Vector and Fluentd was found from a verifiable source. Features on the main branch that have not shipped in a stable release are not presented as shipped anywhere in this guide. The homepage figures for downloads and contributors are vendor claims, so they are not repeated here as verified facts.

Frequently Asked Questions

Is Vector free to use?

Yes, Vector is free and open source under the MPL-2.0 license, with no paid tier for the binary itself, and you can run it on as many machines as you like. The project is maintained by the Community Open Source Engineering team at Datadog, and it holds 22,811 GitHub stars as of 7 November 2026. Datadog also offers a managed Observability Pipelines interface for building pipelines in its UI, but that is a separate product, and this guide covers only the free open-source pipeline.

Does Vector work on Windows?

Yes, Vector ships Windows builds with every release, including a zip archive and an MSI installer, and the same YAML config runs on Windows as on Linux. The one-line install script is for Unix systems, so Windows users take the MSI or the zip from the releases page instead. The config file location differs by install method, so check the installation docs for the Windows path before you start the service.

Why does Vector warn about unknown fields in my config?

Since v0.61.0, Vector warns at startup about config fields it does not recognise, which usually means a typo such as retry_atempts, and the warning names the field so you can fix it. The warning is not fatal, and existing configs keep working, but the field is ignored, so the setting you thought you applied never takes effect. Check the spelling against the component reference first, and remember the include spelling lesson from the first pipeline section, because that is the most common trip point.

Is Vector better than Fluentd?

Vector is the better pick when you want one static binary, a YAML config, and VRL for reshaping events in a single program, while Fluentd is the better pick when your team already knows its directive config or needs a plugin only its ecosystem has. Vector holds 1.68 times as many GitHub stars, but stars are popularity, not proof of speed, and no measured benchmark compares the two. Try Vector for a week on your real logs, keep Fluentd installed beside it, and keep the one that fits your hands.

How do I check whether my pipeline is running?

Run vector top after enabling the API, and it shows live throughput per component, so a stuck pipeline shows up as a flat line. The systemctl status output shows whether the service is active, and the journal shows the startup log, which names the config error when a bad file stops the start. A quicker check is the console sink from the first pipeline, because every event that reaches it proves the source, the transform, and the sink all work.