Hey! I've been trying to setup Vector on my cluster, but am struggling with the config as it seems various resources are outdated, or unclear as to how to approach Loki and Kubernetes.

Loki config:

data_dir = "/var/lib/vector"
[sources.kubernetes]
  type = "kubernetes"
[sinks.loki]
  type = "loki"
  inputs = ["kubernetes"]
  endpoint = "http://loki:3100"
  encoding.codec = "json"

The output of running this is:

➜  ~ kubectl -n vector logs vector-4d5np vector
May 29 18:27:28.838  INFO vector: Log level "info" is enabled.
May 29 18:27:28.838  INFO vector: Loading configs. path=["/etc/vector/vector.toml"]
May 29 18:27:28.841 ERROR vector: Configuration error: "/etc/vector/vector.toml": missing field `labels` for key `sinks.loki`

What sort of labels should I be putting the config?

How do I extract k8s labels and ship them to Loki?

See Loki's fluent-bit config for what sorts of labels (I assume) should be added.

I got the deployment working using the latest from the k8s-impl branch from https://github.com/timberio/vector/pull/2653 However, the process crashes immediately with a "capacity overflow" in the Loki sink configuration.

Here is my config:

  managed.toml: |
    # Data dir is location controlled at the `DaemonSet`.
    data_dir = "/vector-data-dir"

    # Ingest logs from Kubernetes.
    [sources.kubernetes_logs]
      type = "kubernetes_logs"

    [sinks.out]
      # General
      type = "loki"
      inputs = ["kubernetes_logs"]
      endpoint = "http://loki.loki.svc.cluster.local:3100"
      healthcheck = true
      encoding.codec = "json"
      [sinks.out.labels]
        k8s_pod_name = "{{ kubernetes.pod_name }}"
        k8s_pod_uid = "{{ kubernetes.pod_uid }}"

here is the log with backtrace:

Jul 09 21:14:07.635  INFO vector: Vector is starting. version="0.10.0" git_version="v0.9.0-421-g33de5ef" released="Thu, 09 Jul 2020 20:33:25 +0000" arch="x86_64"
Jul 09 21:14:07.635  INFO vector::sources::kubernetes_logs: obtained Kubernetes Node name to collect logs for (self) self_node_name="ip-10-210-70-174.ec2.internal"
thread 'main' panicked at 'capacity overflow', src/liballoc/raw_vec.rs:622:5
stack backtrace:
   0: backtrace::backtrace::libunwind::trace
             at ./cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.46/src/backtrace/libunwind.rs:86
   1: backtrace::backtrace::trace_unsynchronized
             at ./cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.46/src/backtrace/mod.rs:66
   2: std::sys_common::backtrace::_print_fmt
             at src/libstd/sys_common/backtrace.rs:78
   3: <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt
             at src/libstd/sys_common/backtrace.rs:59
   4: core::fmt::write
             at src/libcore/fmt/mod.rs:1069
   5: std::io::Write::write_fmt
             at src/libstd/io/mod.rs:1504
   6: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:62
   7: std::sys_common::backtrace::print
             at src/libstd/sys_common/backtrace.rs:49
   8: std::panicking::default_hook::{{closure}}
             at src/libstd/panicking.rs:198
   9: std::panicking::default_hook
             at src/libstd/panicking.rs:218
  10: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:511
  11: rust_begin_unwind
             at src/libstd/panicking.rs:419
  12: core::panicking::panic_fmt
             at src/libcore/panicking.rs:111
  13: core::panicking::panic
             at src/libcore/panicking.rs:54
  14: alloc::raw_vec::capacity_overflow
             at src/liballoc/raw_vec.rs:622
  15: alloc::raw_vec::RawVec<T,A>::allocate_in::{{closure}}
  16: <vector::sinks::loki::LokiConfig as vector::topology::config::SinkConfig>::build
  17: vector::topology::builder::build_pieces
  18: vector::topology::builder::check_build
  19: vector::topology::validate
  20: vector::main::{{closure}}
  21: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll
  22: <futures_util::compat::compat03as01::Compat<Fut> as futures::future::Future>::poll
  23: futures::task_impl::std::set
  24: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
  25: tokio::runtime::enter::Enter::block_on
  26: tokio::runtime::context::enter
  27: tokio::runtime::Runtime::block_on
  28: tokio_compat::runtime::threadpool::Runtime::block_on
  29: vector::main
  30: std::rt::lang_start::{{closure}}
  31: std::rt::lang_start_internal::{{closure}}
             at src/libstd/rt.rs:52
  32: std::panicking::try::do_call
             at src/libstd/panicking.rs:331
  33: std::panicking::try
             at src/libstd/panicking.rs:274
  34: std::panic::catch_unwind
             at src/libstd/panic.rs:394
  35: std::rt::lang_start_internal
             at src/libstd/rt.rs:51
  36: main
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.