generated from TrueCloudLab/basic
Add Loki log sending #5
No reviewers
Labels
No labels
Infrastructure
blocked
bug
config
discussion
documentation
duplicate
enhancement
go
help wanted
internal
invalid
kludge
observability
perfomance
question
refactoring
wontfix
No milestone
No project
No assignees
3 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference: TrueCloudLab/frostfs-observability#5
Loading…
Reference in a new issue
No description provided.
Delete branch "achuprov/frostfs-observability:loki"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Add package for sending logs to Loki
Related to:
TrueCloudLab/frostfs-dev-env#57
TrueCloudLab/frostfs-node#740
Signed-off-by: Alexander Chuprov a.chuprov@yadro.com
642aeb169b
to87129d7d29
87129d7d29
to04238a7ee2
[#5] logs:Add Lokito [#5] logs: Add Loki04238a7ee2
to8cee7e75c1
8cee7e75c1
tof3480b9511
f3480b9511
toe0b6c97654
e0b6c97654
toeac876b88a
[#5] logs: Add Lokito Add Loki log sending packageAdd Loki log sending packageto Add Loki log sending@ -0,0 +92,4 @@
}
}
func processMissedMessages(batch *[]*logEntry) {
*[]*logEntry
- wow! Could it be simplified?You can pass slice by value, since slice contains only two int's and pointer. Also to append slice using
append
- style is straightforward.fixed
eac876b88a
toa84d2e3af0
@ -0,0 +62,4 @@
batch = append(batch, entry)
}
batch = processMissedMessages(batch)
processBatch(batch)
Now looks like batch reset misses: after
processBatch
batch
still contains old records.This isn't an issue because after this, we end the function and the
batch
will no longer exist@ -0,0 +16,4 @@
encoder zapcore.Encoder
}
func New(original zapcore.Core, call func(msg string, logTime time.Time) error, enabledCall func() bool) *LokiCore {
What is original core here? We can unite anything with
87577d85d5/zapcore/tee.go (L37C18-L37C18)
, so 2 cores are possibleThis is any core that implements the
Core
interface. If you want to pass two cores, you can merge them usingzapcore.NewTee
. In this case, the lowest logging level of the two cores will be used.@ -0,0 +7,4 @@
// If the client is disabled, it does nothing.
// If the entries channel is full, the message is discarded.
func Send(msg string, timestamp time.Time) error {
We have a function here and it's only purpose is to be provided as a callback to lokicore.
Why not replace global client with a client initialized inside
lokicore.New
?@ -0,0 +88,4 @@
func processBatch(batch []*logEntry) []*logEntry {
if len(batch) > 0 {
sendLogs(batch)
batch = batch[:0]
Why is is
[]*logEntry
and not[]logEntry
? For a slice of pointers the memory is leaking a bit afterbatch = batch[:0]
@ -0,0 +1,67 @@
package loki
IMO
loki
is too specific to be on the toplevel in this repo. Can we move it tologging/lokicore/loki
(orlokicore/internal/loki
?@ -0,0 +7,4 @@
"time"
)
const LogEntriesChanSize = 5000
Is this useful for a client of the library?
I don't think it makes sense to adjust this parameter since it's set to an optimal value
Then why is it exported?
fix
@ -0,0 +8,4 @@
)
const LogEntriesChanSize = 5000
const Path string = "/api/prom/push"
Can this path be configurable from the loki side?
normalizeUrl
is broken if so.a84d2e3af0
tofcf7714928
fcf7714928
to5a9e3ce812
@fyrchik fixed
5a9e3ce812
to658a4f3ed8
@ -0,0 +5,4 @@
"time"
)
// If the client is disabled, it does nothing.
It makes no sense to create a client which is disabled, right? Why are there
IsEnabled
checks?fixed
658a4f3ed8
tob3ad3335ff
@ -0,0 +43,4 @@
// E.g. localhost:3100/api/prom/push.
Endpoint string
Labels map[string]string
//Maximum message buffering time.
Space after the
//
.