cartridge metrics

[vasiliy-t]

We could collect such metrics as:

1. We could add new metric issues_count of type gauge. Value is a number of cluster issues this instance knows. This should be good enough for basic alerting - healthy cluster reports 0 issues. - closed in tnt_cartridge_issues gather only local issues #243 and Add Cartridge issues gauge #144
2. Cartridge instance state (like OperationError) as numerical value -- needs design
3. Time since last restart - already present as metric tnt_info_uptime
4. Failover trigger count -- could be transformed from metric tnt_read_only, but needs desing too

[github-actions]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 30 days

[opomuc]

This has been started in 4f4917d and released under v0.6.0.

I will close it for now. Feel free to reopen or create a separate issue regarding new metrics in cartridge.

[yngvar-antonsson]

Reopen with 3 new metric.

[opomuc]

Those are new metrics

1. Cartridge instance state (like OperationError) as numerical value
2. Time since last restart
3. Failover trigger count

[rosik]

Design draft. We've discussed making it a part of cartridge. It would be easier to maintain the list.

local state_numeric_const = {
    [''] = 0,
    ['Unconfigured'] = 10,
    ['ConfigFound'] = 11,
    ['ConfigLoaded'] = 12,

    ['BootstrappingBox'] = 20,
    ['RecoveringSnapshot'] = 21,

    ['ConnectingFullmesh'] = 30,

    ['BoxConfigured'] = 40,

    ['ConfiguringRoles'] = 50,
    ['RolesConfigured'] = 51,

    ['ReloadingRoles'] = 60,

    ['InitError'] = 90,
    ['BootError'] = 91,
    ['OperationError'] = 92,
    ['ReloadError'] = 93,
}

for state, _ in pairs(state_transitions) do
    if state_numeric_const[state] == nil then
        error(string.format('Missing numeric const for %q', state))
    end
end

[yngvar-antonsson]

Closed until someone would need those metrics. Feel free to reopen.