Skip to content

Migrate mixer API and config to this repo. #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Dec 17, 2016
Merged

Migrate mixer API and config to this repo. #2

merged 1 commit into from
Dec 17, 2016

Conversation

geeknoid
Copy link
Contributor

@geeknoid geeknoid commented Dec 16, 2016

This clearly separates mixer's API and configuration surface from
the implementation. This is designed to make it possible to
replace the mixer in a fully compatible way.

Note that I put the config into mixer/config instead of istio/config since it is in fact all specific to the mixer. These configuration definitions aren't used by the proxy or manager themselves.

Note to Doug: this clearly doesn't build as it needs the bazel treatment (mainly a WORKSPACE file I think). I hope you can supply this after this initial checkin.

@geeknoid
Copy link
Contributor Author

I added a WORKSPACE file (which I just copied from the mixer repo) and I'm able to build the config directory, but the API directory fails due to some missing dependencies. Better, but still needs attention from someone that knows what they're doing with bazel...

@douglas-reid
Copy link
Contributor

What if we don't bazel-ify this repo?

We could potentially have the other repos define the build rules for various languages, etc.?

@qiwzhang would that work for the proxy stuff?

@geeknoid
Copy link
Contributor Author

geeknoid commented Dec 16, 2016 via email

@douglas-reid
Copy link
Contributor

Well, hopefully, PR reviews would catch the syntax issues... but that obviously has flaws.

We could also add generated code here too (with a script for doing the generation) and require updates on any commit to both the source and gen'd libs.

Why don't we either (a) commit without bazel now or (b) commit with the broken bits and I'll work with @qiwzhang to get a build setup working for the repo afterwards?

@geeknoid
Copy link
Contributor Author

geeknoid commented Dec 16, 2016 via email

@@ -0,0 +1,112 @@
workspace(name = "com_github_istio_mixer")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

name this com_github_istio_api

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

"google/rpc/status.proto",
],
imports = [
"../../external/com_github_google_protobuf/src",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think to use this setup, we need the .bazelrc file that adds the "--standalone" like bits (as this escapes the sandbox).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@@ -0,0 +1 @@
/usr/local/google/home/mtail/.cache/bazel/_bazel_mtail/1875d8bd5ec708fecb970aa8fdf4f58a/execroot/com_github_istio_mixer
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need a to add a git ignore entry for bazel-*

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

//
// * Any deleted attributes are removed from the attribute context.
//
message AttributeUpdate {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't feel quite right to me. Why not just call this Attributes? And have the dictionary be understood to be context-dependent (either complete or overriding depending on whether or not a context is provided)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed to Attributes.

Added more text about how the dictionary is used. Added the reset_context boolean to force everything to start from scratch. And added a few more comments here and there for clarity.

map<int32, string> email_attributes = 10;

// Attributes that should be removed
repeated int32 deleted_attributes = 11;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we expand on what it means to be removed? Removed from the set of AttributeValues and the Dictionary?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

//
// Once a new dictionary has been sent over, it stays in effect until
// a new dictionary is sent.
map<string, int32> dictionary_update = 1;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this imply that the same attribute can be referred to by different IDs by different proxies, etc., across the system?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. It prevents us having to maintain any kind of registry of attribute ids. It's dynamic in nature, per connection to the mixer. As proxies are updated in a staged roll out and new attributes are added, then the dictionary is sent accordingly.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ultimately I think it makes sense to reserve a block of ids for well-known things. This is what HTTP2 did, I would just add a TODO for the moment.

// The general idea is to leverage the stateful connection from the
// proxy to the mixer to keep to a minimum the 'attribute chatter'.
// Only delta attributes are sent over, multiple concurrent attribute
// contexts can be used to avoid trashing, and attribute indices are used to
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/avoid trashing/avoid thrashing/ ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

string name = 1;

// The set of attributes that are necessary to describe a log entry of this type.
repeated string attributes = 2;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(here and elsewhere) to be clear, these are attribute descriptor names?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes.

// Name = IDENT { "." IDENT } ;
//
// Where `IDENT` must match the regular expression `[a-z][a-z0-9]+`.
string name = 1;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to clarify anything about uniqueness of this name?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DOne.

@geeknoid
Copy link
Contributor Author

PTAL.

Copy link
Contributor

@qiwzhang qiwzhang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will like to Not have any bazel build files.
Other repos that use it can have bazel build files.
They will fail, they they update their "commit"

@@ -0,0 +1,57 @@
load("@org_pubref_rules_protobuf//go:rules.bzl", "go_proto_library")

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't put a BUILD file here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, they're gone.


// Attributes being updated within the specified attribute context. When reset_context is false,
// then these maps are treated as deltas, so only what's changed needs to be included.
map<int32, int64> int64_attributes = 4;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like overkill I would just start with int64, string, boolean and maybe double if we have a case for it.

It seems odd to be mixing semantic types and data types here when we also have all the semantics in descriptors.

I would use string for ipaddress & email & int64 for timestamp, otherwise following this pattern we will end up with separate fields for DNS names, durations, GUIDs, ... and I don't think that's useful.

FWIW 'bytes' might be a better type than string for ip addresses and would cover both types adequately based on the length of the bytestring

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

// An instance of this is delivered to the mixer with every
// API call.
//
// The general idea is to leverage the stateful connection from the
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/connection/stream

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

// proxy to the mixer to keep to a minimum the 'attribute chatter'.
// Only delta attributes are sent over, multiple concurrent attribute
// contexts can be used to avoid thrashing, and attribute indices are used to
// keep the wire protocol maximally efficient.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sounds like a pretty complex protocol. Can we start with one implicit context and add disjoint contexts when we have evidence of need. After all multiple contexts can be handled by having multiple GRPC streams.

Copy link
Contributor

@qiwzhang qiwzhang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we just put the proto files here for now? I am blocked by this.

@douglas-reid
Copy link
Contributor

I vote to put the protos in, without bazel stuff, to unblock everyone.

//
// This method is intended for use by the Istio proxy and not by
// individual services.
rpc Check(stream CheckRequest) returns (stream CheckResponse) {}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Attribute context will span check & report all the time so having them be independent streams seems quite wasteful given all the effort put into dictionary management above. I would either:

1 - punt on dictionary management possibly in favor of a partially static dictionary (see HTTP2 https://http2.github.io/http2-spec/compression.html#static.table.definition)
2. - Allow the client to define the context in which the dictionary grain is useful and to choose which operation to perform in that context

The former seems a lot simpler than the latter frankly

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the lack of special cases in this model, and I like the uniformity. All of the API methods work the same way, both on the server and client, so they'll just be some common code on both ends. The actual runtime overhead of multiple dictionaries is tiny both in CPU time and memory.

I'd like to keep this as-is and see how it pans out.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't strictly mind repeating the same dictionary or attribute context three times so that its available for each operation type for the context at hand. Need to document how to handle the failure mode when an attribute context cannot be allocated by the server, I suspect we should terminate the stream and reset everything and require the client to recover.

I'm fine with this, will add some clarifying doc comments.

string description = 3;

// Types of supported attribute values.
enum ValueType {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Continuing the earlier note about semantic types vs. data types, this enum is mixing both. I think we should have two enums

enum DataType {
INT, //as int64
FLOAT, //as double
BOOL,
STRING,
BYTES
}

enum ValueType {
LABEL,
URI.
DNS_NAME,
EMAIL,
TIMESTAMP,
IP_ADDR,
COUNTER,
GAUGE,
SUMMATION,
DURATION, // can standardize units here
}

This clarifies which fields in Attributes should be used for each descriptor field. It allows us to codify how coercion between DataTypes should occur for a specific ValueType, converting STRING to INT for a COUNTER

Example mappings to (data type, value type)
ipaddress -> (BYTES, IP_ADDR)
username -> (STRING, EMAIL)
k8s pob label -> (STRING, LABEL)
k8s service DNS -> (STRING, DNS_NAME)
system uptime -> (INT, DURATION)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is appealing. Let me think about it for a bit.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been playing with this split and I'm not sure it's useful or desirable.

With my design, as new well-known and/or efficient storage formats are adopted, we just add a new value type for it. For example, Timestamp, Duration, IPv6Address, and likely more over time. With your approach, all those end up being encoded as raw bytes.

I think perhaps what threw you off is the choice of names. What if instead of what I had, we used instead:

enum ValueType {
// A generic 64-bit signed integer.
UNDISCRIMINATED_INT = 0;

// A 64-bit floating-point value.
UNDISCRIMINATED_FLOAT = 1;

// A variable-length string.
UNDISCRIMINATED_STRING = 2;

// Boolean; true or false.
UNDISCRIMINATED_BOOL = 3;

// A point in time, encoded as bytes or string
TIMESTAMP = 4;

// A quantity of time, encoded as bytes or string
DURATION = 5;

// An IP address, encoded as bytes or string
IP_ADDR = 6;

// An email address, encoded as a string
EMAIL_ADDR = 7;

// A URL, encoded as a string
URI = 8;
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Attributes will be used by selectors and policies. We need to know the types to apply expressions on them. That is the most important issue we need to cover. Splitting the value format from value type is attractive, but also adds more complexity. I suggest we don't pay for it initially.

I think the most useful feature needed in our use case is string, int, double, timestamp, ipaddress. Let's start with them first.

While proto does have duration type, most other systems don't have it. It would be much easier to store duration as a double seconds, then it is easy to use in an expression. It is much harder to manipulate 96-bit duration and we almost never need that much precision.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think in either form we'd have to handle migration to concrete schema types over time by coercion so I don't think that's a major difference really.

I do think the names threw me off so I'm OK with the single enum. Comments would have been enough to disambiguate I think so no need to make the names longer I think. What would be most useful would probably be examples.

INT -- Codes and enum like values such as HTTP status. You could argue these are POINT metrics too given that they can have thresholds applied to them, they just can't be aggregated
FLOAT -- I have a hard time coming up with an example that is not really a metric, lat/long maybe but those seem like they should be metrics with a type of POINT ? If we don't have a real use-case lets drop it and add it back?
STRING -- NAME or LABEL would be better choices for the the enum value?
BOOL -- I think its fairly self explanatory, no need to elaborate

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only way to interact with proxies and REST APIs is via HTTP header. So we want to make sure all values are expressible as strings. The expression engine will become insanely complicated as we add more types. We should really limit the choice as much as we can. We can always add more types later, but impossible to remove anything.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. I improved this somewhat. We'll tweak more as we learn where we'd like more discriminated typing and/or more efficient encodings.

string display_name = 8;

// The kind of measurement. It describes how the data is reported.
enum MetricKind {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I see how you're intending to partition the semantic types. Probably want to put DURATION, SUMMATION and maybe TIMESTAMP in here

// for a quota of this type.
repeated string dimension_attributes = 3;

// The unit in which the quota's values are reported. The
Copy link
Contributor

@louiscryan louiscryan Dec 17, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some examples here would be useful

Copy link
Contributor

@mandarjog mandarjog Dec 17, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that all these are istio configs, regardless of where they are used at present.
If it is something that a User sees and interacts with while configuring istio, It is istio config. For example, if this is how someone configures the quota aspect, It surely is istio config.

Unless something is purely between the proxy and mixer, it should be istio config.
Looks like all these descriptors will form a vocabulary of {attributes, quotas, ... }
At configuration time, you either define a quota or a rate limit inline, or use a reference from the vocabulary.

For majority of the config the user should be configuring istio. They should only need to configure individual software components sparingly.

Copy link
Contributor Author

@geeknoid geeknoid Dec 17, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where these types live in the repo is immaterial to what the user experiences, right?

For the programmer, these types exist to configure the mixer, so I prefer the definitions be closely coupled with the mixer.

Consider for example if these types change, the only code that needs to change is in the mixer. If we have multiple implementations of the mixer, they'll each need to deal with these configs. If we have multiple implementations of the proxy and manager, none of them will deal with these types.

// The default imposed maximum amount for values of this quota.
int64 max_amount = 7;

// The type of quota behavior expected, defaults to NORMAL if not specified
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NORMAL not in enum

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

// Whether the quota's current value is tracked precisely or not.
// Precisely tracked quotas only allow a relatively modest level of
// scaling, whereas imprecise quotas can support a nearly unbounded
// level of scaling at the cost of potentially having small
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would eliminate 'small'

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

//
// **Basic units (UNIT)**
//
// * `bit` bit
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need schema for things like bytes? Is this just to facilitate display of aggregates in a standard fashion?

//
// **Grammar**
//
// The grammar includes the dimensionless unit `1`, such as `1/s`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like an odd example in the context of a metric assuming 1/s means '1 per second'.

I assume we just want to present either things where the unit is implied in the name, or things with units that are specified above which is currently limited to bytes & time. Is this worth the effort?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is all directly lifted from One Platform. I've no particular
attachment to this stuff if we want to simplify. We could just delete the
field completely for the time being.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the unit field completely.

//
// The processing order for this state in the mixer is:
//
// * Any dictionary updates are applied
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it an 'update' or a replace, comment on line 56 indicates a replace. Phrasing...

'The new dictionary is installed if provided'

Also it's probably an API error for the first request not to install a dictionary.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

// * Any dictionary updates are applied
//
// * The requested attribute context is looked up. If no such context has been defined, a
// new context is automatically created and initialized to the empty state.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add examples of the events that might cause a context to be created, e.g. a new socket is opened to a server, a new user logs in etc. Could go on the field instead.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

// * If reset_context is true, then the attribute context is reset to the
// empty state.
//
// * Any attribute deltas are applied to the attribute context.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'deltas applied' gives me the impression that aggregation occurs, 'overwrite' seems like a better term

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

// A dictionary that provides a mapping of attribute name to
// a shorthand index value.
//
// This is intended to leverage the stateful connection from the
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Be careful with the term 'connection' here. We're really using GRPCs notion of streams which might cause folks to get confused with TCP connections

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

//
// Dictionaries are independent of the attribute context and are thus global
// for the current gRPC stream.
map<string, int32> dictionary_update = 1;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit. See note above about 'update'. Could just make the field name 'active_dictionary', 'new_dictionary' etc.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

// for the current gRPC stream.
map<string, int32> dictionary_update = 1;

// The attribute context against which the new attributes are applied.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The id of the attribute context...


// When true, resets the current attribute context to the empty state before
// applying any new incoming attributes.
bool reset_context = 3;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider providing guidance about when an attribute context should be reset so that the server can reclaim memory. E.g. If an HTTP2 connection terminated we should be able to reclaim the memory occupied by the context by re-using its id for a new connection later on.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@louiscryan
Copy link
Contributor

louiscryan commented Dec 17, 2016 via email

Copy link
Contributor

@douglas-reid douglas-reid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove BUILD before merge. Otherwise, LGTM.

string description = 3;

// Types of supported attribute values.
enum ValueType {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Attributes will be used by selectors and policies. We need to know the types to apply expressions on them. That is the most important issue we need to cover. Splitting the value format from value type is attractive, but also adds more complexity. I suggest we don't pay for it initially.

I think the most useful feature needed in our use case is string, int, double, timestamp, ipaddress. Let's start with them first.

While proto does have duration type, most other systems don't have it. It would be much easier to store duration as a double seconds, then it is easy to use in an expression. It is much harder to manipulate 96-bit duration and we almost never need that much precision.

DOUBLE = 1;

// A variable-length string.
STRING = 2;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please change STRING to use 0, we use the most popular one for 0. I think string should be more popular than INT64.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

IPV6 = 6;

// An email address.
EMAIL = 7;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's drop this for now and use string instead. Email is mutable user id. Use it for any real purpose is risky, we should discourage it.

IPV4 = 5;

// An IPv6 address.
IPV6 = 6;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's use IP_ADDRESS instead. We can't control which system uses which IP address, mixed IP will be common in many cases. We should force every component can always handle both.

// Precisely tracked quotas only allow a relatively modest level of
// scaling, whereas imprecise quotas can support a nearly unbounded
// level of scaling at the cost of potentially having inaccuracies.
bool precise = 9;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove this. We plan to drop this for Google services. When the traffic is too hard, we automatically degrade the precision. The reason is simple. This is a user option, and there is always human mistake. If a user noticed the traffic is too high, and wants to switch to in-precise, there is no way to support in a live system. Then the service will be broken.


// Quota limit expresses a maximum amount over a rolling time interval
RATE_LIMIT = 1;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is already implied by the quota unit, right?

// values are reset to 0 automatically at a fixed interval.
message QuotaDescriptor {
// The name of this descriptor.
string name = 1;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should drop this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Descriptors must have names so they can be referenced.

// * `NAME` is a sequence of non-blank printable ASCII characters not
// containing '{' or '}'.
//
string unit = 4;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can use a much simpler design. For example, a quota limit of 100 request per second per ip address, it can be expressed as.

int64 limit = 100;
repeated string dimensions = ["origin.ip"];
int32 interval = 1;

So all other fields in this message is not needed. We don't plan to use display_name and description anyway since they are not localized. The UI should automatically generate localized text based on limit, dimensions, interval.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I nuked the unit field.

string description = 3;

// Types of supported attribute values.
enum ValueType {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only way to interact with proxies and REST APIs is via HTTP header. So we want to make sure all values are expressible as strings. The expression engine will become insanely complicated as we add more types. We should really limit the choice as much as we can. We can always add more types later, but impossible to remove anything.

package istio.mixer.v1;

import "google/rpc/status.proto";
import "mixer/api/v1/attribute_update.proto";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be "mixer/api/v1/attributes.proto"; ?

package istio.mixer.v1;

import "google/rpc/status.proto";
import "mixer/api/v1/attribute_update.proto";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

attributes.proto ?

package istio.mixer.v1;

import "google/rpc/status.proto";
import "mixer/api/v1/attribute_update.proto";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

attributes.proto?


import "api/mixer/v1/check.proto";
import "api/mixer/v1/report.proto";
import "api/mixer/v1quota.proto";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing a '/' between 'v1' and 'quota.proto' ?

@wora
Copy link
Contributor

wora commented Dec 17, 2016

On general comment, attribute will become primitive types for the upcoming expression and policy language. The less types we support, the easier for implementation and more usable.

For example, if a component expects EMAIL attribute, you can't use STRING if everything is strongly typed (inconvenient), or we have to define various implicit conversion rules (complicated), or we have to cast everywhere (ugly). We should minimize features until we truly need them.


syntax = "proto3";

package istio.mixer.v1;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i feel like the packages should be mixer.api.v1 instead of istio.mixer.v1.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For Google APIs, we never use the term "api". When you have v1 in the package, it is pretty much an api. Adding "api" everywhere doesn't add much value, almost all protos are api anyway.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we be using mixer.v1 or is istio.mixer.v1 better?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing api is fine. However, these protos live in mixer/api/v1. It seems a bit appropriate to match that structure in the package.

In any case, I feel like "istio.mixer.v1" is not ideal. The generated go code will lead to some imports like "istio.io/istio/mixer/v1". I'd prefer something like "istio.io/mixer/v1".

We can address this later, once we switch the code over.

This clearly separates mixer's API and configuration surface from
the implementation. This is designed to make it possible to
replace the mixer in a fully compatible way.
@geeknoid geeknoid merged commit cf7f8a4 into istio:master Dec 17, 2016
@wora
Copy link
Contributor

wora commented Dec 19, 2016 via email

PER_SECOND_LIMIT = 1;

// Quota limit expresses a maximum amount over a rolling time interval
PER_MINUTE_LIMIT = 1;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why bother with per_second and per_minute ?

@louiscryan louiscryan mentioned this pull request Dec 14, 2017
incfly pushed a commit to incfly/api that referenced this pull request Jun 13, 2018
* Define interface for client.

* Remove per_request transport.

* Add quota interface.

* Update copyright headers.

* Update namespace from google to istio.
@istio-testing istio-testing changed the title Migrate mixer API and config to this repo. Automator: update common-files@master in istio/api@master Jan 24, 2020
@istio-testing istio-testing added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Jan 24, 2020
@clarketm clarketm changed the title Automator: update common-files@master in istio/api@master Migrate mixer API and config to this repo. Jan 24, 2020
nacx pushed a commit to nacx/api that referenced this pull request Apr 15, 2020
Signed-off-by: Dhi Aurrahman <[email protected]>
howardjohn pushed a commit that referenced this pull request Sep 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants