-
Notifications
You must be signed in to change notification settings - Fork 573
Migrate mixer API and config to this repo. #2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
I added a WORKSPACE file (which I just copied from the mixer repo) and I'm able to build the config directory, but the API directory fails due to some missing dependencies. Better, but still needs attention from someone that knows what they're doing with bazel... |
What if we don't bazel-ify this repo? We could potentially have the other repos define the build rules for various languages, etc.? @qiwzhang would that work for the proxy stuff? |
Where would I go to make sure my proto syntax is correct? I'd have to go
to the mixer repo for example? Seems a bit odd.
But you're the bazel guy, if that's what you want I'll nuke the BUILD files.
…On Fri, Dec 16, 2016 at 1:02 PM, Douglas Reid ***@***.***> wrote:
What if we don't bazel-ify this repo?
We could potentially have the other repos define the build rules for
various languages, etc.?
@qiwzhang <https://github.com/qiwzhang> would that work for the proxy
stuff?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#2 (comment)>, or mute the
thread
<https://github.com/notifications/unsubscribe-auth/AVucHcYo3_n6U2nYUwfB5CaLqzreuBRpks5rIvxLgaJpZM4LPg6Q>
.
|
Well, hopefully, PR reviews would catch the syntax issues... but that obviously has flaws. We could also add generated code here too (with a script for doing the generation) and require updates on any commit to both the source and gen'd libs. Why don't we either (a) commit without bazel now or (b) commit with the broken bits and I'll work with @qiwzhang to get a build setup working for the repo afterwards? |
OK, I'll commit as-is and then you can decide to update/delete the build
files.
…On Fri, Dec 16, 2016 at 1:26 PM, Douglas Reid ***@***.***> wrote:
Well, hopefully, PR reviews would catch the syntax issues... but that
obviously has flaws.
We could also add generated code here too (with a script for doing the
generation) and require updates on any commit to both the source and gen'd
libs.
Why don't we either (a) commit without bazel now or (b) commit with the
broken bits and I'll work with @qiwzhang <https://github.com/qiwzhang> to
get a build setup working for the repo afterwards?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#2 (comment)>, or mute the
thread
<https://github.com/notifications/unsubscribe-auth/AVucHRd8WYphVy6LSFgvXiIpzdDusMIUks5rIwH1gaJpZM4LPg6Q>
.
|
@@ -0,0 +1,112 @@ | |||
workspace(name = "com_github_istio_mixer") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
name this com_github_istio_api
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
"google/rpc/status.proto", | ||
], | ||
imports = [ | ||
"../../external/com_github_google_protobuf/src", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i think to use this setup, we need the .bazelrc file that adds the "--standalone" like bits (as this escapes the sandbox).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
@@ -0,0 +1 @@ | |||
/usr/local/google/home/mtail/.cache/bazel/_bazel_mtail/1875d8bd5ec708fecb970aa8fdf4f58a/execroot/com_github_istio_mixer |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we need a to add a git ignore entry for bazel-*
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
// | ||
// * Any deleted attributes are removed from the attribute context. | ||
// | ||
message AttributeUpdate { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This doesn't feel quite right to me. Why not just call this Attributes? And have the dictionary be understood to be context-dependent (either complete or overriding depending on whether or not a context is provided)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Renamed to Attributes.
Added more text about how the dictionary is used. Added the reset_context boolean to force everything to start from scratch. And added a few more comments here and there for clarity.
map<int32, string> email_attributes = 10; | ||
|
||
// Attributes that should be removed | ||
repeated int32 deleted_attributes = 11; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we expand on what it means to be removed? Removed from the set of AttributeValues and the Dictionary?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
// | ||
// Once a new dictionary has been sent over, it stays in effect until | ||
// a new dictionary is sent. | ||
map<string, int32> dictionary_update = 1; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
does this imply that the same attribute can be referred to by different IDs by different proxies, etc., across the system?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. It prevents us having to maintain any kind of registry of attribute ids. It's dynamic in nature, per connection to the mixer. As proxies are updated in a staged roll out and new attributes are added, then the dictionary is sent accordingly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ultimately I think it makes sense to reserve a block of ids for well-known things. This is what HTTP2 did, I would just add a TODO for the moment.
// The general idea is to leverage the stateful connection from the | ||
// proxy to the mixer to keep to a minimum the 'attribute chatter'. | ||
// Only delta attributes are sent over, multiple concurrent attribute | ||
// contexts can be used to avoid trashing, and attribute indices are used to |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
s/avoid trashing/avoid thrashing/ ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
string name = 1; | ||
|
||
// The set of attributes that are necessary to describe a log entry of this type. | ||
repeated string attributes = 2; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(here and elsewhere) to be clear, these are attribute descriptor names?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes.
// Name = IDENT { "." IDENT } ; | ||
// | ||
// Where `IDENT` must match the regular expression `[a-z][a-z0-9]+`. | ||
string name = 1; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we want to clarify anything about uniqueness of this name?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
DOne.
PTAL. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will like to Not have any bazel build files.
Other repos that use it can have bazel build files.
They will fail, they they update their "commit"
@@ -0,0 +1,57 @@ | |||
load("@org_pubref_rules_protobuf//go:rules.bzl", "go_proto_library") | |||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please don't put a BUILD file here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, they're gone.
|
||
// Attributes being updated within the specified attribute context. When reset_context is false, | ||
// then these maps are treated as deltas, so only what's changed needs to be included. | ||
map<int32, int64> int64_attributes = 4; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems like overkill I would just start with int64, string, boolean and maybe double if we have a case for it.
It seems odd to be mixing semantic types and data types here when we also have all the semantics in descriptors.
I would use string for ipaddress & email & int64 for timestamp, otherwise following this pattern we will end up with separate fields for DNS names, durations, GUIDs, ... and I don't think that's useful.
FWIW 'bytes' might be a better type than string for ip addresses and would cover both types adequately based on the length of the bytestring
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
// An instance of this is delivered to the mixer with every | ||
// API call. | ||
// | ||
// The general idea is to leverage the stateful connection from the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
s/connection/stream
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
// proxy to the mixer to keep to a minimum the 'attribute chatter'. | ||
// Only delta attributes are sent over, multiple concurrent attribute | ||
// contexts can be used to avoid thrashing, and attribute indices are used to | ||
// keep the wire protocol maximally efficient. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This sounds like a pretty complex protocol. Can we start with one implicit context and add disjoint contexts when we have evidence of need. After all multiple contexts can be handled by having multiple GRPC streams.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we just put the proto files here for now? I am blocked by this.
I vote to put the protos in, without bazel stuff, to unblock everyone. |
// | ||
// This method is intended for use by the Istio proxy and not by | ||
// individual services. | ||
rpc Check(stream CheckRequest) returns (stream CheckResponse) {} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Attribute context will span check & report all the time so having them be independent streams seems quite wasteful given all the effort put into dictionary management above. I would either:
1 - punt on dictionary management possibly in favor of a partially static dictionary (see HTTP2 https://http2.github.io/http2-spec/compression.html#static.table.definition)
2. - Allow the client to define the context in which the dictionary grain is useful and to choose which operation to perform in that context
The former seems a lot simpler than the latter frankly
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like the lack of special cases in this model, and I like the uniformity. All of the API methods work the same way, both on the server and client, so they'll just be some common code on both ends. The actual runtime overhead of multiple dictionaries is tiny both in CPU time and memory.
I'd like to keep this as-is and see how it pans out.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't strictly mind repeating the same dictionary or attribute context three times so that its available for each operation type for the context at hand. Need to document how to handle the failure mode when an attribute context cannot be allocated by the server, I suspect we should terminate the stream and reset everything and require the client to recover.
I'm fine with this, will add some clarifying doc comments.
string description = 3; | ||
|
||
// Types of supported attribute values. | ||
enum ValueType { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Continuing the earlier note about semantic types vs. data types, this enum is mixing both. I think we should have two enums
enum DataType {
INT, //as int64
FLOAT, //as double
BOOL,
STRING,
BYTES
}
enum ValueType {
LABEL,
URI.
DNS_NAME,
EMAIL,
TIMESTAMP,
IP_ADDR,
COUNTER,
GAUGE,
SUMMATION,
DURATION, // can standardize units here
}
This clarifies which fields in Attributes should be used for each descriptor field. It allows us to codify how coercion between DataTypes should occur for a specific ValueType, converting STRING to INT for a COUNTER
Example mappings to (data type, value type)
ipaddress -> (BYTES, IP_ADDR)
username -> (STRING, EMAIL)
k8s pob label -> (STRING, LABEL)
k8s service DNS -> (STRING, DNS_NAME)
system uptime -> (INT, DURATION)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is appealing. Let me think about it for a bit.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've been playing with this split and I'm not sure it's useful or desirable.
With my design, as new well-known and/or efficient storage formats are adopted, we just add a new value type for it. For example, Timestamp, Duration, IPv6Address, and likely more over time. With your approach, all those end up being encoded as raw bytes.
I think perhaps what threw you off is the choice of names. What if instead of what I had, we used instead:
enum ValueType {
// A generic 64-bit signed integer.
UNDISCRIMINATED_INT = 0;
// A 64-bit floating-point value.
UNDISCRIMINATED_FLOAT = 1;
// A variable-length string.
UNDISCRIMINATED_STRING = 2;
// Boolean; true or false.
UNDISCRIMINATED_BOOL = 3;
// A point in time, encoded as bytes or string
TIMESTAMP = 4;
// A quantity of time, encoded as bytes or string
DURATION = 5;
// An IP address, encoded as bytes or string
IP_ADDR = 6;
// An email address, encoded as a string
EMAIL_ADDR = 7;
// A URL, encoded as a string
URI = 8;
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Attributes will be used by selectors and policies. We need to know the types to apply expressions on them. That is the most important issue we need to cover. Splitting the value format from value type is attractive, but also adds more complexity. I suggest we don't pay for it initially.
I think the most useful feature needed in our use case is string, int, double, timestamp, ipaddress. Let's start with them first.
While proto does have duration type, most other systems don't have it. It would be much easier to store duration as a double seconds, then it is easy to use in an expression. It is much harder to manipulate 96-bit duration and we almost never need that much precision.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think in either form we'd have to handle migration to concrete schema types over time by coercion so I don't think that's a major difference really.
I do think the names threw me off so I'm OK with the single enum. Comments would have been enough to disambiguate I think so no need to make the names longer I think. What would be most useful would probably be examples.
INT -- Codes and enum like values such as HTTP status. You could argue these are POINT metrics too given that they can have thresholds applied to them, they just can't be aggregated
FLOAT -- I have a hard time coming up with an example that is not really a metric, lat/long maybe but those seem like they should be metrics with a type of POINT ? If we don't have a real use-case lets drop it and add it back?
STRING -- NAME or LABEL would be better choices for the the enum value?
BOOL -- I think its fairly self explanatory, no need to elaborate
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The only way to interact with proxies and REST APIs is via HTTP header. So we want to make sure all values are expressible as strings. The expression engine will become insanely complicated as we add more types. We should really limit the choice as much as we can. We can always add more types later, but impossible to remove anything.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done. I improved this somewhat. We'll tweak more as we learn where we'd like more discriminated typing and/or more efficient encodings.
string display_name = 8; | ||
|
||
// The kind of measurement. It describes how the data is reported. | ||
enum MetricKind { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, I see how you're intending to partition the semantic types. Probably want to put DURATION, SUMMATION and maybe TIMESTAMP in here
// for a quota of this type. | ||
repeated string dimension_attributes = 3; | ||
|
||
// The unit in which the quota's values are reported. The |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some examples here would be useful
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think that all these are istio configs, regardless of where they are used at present.
If it is something that a User sees and interacts with while configuring istio, It is istio config. For example, if this is how someone configures the quota aspect, It surely is istio config.
Unless something is purely between the proxy and mixer, it should be istio config.
Looks like all these descriptors will form a vocabulary of {attributes, quotas, ... }
At configuration time, you either define a quota or a rate limit inline, or use a reference from the vocabulary.
For majority of the config the user should be configuring istio. They should only need to configure individual software components sparingly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Where these types live in the repo is immaterial to what the user experiences, right?
For the programmer, these types exist to configure the mixer, so I prefer the definitions be closely coupled with the mixer.
Consider for example if these types change, the only code that needs to change is in the mixer. If we have multiple implementations of the mixer, they'll each need to deal with these configs. If we have multiple implementations of the proxy and manager, none of them will deal with these types.
// The default imposed maximum amount for values of this quota. | ||
int64 max_amount = 7; | ||
|
||
// The type of quota behavior expected, defaults to NORMAL if not specified |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
NORMAL not in enum
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
// Whether the quota's current value is tracked precisely or not. | ||
// Precisely tracked quotas only allow a relatively modest level of | ||
// scaling, whereas imprecise quotas can support a nearly unbounded | ||
// level of scaling at the cost of potentially having small |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would eliminate 'small'
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
// | ||
// **Basic units (UNIT)** | ||
// | ||
// * `bit` bit |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we really need schema for things like bytes? Is this just to facilitate display of aggregates in a standard fashion?
// | ||
// **Grammar** | ||
// | ||
// The grammar includes the dimensionless unit `1`, such as `1/s`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This seems like an odd example in the context of a metric assuming 1/s means '1 per second'.
I assume we just want to present either things where the unit is implied in the name, or things with units that are specified above which is currently limited to bytes & time. Is this worth the effort?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is all directly lifted from One Platform. I've no particular
attachment to this stuff if we want to simplify. We could just delete the
field completely for the time being.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removed the unit field completely.
// | ||
// The processing order for this state in the mixer is: | ||
// | ||
// * Any dictionary updates are applied |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it an 'update' or a replace, comment on line 56 indicates a replace. Phrasing...
'The new dictionary is installed if provided'
Also it's probably an API error for the first request not to install a dictionary.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
// * Any dictionary updates are applied | ||
// | ||
// * The requested attribute context is looked up. If no such context has been defined, a | ||
// new context is automatically created and initialized to the empty state. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add examples of the events that might cause a context to be created, e.g. a new socket is opened to a server, a new user logs in etc. Could go on the field instead.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
// * If reset_context is true, then the attribute context is reset to the | ||
// empty state. | ||
// | ||
// * Any attribute deltas are applied to the attribute context. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
'deltas applied' gives me the impression that aggregation occurs, 'overwrite' seems like a better term
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
// A dictionary that provides a mapping of attribute name to | ||
// a shorthand index value. | ||
// | ||
// This is intended to leverage the stateful connection from the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Be careful with the term 'connection' here. We're really using GRPCs notion of streams which might cause folks to get confused with TCP connections
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
// | ||
// Dictionaries are independent of the attribute context and are thus global | ||
// for the current gRPC stream. | ||
map<string, int32> dictionary_update = 1; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit. See note above about 'update'. Could just make the field name 'active_dictionary', 'new_dictionary' etc.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
// for the current gRPC stream. | ||
map<string, int32> dictionary_update = 1; | ||
|
||
// The attribute context against which the new attributes are applied. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The id of the attribute context...
|
||
// When true, resets the current attribute context to the empty state before | ||
// applying any new incoming attributes. | ||
bool reset_context = 3; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Consider providing guidance about when an attribute context should be reset so that the server can reclaim memory. E.g. If an HTTP2 connection terminated we should be able to reclaim the memory occupied by the context by re-using its id for a new connection later on.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
Makes sense. LGTM
…-louis (from phone)
On Dec 16, 2016 8:46 PM, "Martin Taillefer" <[email protected]> wrote:
PTAL.
I'd like to go ahead with this change as-is so we can unblock other work.
Subsequent adjustments can be done in follow-on PRs.
Thanks.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#2 (comment)>, or mute the
thread
<https://github.com/notifications/unsubscribe-auth/AIoKPGPOba8zlYw9X0nkO8fHaCZcCXnyks5rI2k3gaJpZM4LPg6Q>
.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please remove BUILD before merge. Otherwise, LGTM.
string description = 3; | ||
|
||
// Types of supported attribute values. | ||
enum ValueType { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Attributes will be used by selectors and policies. We need to know the types to apply expressions on them. That is the most important issue we need to cover. Splitting the value format from value type is attractive, but also adds more complexity. I suggest we don't pay for it initially.
I think the most useful feature needed in our use case is string, int, double, timestamp, ipaddress. Let's start with them first.
While proto does have duration type, most other systems don't have it. It would be much easier to store duration as a double seconds, then it is easy to use in an expression. It is much harder to manipulate 96-bit duration and we almost never need that much precision.
DOUBLE = 1; | ||
|
||
// A variable-length string. | ||
STRING = 2; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please change STRING to use 0, we use the most popular one for 0. I think string should be more popular than INT64.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
IPV6 = 6; | ||
|
||
// An email address. | ||
EMAIL = 7; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's drop this for now and use string instead. Email is mutable user id. Use it for any real purpose is risky, we should discourage it.
IPV4 = 5; | ||
|
||
// An IPv6 address. | ||
IPV6 = 6; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's use IP_ADDRESS instead. We can't control which system uses which IP address, mixed IP will be common in many cases. We should force every component can always handle both.
// Precisely tracked quotas only allow a relatively modest level of | ||
// scaling, whereas imprecise quotas can support a nearly unbounded | ||
// level of scaling at the cost of potentially having inaccuracies. | ||
bool precise = 9; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please remove this. We plan to drop this for Google services. When the traffic is too hard, we automatically degrade the precision. The reason is simple. This is a user option, and there is always human mistake. If a user noticed the traffic is too high, and wants to switch to in-precise, there is no way to support in a live system. Then the service will be broken.
|
||
// Quota limit expresses a maximum amount over a rolling time interval | ||
RATE_LIMIT = 1; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is already implied by the quota unit, right?
// values are reset to 0 automatically at a fixed interval. | ||
message QuotaDescriptor { | ||
// The name of this descriptor. | ||
string name = 1; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should drop this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Descriptors must have names so they can be referenced.
// * `NAME` is a sequence of non-blank printable ASCII characters not | ||
// containing '{' or '}'. | ||
// | ||
string unit = 4; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we can use a much simpler design. For example, a quota limit of 100 request per second per ip address, it can be expressed as.
int64 limit = 100;
repeated string dimensions = ["origin.ip"];
int32 interval = 1;
So all other fields in this message is not needed. We don't plan to use display_name and description anyway since they are not localized. The UI should automatically generate localized text based on limit, dimensions, interval.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I nuked the unit field.
string description = 3; | ||
|
||
// Types of supported attribute values. | ||
enum ValueType { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The only way to interact with proxies and REST APIs is via HTTP header. So we want to make sure all values are expressible as strings. The expression engine will become insanely complicated as we add more types. We should really limit the choice as much as we can. We can always add more types later, but impossible to remove anything.
package istio.mixer.v1; | ||
|
||
import "google/rpc/status.proto"; | ||
import "mixer/api/v1/attribute_update.proto"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should this be "mixer/api/v1/attributes.proto"; ?
package istio.mixer.v1; | ||
|
||
import "google/rpc/status.proto"; | ||
import "mixer/api/v1/attribute_update.proto"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
attributes.proto ?
package istio.mixer.v1; | ||
|
||
import "google/rpc/status.proto"; | ||
import "mixer/api/v1/attribute_update.proto"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
attributes.proto?
|
||
import "api/mixer/v1/check.proto"; | ||
import "api/mixer/v1/report.proto"; | ||
import "api/mixer/v1quota.proto"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
missing a '/' between 'v1' and 'quota.proto' ?
On general comment, attribute will become primitive types for the upcoming expression and policy language. The less types we support, the easier for implementation and more usable. For example, if a component expects EMAIL attribute, you can't use STRING if everything is strongly typed (inconvenient), or we have to define various implicit conversion rules (complicated), or we have to cast everywhere (ugly). We should minimize features until we truly need them. |
|
||
syntax = "proto3"; | ||
|
||
package istio.mixer.v1; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i feel like the packages should be mixer.api.v1 instead of istio.mixer.v1.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For Google APIs, we never use the term "api". When you have v1 in the package, it is pretty much an api. Adding "api" everywhere doesn't add much value, almost all protos are api anyway.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we be using mixer.v1 or is istio.mixer.v1 better?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removing api is fine. However, these protos live in mixer/api/v1. It seems a bit appropriate to match that structure in the package.
In any case, I feel like "istio.mixer.v1" is not ideal. The generated go code will lead to some imports like "istio.io/istio/mixer/v1". I'd prefer something like "istio.io/mixer/v1".
We can address this later, once we switch the code over.
This clearly separates mixer's API and configuration surface from the implementation. This is designed to make it possible to replace the mixer in a fully compatible way.
I think the istio.mixer.v1 is better choice. We should not care what it
looks like in Go. If we design the API as proto, we can't make it happy
with every programming language.
We can not own the top level proto package mixer, it implies we own entire
proto namespace. Everything shall be under istio package name, so we can
accept other people's contribution.
|
PER_SECOND_LIMIT = 1; | ||
|
||
// Quota limit expresses a maximum amount over a rolling time interval | ||
PER_MINUTE_LIMIT = 1; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why bother with per_second and per_minute ?
* Define interface for client. * Remove per_request transport. * Add quota interface. * Update copyright headers. * Update namespace from google to istio.
Signed-off-by: Dhi Aurrahman <[email protected]>
Add ambient mode to meshconfig
This clearly separates mixer's API and configuration surface from
the implementation. This is designed to make it possible to
replace the mixer in a fully compatible way.
Note that I put the config into mixer/config instead of istio/config since it is in fact all specific to the mixer. These configuration definitions aren't used by the proxy or manager themselves.
Note to Doug: this clearly doesn't build as it needs the bazel treatment (mainly a WORKSPACE file I think). I hope you can supply this after this initial checkin.