Refined and documented error handling for functions

fluent.runtime/CHANGELOG.rst

@@ -6,6 +6,8 @@ fluent.runtime development version (unreleased)
 
 * Support for Fluent spec 0.8 (``fluent.syntax`` 0.10), including parameterized
   terms.
+* Refined error handling regarding function calls to be more tolerant of errors
+  in FTL files, while silencing developer errors less.
 
 fluent.runtime 0.1 (January 21, 2019)
 -------------------------------------

fluent.runtime/docs/errors.rst

@@ -0,0 +1,43 @@
+Error handling
+==============
+
+The Fluent philosophy is to try to recover from errors, and not throw
+exceptions, on the basis that a partial translation is usually better than one
+that is entirely missing or a 500 page.
+
+python-fluent adopts that philosophy, but also tries to abide by the Zen of
+Python - “Errors should never pass silently. Unless explicitly silenced.”
+
+The combination of these two different philosophies works as follows:
+
+* Errors made by **translators** in the contents of FTL files do not raise
+  exceptions. Instead the errors are collected in the ``errors`` argument returned
+  by ``FluentBundle.format``, and some kind of substitute string is returned.
+  For example, if a non-existent term ``-brand-name`` is referenced from a
+  message, the string ``-brand-name`` is inserted into the returned string.
+
+  Also, if the translator uses a function and passes the wrong number of
+  positional arguments, or unavailable keyword arguments, this error will be
+  caught and reported, without allowing the exception to propagate.
+
+* Exceptions triggered by **developer** errors (whether the authors of
+  python-fluent or a user of python-fluent) are not caught, but are allowed to
+  propagate. For example:
+
+  * An incorrect message ID passed to ``FluentBundle.format`` is most likely a
+    developer error (a typo in the message ID), and so causes an exception to be
+    raised.
+
+    A message ID that is correct but missing in some languages will cause the
+    same error, but it is expected that to cover this eventuality
+    ``FluentBundle.format`` will be wrapped with functions that automatically
+    perform fallback to languages that have all messages defined. This fallback
+    mechanism is outside the scope of ``fluent.runtime`` itself.
+
+  * Message arguments of unexpected types will raise exceptions, since it is the
+    developer's job to ensure the right arguments are being passed to the
+    ``FluentBundle.format`` method.
+
+  * Exceptions raised by custom functions are also assumed to be developer
+    errors (as documented in :doc:`functions`, these functions should not raise
+    exceptions), and are not caught.

fluent.runtime/docs/functions.rst

@@ -0,0 +1,84 @@
+Custom functions
+----------------
+
+You can add functions to the ones available to FTL authors by passing a
+``functions`` dictionary to the ``FluentBundle`` constructor:
+
+.. code-block:: python
+
+    >>> import platform
+    >>> def os_name():
+    ...    """Returns linux/mac/windows/other"""
+    ...    return {'Linux': 'linux',
+    ...            'Darwin': 'mac',
+    ...            'Windows': 'windows'}.get(platform.system(), 'other')
+
+    >>> bundle = FluentBundle(['en-US'], functions={'OS': os_name})
+    >>> bundle.add_messages("""
+    ... welcome = { OS() ->
+    ...    [linux]    Welcome to Linux
+    ...    [mac]      Welcome to Mac
+    ...    [windows]  Welcome to Windows
+    ...   *[other]    Welcome
+    ...   }
+    ... """)
+    >>> print(bundle.format('welcome')[0]
+    Welcome to Linux
+
+These functions can accept positional and keyword arguments, like the ``NUMBER``
+and ``DATETIME`` builtins. They must accept the following types of objects
+passed as arguments:
+
+- unicode strings (i.e. ``unicode`` on Python 2, ``str`` on Python 3)
+- ``fluent.runtime.types.FluentType`` subclasses, namely:
+
+  - ``FluentNumber`` - ``int``, ``float`` or ``Decimal`` objects passed in
+    externally, or expressed as literals, are wrapped in these. Note that these
+    objects also subclass builtin ``int``, ``float`` or ``Decimal``, so can be
+    used as numbers in the normal way.
+  - ``FluentDateType`` - ``date`` or ``datetime`` objects passed in are wrapped in
+    these. Again, these classes also subclass ``date`` or ``datetime``, and can
+    be used as such.
+  - ``FluentNone`` - in error conditions, such as a message referring to an
+    argument that hasn't been passed in, objects of this type are passed in.
+
+Custom functions should not throw errors, but return ``FluentNone`` instances to
+indicate an error or missing data. Otherwise they should return unicode strings,
+or instances of a ``FluentType`` subclass as above. Returned numbers and
+datetimes should be converted to ``FluentNumber`` or ``FluentDateType``
+subclasses using ``fluent.types.fluent_number`` and ``fluent.types.fluent_date``
+respectively.
+
+The type signatures of custom functions are checked before they are used, to
+ensure the right the number of positional arguments are used, and only available
+keyword arguments are used - otherwise a ``TypeError`` will be appended to the
+``errors`` list. Using ``*args`` or ``**kwargs`` to allow any number of
+positional or keyword arguments is supported, but you should ensure that your
+function actually does allow all positional or keyword arguments.
+
+If you want to override the detected type signature (for example, to limit the
+arguments that can be used in an FTL file, or to provide a proper signature for
+a function that has a signature using ``*args`` and ``**kwargs`` but is more
+restricted in reality), you can add an ``ftl_arg_spec`` attribute to the
+function. The value should be a two-tuple containing 1) an integer specifying
+the number of positional arguments, and 2) a list of allowed keyword arguments.
+For example, for a custom function ``my_func`` the following will stop the
+``restricted`` keyword argument from being used from FTL files, while allowing
+``allowed``, and will require that a single positional argument is passed:
+
+.. code-block:: python
+
+    def my_func(arg1, allowed=None, restricted=None):
+        pass
+
+    my_func.ftl_arg_spec = (1, ['allowed'])
+
+The Fluent spec allows keyword arguments with hyphens (``-``) in them. These are
+not valid identifiers in Python, so if you need to a custom function to accept
+keyword arguments like this, you will have to use ``**kwargs`` syntax e.g.:
+
+    def my_func(kwarg1=None, **kwargs):
+        kwarg_with_hyphens = kwargs.pop('kwarg-with-hyphens', None)
+        # etc.
+
+    my_func.ftl_arg_spec = (0, ['kwarg1', 'kwarg-with-hyphens'])

fluent.runtime/docs/index.rst

@@ -15,4 +15,6 @@ significant changes.
 
    installation
    usage
+   functions
+   errors
    history

fluent.runtime/docs/usage.rst

@@ -202,54 +202,6 @@ ways:
        >>> val
        'Now is Jun 17, 2018, 3:15:05 PM'
 
-Custom functions
-~~~~~~~~~~~~~~~~
-
-You can add functions to the ones available to FTL authors by passing a
-``functions`` dictionary to the ``FluentBundle`` constructor:
-
-.. code-block:: python
-
-    >>> import platform
-    >>> def os_name():
-    ...    """Returns linux/mac/windows/other"""
-    ...    return {'Linux': 'linux',
-    ...            'Darwin': 'mac',
-    ...            'Windows': 'windows'}.get(platform.system(), 'other')
-
-    >>> bundle = FluentBundle(['en-US'], functions={'OS': os_name})
-    >>> bundle.add_messages("""
-    ... welcome = { OS() ->
-    ...    [linux]    Welcome to Linux
-    ...    [mac]      Welcome to Mac
-    ...    [windows]  Welcome to Windows
-    ...   *[other]    Welcome
-    ...   }
-    ... """)
-    >>> print(bundle.format('welcome')[0]
-    Welcome to Linux
-
-These functions can accept positional and keyword arguments (like the
-``NUMBER`` and ``DATETIME`` builtins), and in this case must accept the
-following types of arguments:
-
--  unicode strings (i.e. ``unicode`` on Python 2, ``str`` on Python 3)
--  ``fluent.runtime.types.FluentType`` subclasses, namely:
--  ``FluentNumber`` - ``int``, ``float`` or ``Decimal`` objects passed
-   in externally, or expressed as literals, are wrapped in these. Note
-   that these objects also subclass builtin ``int``, ``float`` or
-   ``Decimal``, so can be used as numbers in the normal way.
--  ``FluentDateType`` - ``date`` or ``datetime`` objects passed in are
-   wrapped in these. Again, these classes also subclass ``date`` or
-   ``datetime``, and can be used as such.
--  ``FluentNone`` - in error conditions, such as a message referring to
-   an argument that hasn't been passed in, objects of this type are
-   passed in.
-
-Custom functions should not throw errors, but return ``FluentNone``
-instances to indicate an error or missing data. Otherwise they should
-return unicode strings, or instances of a ``FluentType`` subclass as
-above.
 
 Known limitations and bugs
 ~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -268,3 +220,10 @@ Known limitations and bugs
   <https://github.com/andyearnshaw/Intl.js/blob/master/src/12.datetimeformat.js>`_.
 
 Help with the above would be welcome!
+
+
+Other features and further information
+--------------------------------------
+
+* :doc:`functions`
+* :doc:`errors`

fluent.runtime/fluent/runtime/resolver.py

@@ -1,17 +1,15 @@
 from __future__ import absolute_import, unicode_literals
 
 import contextlib
-from datetime import date, datetime
-from decimal import Decimal
 
 import attr
 import six
 
 from fluent.syntax import ast as FTL
-from .errors import FluentCyclicReferenceError, FluentFormatError, FluentReferenceError
-from .types import FluentType, FluentNone, FluentInt, FluentFloat
-from .utils import reference_to_id, unknown_reference_error_obj
 
+from .errors import FluentCyclicReferenceError, FluentFormatError, FluentReferenceError
+from .types import FluentFloat, FluentInt, FluentNone, FluentType
+from .utils import args_match, inspect_function_args, reference_to_id, unknown_reference_error_obj
 
 """
 The classes in this module are used to transform the source
@@ -345,11 +343,12 @@ def __call__(self, env):
                                                    .format(function_name)))
             return FluentNone(function_name + "()")
 
-        try:
-            return function(*args, **kwargs)
-        except Exception as e:
-            env.errors.append(e)
-            return FluentNoneResolver(function_name + "()")
+        arg_spec = inspect_function_args(function, function_name, env.errors)
+        match, sanitized_args, sanitized_kwargs, errors = args_match(function_name, args, kwargs, arg_spec)
+        env.errors.extend(errors)
+        if match:
+            return function(*sanitized_args, **sanitized_kwargs)
+        return FluentNone(function_name + "()")
 
 
 class NamedArgument(FTL.NamedArgument, BaseResolver):

fluent.runtime/fluent/runtime/types.py

@@ -67,6 +67,8 @@ class NumberFormatOptions(object):
     # rather than using underscores as per PEP8, so that
     # we can stick to Fluent spec more easily.
 
+    # Keyword args available to FTL authors must be synced to fluent_number.ftl_arg_spec below
+
     # See https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/NumberFormat
     style = attr.ib(default=FORMAT_STYLE_DECIMAL,
                     validator=attr.validators.in_(FORMAT_STYLE_OPTIONS))
@@ -228,10 +230,24 @@ def fluent_number(number, **kwargs):
     elif isinstance(number, FluentNone):
         return number
     else:
-        raise TypeError("Can't use fluent_number with object {0} for type {1}"
+        raise TypeError("Can't use fluent_number with object {0} of type {1}"
                         .format(number, type(number)))
 
 
+# Specify arg spec manually, for three reasons:
+# 1. To avoid having to specify kwargs explicitly, which results
+#    in duplication, and in unnecessary work inside FluentNumber
+# 2. To stop 'style' and 'currency' being used inside FTL files
+# 3. To avoid needing inspection to do this work.
+fluent_number.ftl_arg_spec = (1, ['currencyDisplay',
+                                  'useGrouping',
+                                  'minimumIntegerDigits',
+                                  'minimumFractionDigits',
+                                  'maximumFractionDigits',
+                                  'minimumSignificantDigits',
+                                  'maximumSignificantDigits'])
+
+
 _UNGROUPED_PATTERN = parse_pattern("#0")
 
 
@@ -255,6 +271,8 @@ class DateFormatOptions(object):
     timeZone = attr.ib(default=None)
 
     # Other
+    # Keyword args available to FTL authors must be synced to fluent_date.ftl_arg_spec below
+
     hour12 = attr.ib(default=None)
     weekday = attr.ib(default=None)
     era = attr.ib(default=None)
@@ -361,3 +379,19 @@ def fluent_date(dt, **kwargs):
     else:
         raise TypeError("Can't use fluent_date with object {0} of type {1}"
                         .format(dt, type(dt)))
+
+
+fluent_date.ftl_arg_spec = (1,
+                            ['hour12',
+                             'weekday',
+                             'era',
+                             'year',
+                             'month',
+                             'day',
+                             'hour',
+                             'minute',
+                             'second',
+                             'timeZoneName',
+                             'dateStyle',
+                             'timeStyle',
+                             ])

fluent.runtime/fluent/runtime/utils.py

@@ -1,17 +1,43 @@
 from __future__ import absolute_import, unicode_literals
 
+import inspect
+import keyword
+import re
+import sys
 from datetime import date, datetime
 from decimal import Decimal
 
+import six
+
 from fluent.syntax.ast import AttributeExpression, Term, TermReference
 
-from .types import FluentInt, FluentFloat, FluentDecimal, FluentDate, FluentDateTime
-from .errors import FluentReferenceError
+from .errors import FluentFormatError, FluentReferenceError
+from .types import FluentDate, FluentDateTime, FluentDecimal, FluentFloat, FluentInt
 
 TERM_SIGIL = '-'
 ATTRIBUTE_SEPARATOR = '.'
 
 
+class Any(object):
+    pass
+
+
+Any = Any()
+
+
+# From spec:
+#    NamedArgument ::= Identifier blank? ":" blank? (StringLiteral | NumberLiteral)
+#    Identifier ::= [a-zA-Z] [a-zA-Z0-9_-]*
+
+NAMED_ARG_RE = re.compile(r'^[a-zA-Z][a-zA-Z0-9_-]*$')
+
+
+def allowable_keyword_arg_name(name):
+    # We limit to what Fluent allows for NamedArgument - Python allows anything
+    # if you use **kwarg call and receiving syntax.
+    return NAMED_ARG_RE.match(name)
+
+
 def ast_to_id(ast):
     """
     Returns a string reference for a Term or Message
@@ -21,6 +47,107 @@ def ast_to_id(ast):
     return ast.id.name
 
 
+if sys.version_info < (3,):
+    # Python 3 has builtin str.isidentifier method, for Python 2 we refer to
+    # https://docs.python.org/2/reference/lexical_analysis.html#identifiers
+    identifer_re = re.compile('^[a-zA-Z_][a-zA-Z0-9_]*$')
+
+    def allowable_name(ident, for_method=False, allow_builtin=False):
+        """
+        Determines if argument is valid to be used as Python name/identifier.
+        If for_method=True is passed, checks whether it can be used as a method name
+        If allow_builtin=True is passed, names of builtin functions can be used.
+        """
+
+        if keyword.iskeyword(ident):
+            return False
+
+        # For methods, there is no clash with builtins so we have looser checks.
+        # We also sometimes want to be able to use builtins (e.g. when calling
+        # them), so need an exception for that. Otherwise we want to eliminate
+        # the possibility of shadowing things like 'True' or 'str' that are
+        # technically valid identifiers.
+
+        if not (for_method or allow_builtin):
+            if ident in six.moves.builtins.__dict__:
+                return False
+
+        if not identifer_re.match(ident):
+            return False
+
+        return True
+
+else:
+    def allowable_name(ident, for_method=False, allow_builtin=False):
+
+        if keyword.iskeyword(ident):
+            return False
+
+        if not (for_method or allow_builtin):
+            if ident in six.moves.builtins.__dict__:
+                return False
+
+        if not ident.isidentifier():
+            return False
+
+        return True
+
+
+if hasattr(inspect, 'signature'):
+    def inspect_function_args(function, name, errors):
+        """
+        For a Python function, returns a 2 tuple containing:
+        (number of positional args or Any,
+        set of keyword args or Any)
+
+        Keyword args are defined as those with default values.
+        'Keyword only' args with no default values are not supported.
+        """
+        if hasattr(function, 'ftl_arg_spec'):
+            return sanitize_function_args(function.ftl_arg_spec, name, errors)
+        sig = inspect.signature(function)
+        parameters = list(sig.parameters.values())
+
+        positional = (
+            Any if any(p.kind == inspect.Parameter.VAR_POSITIONAL
+                       for p in parameters)
+            else len(list(p for p in parameters
+                          if p.default == inspect.Parameter.empty and
+                          p.kind == inspect.Parameter.POSITIONAL_OR_KEYWORD)))
+
+        keywords = (
+            Any if any(p.kind == inspect.Parameter.VAR_KEYWORD
+                       for p in parameters)
+            else [p.name for p in parameters
+                  if p.default != inspect.Parameter.empty])
+        return sanitize_function_args((positional, keywords), name, errors)
+else:
+    def inspect_function_args(function, name, errors):
+        """
+        For a Python function, returns a 2 tuple containing:
+        (number of positional args or Any,
+        set of keyword args or Any)
+
+        Keyword args are defined as those with default values.
+        'Keyword only' args with no default values are not supported.
+        """
+        if hasattr(function, 'ftl_arg_spec'):
+            return sanitize_function_args(function.ftl_arg_spec, name, errors)
+        args = inspect.getargspec(function)
+
+        num_defaults = 0 if args.defaults is None else len(args.defaults)
+        positional = (
+            Any if args.varargs is not None
+            else len(args.args) - num_defaults
+        )
+
+        keywords = (
+            Any if args.keywords is not None
+            else ([] if num_defaults == 0 else args.args[-num_defaults:])
+        )
+        return sanitize_function_args((positional, keywords), name, errors)
+
+
 def native_to_fluent(val):
     """
     Convert a python type to a Fluent Type.
@@ -39,6 +166,49 @@ def native_to_fluent(val):
     return val
 
 
+def args_match(function_name, args, kwargs, arg_spec):
+    """
+    Checks the passed in args/kwargs against the function arg_spec
+    and returns data for calling the function correctly.
+
+    Return value is a tuple
+
+    (match, santized args, santized keyword args, errors)
+
+    match is False if the function should not be called at all.
+
+    """
+    # For the errors returned, we try to match the TypeError raised by Python
+    # when calling functions with wrong arguments, for the sake of something
+    # recognisable.
+    errors = []
+    sanitized_kwargs = {}
+    positional_arg_count, allowed_kwargs = arg_spec
+    match = True
+    for kwarg_name, kwarg_val in kwargs.items():
+        if ((allowed_kwargs is Any and allowable_keyword_arg_name(kwarg_name)) or
+                (allowed_kwargs is not Any and kwarg_name in allowed_kwargs)):
+            sanitized_kwargs[kwarg_name] = kwarg_val
+        else:
+            errors.append(
+                TypeError("{0}() got an unexpected keyword argument '{1}'"
+                          .format(function_name, kwarg_name)))
+    if positional_arg_count is Any:
+        sanitized_args = args
+    else:
+        sanitized_args = tuple(args[0:positional_arg_count])
+        len_args = len(args)
+        if len_args > positional_arg_count:
+            errors.append(TypeError("{0}() takes {1} positional arguments but {2} were given"
+                                    .format(function_name, positional_arg_count, len_args)))
+        elif len_args < positional_arg_count:
+            errors.append(TypeError("{0}() takes {1} positional arguments but {2} were given"
+                                    .format(function_name, positional_arg_count, len_args)))
+            match = False
+
+    return (match, sanitized_args, sanitized_kwargs, errors)
+
+
 def reference_to_id(ref):
     """
     Returns a string reference for a MessageReference, TermReference or AttributeExpression
@@ -58,6 +228,24 @@ def reference_to_id(ref):
     return ref.id.name
 
 
+def sanitize_function_args(arg_spec, name, errors):
+    """
+    Check function arg spec is legitimate, returning a cleaned
+    up version, and adding any errors to errors list.
+    """
+    positional_args, keyword_args = arg_spec
+    if keyword_args is Any:
+        cleaned_kwargs = keyword_args
+    else:
+        cleaned_kwargs = []
+        for kw in keyword_args:
+            if allowable_keyword_arg_name(kw):
+                cleaned_kwargs.append(kw)
+            else:
+                errors.append(FluentFormatError("{0}() has invalid keyword argument name '{1}'".format(name, kw)))
+    return (positional_args, cleaned_kwargs)
+
+
 def unknown_reference_error_obj(ref_id):
     if ATTRIBUTE_SEPARATOR in ref_id:
         return FluentReferenceError("Unknown attribute: {0}".format(ref_id))

fluent.runtime/tests/format/test_builtins.py

@@ -20,9 +20,10 @@ def setUp(self):
             implicit-call2   = { $arg }
             defaults         = { NUMBER(123456) }
             percent-style    = { NUMBER(1.234, style: "percent") }
-            currency-style   = { NUMBER(123456, style: "currency", currency: "USD") }
             from-arg         = { NUMBER($arg) }
             merge-params     = { NUMBER($arg, useGrouping: 0) }
+            bad-kwarg        = { NUMBER(1, badkwarg: 0) }
+            bad-arity        = { NUMBER(1, 2) }
         """))
 
     def test_implicit_call(self):
@@ -50,14 +51,15 @@ def test_defaults(self):
         self.assertEqual(val, "123,456")
         self.assertEqual(len(errs), 0)
 
-    def test_percent_style(self):
+    def test_style_in_ftl(self):
+        # style is only allowed as developer option
         val, errs = self.ctx.format('percent-style', {})
-        self.assertEqual(val, "123%")
-        self.assertEqual(len(errs), 0)
+        self.assertEqual(val, "1.234")
+        self.assertEqual(len(errs), 1)
 
-    def test_currency_style(self):
-        val, errs = self.ctx.format('currency-style', {})
-        self.assertEqual(val, "$123,456.00")
+    def test_percent_style(self):
+        val, errs = self.ctx.format('from-arg', {'arg': fluent_number(1.234, style="percent")})
+        self.assertEqual(val, "123%")
         self.assertEqual(len(errs), 0)
 
     def test_from_arg_int(self):
@@ -95,6 +97,18 @@ def test_merge_params(self):
         self.assertEqual(val, "$123456.78")
         self.assertEqual(len(errs), 0)
 
+    def test_bad_kwarg(self):
+        val, errs = self.ctx.format('bad-kwarg')
+        self.assertEqual(val, "1")
+        self.assertEqual(len(errs), 1)
+        self.assertEqual(type(errs[0]), TypeError)
+
+    def test_bad_arity(self):
+        val, errs = self.ctx.format('bad-arity')
+        self.assertEqual(val, "1")
+        self.assertEqual(len(errs), 1)
+        self.assertEqual(type(errs[0]), TypeError)
+
 
 class TestDatetimeBuiltin(unittest.TestCase):
 

fluent.runtime/tests/format/test_functions.py

@@ -2,28 +2,74 @@
 
 import unittest
 
+import six
+
 from fluent.runtime import FluentBundle
 from fluent.runtime.errors import FluentReferenceError
-from fluent.runtime.types import FluentNone
+from fluent.runtime.types import FluentNone, fluent_number
 
 from ..utils import dedent_ftl
 
 
 class TestFunctionCalls(unittest.TestCase):
 
     def setUp(self):
-        self.ctx = FluentBundle(['en-US'], use_isolating=False,
-                                  functions={'IDENTITY': lambda x: x})
+        def IDENTITY(x):
+            return x
+
+        def WITH_KEYWORD(x, y=0):
+            return six.text_type(x + y)
+
+        def RUNTIME_ERROR(x):
+            return 1/0
+
+        def RUNTIME_TYPE_ERROR(arg):
+            return arg + 1
+
+        def ANY_ARGS(*args, **kwargs):
+            return (' '.join(map(six.text_type, args)) + " " +
+                    ' '.join("{0}={1}".format(k, v) for k, v in sorted(kwargs.items())))
+
+        def RESTRICTED(allowed=None, notAllowed=None):
+            return allowed if allowed is not None else 'nothing passed'
+
+        def BAD_OUTPUT():
+            class Unsupported(object):
+                pass
+            return Unsupported()
+
+        RESTRICTED.ftl_arg_spec = (0, ['allowed'])
+
+        self.ctx = FluentBundle(
+            ['en-US'], use_isolating=False,
+            functions={'IDENTITY': IDENTITY,
+                       'WITH_KEYWORD': WITH_KEYWORD,
+                       'RUNTIME_ERROR': RUNTIME_ERROR,
+                       'RUNTIME_TYPE_ERROR': RUNTIME_TYPE_ERROR,
+                       'ANY_ARGS': ANY_ARGS,
+                       'RESTRICTED': RESTRICTED,
+                       'BAD_OUTPUT': BAD_OUTPUT,
+                       })
         self.ctx.add_messages(dedent_ftl("""
             foo = Foo
                 .attr = Attribute
-            pass-nothing       = { IDENTITY() }
             pass-string        = { IDENTITY("a") }
             pass-number        = { IDENTITY(1) }
             pass-message       = { IDENTITY(foo) }
             pass-attr          = { IDENTITY(foo.attr) }
             pass-external      = { IDENTITY($ext) }
             pass-function-call = { IDENTITY(IDENTITY(1)) }
+            too-few-pos-args   = { IDENTITY() }
+            too-many-pos-args  = { IDENTITY(2, 3, 4) }
+            use-good-kwarg     = { WITH_KEYWORD(1, y: 1) }
+            use-bad-kwarg      = { WITH_KEYWORD(1, bad: 1) }
+            runtime-error      = { RUNTIME_ERROR(1) }
+            runtime-type-error = { RUNTIME_TYPE_ERROR("hello") }
+            use-any-args       = { ANY_ARGS(1, 2, 3, x:1) }
+            use-restricted-ok  = { RESTRICTED(allowed: 1) }
+            use-restricted-bad = { RESTRICTED(notAllowed: 1) }
+            bad-output         = { BAD_OUTPUT() }
+            non-identfier-arg  = { ANY_ARGS(1, foo: 2, non-identifier: 3) }
         """))
 
     def test_accepts_strings(self):
@@ -56,12 +102,66 @@ def test_accepts_function_calls(self):
         self.assertEqual(val, "1")
         self.assertEqual(len(errs), 0)
 
-    def test_wrong_arity(self):
-        val, errs = self.ctx.format('pass-nothing', {})
+    def test_too_few_pos_args(self):
+        val, errs = self.ctx.format('too-few-pos-args', {})
         self.assertEqual(val, "IDENTITY()")
         self.assertEqual(len(errs), 1)
         self.assertEqual(type(errs[0]), TypeError)
 
+    def test_too_many_pos_args(self):
+        val, errs = self.ctx.format('too-many-pos-args', {})
+        self.assertEqual(val, "2")
+        self.assertEqual(len(errs), 1)
+        self.assertEqual(type(errs[0]), TypeError)
+
+    def test_good_kwarg(self):
+        val, errs = self.ctx.format('use-good-kwarg')
+        self.assertEqual(val, "2")
+        self.assertEqual(len(errs), 0)
+
+    def test_bad_kwarg(self):
+        val, errs = self.ctx.format('use-bad-kwarg')
+        self.assertEqual(val, "1")
+        self.assertEqual(len(errs), 1)
+        self.assertEqual(type(errs[0]), TypeError)
+
+    def test_runtime_error(self):
+        self.assertRaises(ZeroDivisionError,
+                          self.ctx.format,
+                          'runtime-error')
+
+    def test_runtime_type_error(self):
+        self.assertRaises(TypeError,
+                          self.ctx.format,
+                          'runtime-type-error')
+
+    def test_use_any_args(self):
+        val, errs = self.ctx.format('use-any-args')
+        self.assertEqual(val, "1 2 3 x=1")
+        self.assertEqual(len(errs), 0)
+
+    def test_restricted_ok(self):
+        val, errs = self.ctx.format('use-restricted-ok')
+        self.assertEqual(val, "1")
+        self.assertEqual(len(errs), 0)
+
+    def test_restricted_bad(self):
+        val, errs = self.ctx.format('use-restricted-bad')
+        self.assertEqual(val, "nothing passed")
+        self.assertEqual(len(errs), 1)
+        self.assertEqual(type(errs[0]), TypeError)
+
+    def test_bad_output(self):
+        # This is a developer error, so should raise an exception
+        with self.assertRaises(TypeError) as cm:
+            self.ctx.format('bad-output')
+        self.assertIn("Unsupported", cm.exception.args[0])
+
+    def test_non_identifier_python_keyword_args(self):
+        val, errs = self.ctx.format('non-identfier-arg')
+        self.assertEqual(val, '1 foo=2 non-identifier=3')
+        self.assertEqual(len(errs), 0)
+
 
 class TestMissing(unittest.TestCase):
 
@@ -87,9 +187,10 @@ def number_processor(number):
             self.args_passed.append(number)
             return number
 
-        self.ctx = FluentBundle(['en-US'], use_isolating=False,
-                                  functions={'NUMBER_PROCESSOR':
-                                             number_processor})
+        self.ctx = FluentBundle(
+            ['en-US'], use_isolating=False,
+            functions={'NUMBER_PROCESSOR':
+                       number_processor})
 
         self.ctx.add_messages(dedent_ftl("""
             pass-number = { NUMBER_PROCESSOR(1) }
@@ -101,12 +202,14 @@ def test_args_passed_as_numbers(self):
         self.assertEqual(val, "1")
         self.assertEqual(len(errs), 0)
         self.assertEqual(self.args_passed, [1])
+        self.assertEqual(self.args_passed, [fluent_number(1)])
 
     def test_literals_passed_as_numbers(self):
         val, errs = self.ctx.format('pass-number', {})
         self.assertEqual(val, "1")
         self.assertEqual(len(errs), 0)
         self.assertEqual(self.args_passed, [1])
+        self.assertEqual(self.args_passed, [fluent_number(1)])
 
 
 class TestKeywordArgs(unittest.TestCase):
@@ -118,8 +221,9 @@ def my_function(arg, kwarg1=None, kwarg2="default"):
             self.args_passed.append((arg, kwarg1, kwarg2))
             return arg
 
-        self.ctx = FluentBundle(['en-US'], use_isolating=False,
-                                  functions={'MYFUNC': my_function})
+        self.ctx = FluentBundle(
+            ['en-US'], use_isolating=False,
+            functions={'MYFUNC': my_function})
         self.ctx.add_messages(dedent_ftl("""
             pass-arg        = { MYFUNC("a") }
             pass-kwarg1     = { MYFUNC("a", kwarg1: 1) }

fluent.runtime/tests/test_utils.py

@@ -0,0 +1,43 @@
+from __future__ import absolute_import, unicode_literals
+
+import unittest
+
+from fluent.runtime.utils import inspect_function_args, Any
+from fluent.runtime.errors import FluentFormatError
+
+
+class TestInspectFunctionArgs(unittest.TestCase):
+
+    def test_inspect_function_args_positional(self):
+        self.assertEqual(inspect_function_args(lambda: None, 'name', []),
+                         (0, []))
+        self.assertEqual(inspect_function_args(lambda x: None, 'name', []),
+                         (1, []))
+        self.assertEqual(inspect_function_args(lambda x, y: None, 'name', []),
+                         (2, []))
+
+    def test_inspect_function_args_var_positional(self):
+        self.assertEqual(inspect_function_args(lambda *args: None, 'name', []),
+                         (Any, []))
+
+    def test_inspect_function_args_keywords(self):
+        self.assertEqual(inspect_function_args(lambda x, y=1, z=2: None, 'name', []),
+                         (1, ['y', 'z']))
+
+    def test_inspect_function_args_var_keywords(self):
+        self.assertEqual(inspect_function_args(lambda x, **kwargs: None, 'name', []),
+                         (1, Any))
+
+    def test_inspect_function_args_var_positional_plus_keywords(self):
+        self.assertEqual(inspect_function_args(lambda x, y=1, *args: None, 'name', []),
+                         (Any, ['y']))
+
+    def test_inspect_function_args_bad_keyword_args(self):
+        def foo():
+            pass
+        foo.ftl_arg_spec = (0, ['bad kwarg', 'good', 'this-is-fine-too'])
+        errors = []
+        self.assertEqual(inspect_function_args(foo, 'FOO', errors),
+                         (0, ['good', 'this-is-fine-too']))
+        self.assertEqual(errors,
+                         [FluentFormatError("FOO() has invalid keyword argument name 'bad kwarg'")])