get_attribute swallows tracebacks for KeyError/AttributeError

[johtso]

get_attribute currently catches KeyErrors and AttributeErrors when trying to retrieve fields, assuming that those errors resulted from the top level attribute getting or key getting.

The problem with this is that if anything else inside a property causes one of those errors, the traceback is swallowed and it's a nightmare to debug!

Is there maybe a way to make sure that the exception doesn't come from deeper down?

Contrived illustration:

E AttributeError: Got AttributeError when attempting to get a value for field my_prop on serializer WidgetSerializer.
E The serializer field might be named incorrectly and not match any attribute or key on the Widget instance.
E Original exception text was: 'something_entirely_unrelated' object has no attribute 'some_attribute'.

[johtso]

Hmm, I think I have a possible solution.

First of all this error message logic could be moved to the base get_attribute function. It doesn't seem right that the error handling logic is split between the get_attribute function and the method. One catches ObjectDoesNotExist, the other catches KeyError and AttributeError. Why not do that all in one place.

Secondly, instead of catching KeyError and AttributeError exceptions, we could use getitem and getattr with defaults and respond to the default being returned.

Then we can be precise about the issue we're trying to catch, and avoid confusing a exception thrown by code triggered by the getattr or getitem / property call, for an error retrieving the key/attr itself.

How does that sound?

[sirpengi]

I'm having this issue as well, just here to add some more background.

This is somewhat related to #2598, but instead of callables raising an attribute exception this is on property fields. Here's a reproduction:

"""
requirements.txt:
Django==1.8.2
djangorestframework==3.1.3
"""

from django.conf import settings
SETTINGS = dict(
    DATABASES = {},
    DEBUG=True,
    TEMPLATE_DEBUG=True,
    ROOT_URLCONF = None,
)
settings.configure(**SETTINGS)


from rest_framework import serializers

class ExampleSerializer(serializers.Serializer):
        example_property = serializers.CharField(read_only=True)
        example_fail_property = serializers.CharField(read_only=True)

class ExampleInstance(object):
        @property
        def example_property(self):
                return "foobar"

        @property
        def example_fail_property(self):
                raise AttributeError('property failed')

serializer = ExampleSerializer(ExampleInstance())
print(serializer.data.items()) # ItemsView({'example_property': 'foobar'})

[xordoquy]

@johtso my first thought is that is feels wrong to have a KeyError or AttributeError.
I'm going to keep this opened to dig a bit more into the issue before I make a final decision.

[sirpengi]

Just another note as I'm digging further into this: this problem only manifests for read_only=True fields. The exceptions otherwise bubble up properly.

[johtso]

Just got bitten by this again.. any more thoughts on this problem/solution?

[drew-moore]

just got bit. ouch.

[lovelydinosaur]

Not obvious how to proceed with this. Thoughts/suggestions welcome.

[chidg]

Just for any googlers arriving here trying to solve a KeyError on get_attribute, it may be due to accessing serializer.data in a situation where validation has already been done. In that case you should be using serializer.validated_data.

http://stackoverflow.com/questions/32236958/keyerror-on-relation-field-in-django-rest

[rpkilby]

Something to think about later - #5600 could be adapted to fix this.

[cgi1]

@chidg: Good point; still appears in 2018.

[munsu]

Was caught by this. Some other package I rely on for a property had a non-obvious breaking change. My serializer tests failed instantly -- just dropped the field from serializer.data with no warning/error. Spent half a day tracing the source (thought it was something else breaking the dict comparison in my tests).

A warning log at the least would have been immensely helpful.

[AndersonSMed]

Just found out about this discussion and the very same thing still happens today. I also created a simple code snippet that illustrates this behavior:

from rest_framework import serializers

class ExampleSerializer(serializers.Serializer):
    title = serializers.CharField(read_only=True)

class ExampleObject:
    @property
    def title(self):
        return self.foo

obj = ExampleObject()
serializer = ExampleSerializer()

# Doesn't trigger any error
serializer.to_representation(obj)

# Triggers an AttributeError
obj.title