Error on pytorch Linear modules with bias=False

[xianghang]

When running the following code, I got an error

import torchopt
import torch

torchopt.MetaAdam(torch.nn.Linear(10, 2, bias=False), lr=1e-3)

The detailed error message is

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
[<ipython-input-5-ec1c724f63ae>](https://localhost:8080/#) in <module>
----> 1 torchopt.MetaAdam(torch.nn.Linear(10, 2, bias=False), lr=1e-3)

8 frames
[/usr/local/lib/python3.7/dist-packages/torchopt/_src/transform.py](https://localhost:8080/#) in <lambda>(t)
    305     def init_fn(params):
    306         zero = tree_map(  # count init
--> 307             lambda t: torch.zeros(1, dtype=torch.int32, device=t.device).squeeze_(), params
    308         )
    309         mu = tree_map(  # first moment

AttributeError: 'NoneType' object has no attribute 'device'

However, if I change bias=True, i.e.

torchopt.MetaAdam(torch.nn.Linear(10, 2, bias=True), lr=1e-3)

it seems to work fine.

Is there a bug?

[xianghang]

I guess the issue is that for Linear modules with bias=False, pytorch will register a None parameter for bias
https://pytorch.org/docs/stable/_modules/torch/nn/modules/linear.html#Linear

self.register_parameter('bias', None)

It is probably good to filter out None parameters.

[XuehaiPan]

@xianghang Thanks for reporting this. It's a bug. We will fix it and publish a new wheel right away (released as 0.5.0.post4). Upgrade your installation:

pip3 install --upgrade torchopt