Description:
Implement a PyTorch-based "DynamicModel" that constructs a nn.Sequential model at runtime based on user-provided layer dimensions, task type (regression or classification), and optional configuration parameters. This should closely mirror the functionality of the existing SimpleNeuralNetwork, but will leverage PyTorch’s built-in operations to reduce code complexity.
The Goal is to start with some conceptual parity, keeping things flexible, before diving in and moving a different direction.
Requirements:
- Input: A list of layer dimensions (e.g. [input_dim, 128, 64, output_dim]) and a task type (classification or regression).
- Creates nn.Linear layers for each pair of consecutive dimensions.
- Inserts default (and later user-specified) activation functions (ReLU for hidden layers, Sigmoid or Softmax for classification outputs, none or Identity for regression).
To incorporate or not to incorporate
- Ability to use a shared config object, the SimpleNN's use of NeuralNetworkConfig? This might couple the PyTorch model too much with the SimpleNN, but could be generalizable conceptually. At first, this could be useful for testing the SimpleNN alongisde the PyTorch analogue.
Nice-to-haves
- Incorporation of dropout layers when specified (e.g. based on keep_prob).
- Custom initialization schemes ... Should these be Pydantic-esque data models? Think of the SimpleNN NeuralNetworkConfig!!!
- Tests comparing the performance and output of analogously similar DynamicModel against SimpleNeuralNetwork.
- Docs, once a standard approach is settled on.
Caveats
- No need for an exact replica of the SimpleNN. I'm trying to aim for some conceptual parity though.
- Keep configuration flexible but not overly complex. Start simple and add complexity later if needed.
Description:
Implement a PyTorch-based "DynamicModel" that constructs a nn.Sequential model at runtime based on user-provided layer dimensions, task type (regression or classification), and optional configuration parameters. This should closely mirror the functionality of the existing SimpleNeuralNetwork, but will leverage PyTorch’s built-in operations to reduce code complexity.
The Goal is to start with some conceptual parity, keeping things flexible, before diving in and moving a different direction.
Requirements:
To incorporate or not to incorporate
Nice-to-haves
Caveats