Skip to content

Add a Dynamic PyTorch Model Builder #24

Description

@DeepBlockDeepak

Description:

Implement a PyTorch-based "DynamicModel" that constructs a nn.Sequential model at runtime based on user-provided layer dimensions, task type (regression or classification), and optional configuration parameters. This should closely mirror the functionality of the existing SimpleNeuralNetwork, but will leverage PyTorch’s built-in operations to reduce code complexity.
The Goal is to start with some conceptual parity, keeping things flexible, before diving in and moving a different direction.

Requirements:

  • Input: A list of layer dimensions (e.g. [input_dim, 128, 64, output_dim]) and a task type (classification or regression).
  • Creates nn.Linear layers for each pair of consecutive dimensions.
  • Inserts default (and later user-specified) activation functions (ReLU for hidden layers, Sigmoid or Softmax for classification outputs, none or Identity for regression).

To incorporate or not to incorporate

  • Ability to use a shared config object, the SimpleNN's use of NeuralNetworkConfig? This might couple the PyTorch model too much with the SimpleNN, but could be generalizable conceptually. At first, this could be useful for testing the SimpleNN alongisde the PyTorch analogue.

Nice-to-haves

  • Incorporation of dropout layers when specified (e.g. based on keep_prob).
  • Custom initialization schemes ... Should these be Pydantic-esque data models? Think of the SimpleNN NeuralNetworkConfig!!!
  • Tests comparing the performance and output of analogously similar DynamicModel against SimpleNeuralNetwork.
  • Docs, once a standard approach is settled on.

Caveats

  • No need for an exact replica of the SimpleNN. I'm trying to aim for some conceptual parity though.
  • Keep configuration flexible but not overly complex. Start simple and add complexity later if needed.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions