See the tutorial. It's quite simple (Keras provides simpler interface though).
A neural network is defined as a class that has a forward function. The forward function defines how input data is processed through the network. PyTorch automatically defines a corresponding backward process (maybe only when the forward process is differentiable).
From the tutorial, the class defines all layers in init() and defines how input data processes in forward().
