Skip to content
This repository was archived by the owner on Oct 11, 2023. It is now read-only.
This repository was archived by the owner on Oct 11, 2023. It is now read-only.

Learning rate for NN training  #26

@pamio

Description

@pamio

in tensorflow or torch, I have observed that the learning rate is usually one number like 0.001 or 0.01 etc. But in this tutorial I find that the learning rate for 16 classes is something like this

cntk_lr_per_image = [0.01] * 10 + [0.001] * 5 + [0.0001] which is array of 16 items which is equal to the number of classes. So if I have 80 classes to train should this array cntk_lr_per_image contain 80 learning rate ?

Something like this cntk_lr_per_image = [0.01] * 40 + [0.001] * 30 + [0.0001]*10 ?
I understand what learning rate is and how it impacts the overall learning of the model and there's no straight forward way to choose a learning rate but this tutorial uses an array of learning rates which is what confuses me.

Can you help me with understanding how learning rate us used in this example ?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions