The core of mlcore is computation graph, which performs the forward and backward calculation and computes gradient automatically. The abstraction of variable and optimizer makes mlcore can run in everywhere include single node, Angel, Spark and so on. here is the architecture of mlcore:
here is the runtime architecture of mlcore:
pull parameters from local or parameter server (PS)
perform the forward calculation
perform the backward calculation to calculate gradient
push gradient to local or PS
finally, update parameter in local or PS
2. Variable and optimizer
The variable is a vector or matrix with slots and updater. The updater is used to update the value of variable and slots are the auxiliary data of updater. The number of slots is decided by the type of updater. Usually, the shape of value is the same as that of slot.
The variable and updater are interfaces in mlcore. Different distributed systems can implement their own variables and updaters. In this way, mlcore is easy to embed into other distributed systems.
The basic operation of variable
create: create a variable in PS or local
init: initial a variable in PS or local
load: load data from disk to initial a variable in PS or local
pull: pull the value of a variable from PS or local
push: push gradient of a variable to PS or local
update: update a variable in PS or local, the slot attached will also updated if necessary.
saveWithSlot/saveWithoutSlot/checkpoint: save a variable in PS or local. as mentioned about, variable usually with slots, you can choose to save slots or not. note: checkpoint is the same as saveWithSlot
release: release a variable in PS or local
The status and life cycle of a variable:
The top abstraction of updater:
trait Updater extends Serializable {
val numSlot: Int
def update[T](variable: Variable, epoch: Int, batchSize: Int): Future[T]
}
3. Computation graph
The computation graph in mlcore is coarse grain, the basic operator is layer. The coarse grain computation graph has a smooth learning curve. Consequently, it is user friendly.
mlcore
a stand alone machine learning suite which can easy to integrate with angel ps
1. Architecture of mlcore
The core of mlcore is computation graph, which performs the forward and backward calculation and computes gradient automatically. The abstraction of
variableandoptimizermakes mlcore can run in everywhere include single node, Angel, Spark and so on. here is the architecture of mlcore:here is the runtime architecture of mlcore:
2. Variable and optimizer
The
variableis a vector or matrix withslotsandupdater. Theupdateris used to update the value of variable andslotsare the auxiliary data ofupdater. The number ofslotsis decided by the type ofupdater. Usually, the shape of value is the same as that of slot.The
variableandupdaterare interfaces in mlcore. Different distributed systems can implement their own variables and updaters. In this way, mlcore is easy to embed into other distributed systems.The basic operation of
variablevariablein PS or localvariablein PS or localvariablein PS or localvariablefrom PS or localvariableto PS or localvariablein PS or local, theslotattached will also updated if necessary.variablein PS or local. as mentioned about,variableusually with slots, you can choose to save slots or not. note: checkpoint is the same as saveWithSlotvariablein PS or localThe status and life cycle of a
variable:The top abstraction of
updater:3. Computation graph
The computation graph in mlcore is coarse grain, the basic operator is layer. The coarse grain computation graph has a smooth learning curve. Consequently, it is user friendly.