Skip to content

训练完毕后,模型的存储参数以稀疏(sparse)形式存储的使用场景?  #948

@backyes

Description

@backyes
  CpuVector vec(*bufs_[PARAMETER_VALUE].get());
  Header header;
  header.version = kFormatVersion;
  header.valueSize = sizeof(real);
  header.size = getSize();

  CHECK_EQ(header.size, vec.getSize());

  CHECK(s.write(reinterpret_cast<char*>(&header), sizeof(header)))
      << "Fail to write parameter " << getName();

  CHECK(s.write(reinterpret_cast<char*>(vec.getData()),
                header.size * sizeof(real)))
      << "Fail to write parameter " << getName();
  if (config_.is_sparse()) {
    CpuIVector rows(*intBufs_[PARAMETER_ROWS].get());
    CpuIVector cols(*intBufs_[PARAMETER_COLS].get());
    CHECK(s.write(reinterpret_cast<char*>(rows.getData()),
                  rows.getSize() * sizeof(int)))
        << "Fail to write parameter " << getName();
    CHECK(s.write(reinterpret_cast<char*>(cols.getData()),
                  cols.getSize() * sizeof(int)))
        << "Fail to write parameter " << getName();
  }

  return true;
}

Parameter稀疏存储(特指训练完毕后,存储并用户预测的情况)的使用场景 ?

请给出sparse的全面解释。

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions