Data Management¶

Data management is the foundation of a data-based project, not only IC development, but also many other development. Version Control is the core of data management, it helps different teams working together, in their own pipelines.

QUICK tries to handle different Version Control Tools with unified interface, so users will not notice the diffence when use this tool. Currently QUICK support: git, svn, clearcase.

QUICK uses Project and Subproject as container of data, and data is split into serveral Vobs (Versioned Object Base), which is Repository in some Version Control Tools, and project administrator would configure the locations of all Vobs, and the hierarchy of Projects and Subprojects, and a particular Subproject contains which Vobs.

Project¶

Project is used for access control of project data, and also container of Subproject.

Normally a project group should be assigned to a project, and all project members would be added into the group. So the proposal is that create a project account as a normal user, and allocate home directory for each project. SubProject will be a sub-directory under the home directory, and project data will be stored under the sub-directory.

Hint

For centralized Version Control Tool like SVN, has its own access control strategy, project group concept can be ignored.

Subproject¶

Project would have one or more Subprojects, which are product related. Each Subproject is a container of Vobs. Project administrator will define the binding between Subproject and Vobs, and how many Views the Subproject will have, and other configuration.

Vob¶

A VOB is exactly a Repository, it has name and repository type (which Version Control Tool used), and the Repository it is bound to.

View¶

A View is a working copy of the Vobs. Different Views of same Subproject are separate space, since they are different working copies.

Enter a project¶

To enter a project, user just run command: qk.

Tool will determine which projects the user is assigned, and then give the choices if the user has more than one project, user can type the begining letters of the choice, then type <TAB>, tool will complete the full word if it is not ambiguous. Then Subproject choices and View choices will be provided, too.

After a concrete Project, Subproject, View are selected, tool will clone / checkout the Vobs bound with the Subproject, if the View is the first time entered. Otherwise this step will be skipped.

Vobs will be cloned / checkouted to a configured location, and then mounted to a predefined mount-point. Normally the mount-point will be: /vobs/xxx, xxx is the name of a Vob.

Note

The mount is process isolated, it’s only visible in the process user entered the project. So you don’t need to worry about the conflict when more than one users enter into same Subproject.

The command looks like below:

$> qk
********************************************************************************
*                   QUICK / Quite Userful IC Kit
*                       Version 0.1.0
*           All Copyright (2019-2021) Are Reserved By Sandy
********************************************************************************
[INFO] Please select in below projects:
         example
         test
[INFO] Please select ("." to exit): example
[INFO] Selected project: example
[INFO] Please select in below subprojects:
         example1
         example2
[INFO] Please select ("." to exit): example1
[INFO] Selected subproject: example1
[INFO] Please select in below views:
         default
         ex1
[INFO] Please select ("." to exit): default
{sandy.example.example1.default}:/vobs/vob1>

Or you can specify the Project, Subproject, View in the command line:

$> qk example example1 default
********************************************************************************
*                   QUICK / Quite Userful IC Kit
*                       Version 0.1.0
*           All Copyright (2019-2021) Are Reserved By Sandy
********************************************************************************
[INFO] Selected project: example
[INFO] Selected subproject: example1
[INFO] Selected view: default
{sandy.example.example1.default}:/vobs/vob1>

After enter the project, linux PROMPT will change to: {username.project.subproject.view}, in blue color. This is a reminder that you are already in QUICK environment.

Env Variables¶

When a project is entered, some Environment Variables will be set, their names all starts with ‘X’. QUICK will use these variables for different purposes. User’s scripts also can use them.

XPROJECT - Project name
XSUBPROJECT - Subproject name
XVIEW - View full name
XVOB - Main Vob root path
XVIEW_ROOT - View root location for working copy
XVOB_ROOT - Vob root mount point

Configuration¶

Configuration is divided into 2 parts: global configuration and project configuration.

Global configuration targets file: conf/global.ini. Looks like below:

[global]
PROJECTS        = $QKHOME/conf/projects.yaml
VOBS            = $QKHOME/conf/vobs.yaml
LIBRARIES       = $QKHOME/conf/libraries.yaml
VOB-STORAGE     = /var/repos
VIEW-STORAGE    = /var/views
VOBMOUNT        = /var/vobs
VOBROOT         = /vobs
DEFAULT-VOBTYPE = svn

Normally you only need to update: VIEW-STORAGE, VOBMOUNT;

VIEW-STORAGE

This variable defines the view storage directory, all team members’ views data will be stored under it. This directory should be controlled by root, mode as 555.

In practice, we usually set this fold as a hidden fold, i.e. : /path/to/view/.views .

VOBMOUNT

This variable defines a place that tool can create mount points. Normally its value is same with VOBROOT;

VOBROOT

This is the root mount point of all vobs. Administrator should make this common mount point with minimum space, normally 100M bytes is enough.

Project configuration targets: conf/projects.yaml, conf/vobs.yaml. Looks like below:

conf/projects.yaml¶

- name: example
  subprojects:
  - name: example1
    views:
    - default
    - ex1
    vobs:
    - vob1
  - name: example2
    vobs:
    - vob2
    views:
    - default
    - ex1
- name: test
  subprojects:
  - name: test1
    vobs:
    - vob1

Project is the main entry of a project, it has subproject(s); Subproject defines vobs and views, if no view is defined, a builtin default view will be used. Vobs are defined in conf/vobs.yaml, vob names will be used in conf/projects.yaml, to make a vob bound to a project.

A vob can be bound to more than one project, and a project can contain more than one vob.

Project and subproject can define vobmount, vobroot, viewstorage to override the global definition of VOBMOUNT, VOBROOT, VIEW-STORAGE. The priority is subproject > project > global.

conf/vobs.yaml¶

- name: vob1
  repo: svn:///var/repos/vob1
  type: svn
- name: vob2
  repo: svn:///var/repos/vob2
  type: svn

Vob is defined by 3 attributes: name, repo, type. Vob type can be: git, svn, clearcase.

Repo value is the form that when you clone a git repo or checkout a svn repo, as in the example above.

Extension Tools¶

To have extended features based on open source Version Control Tool, QUICK borrow some concepts from ClearCase. Those extended features are mainly from xTool.

Since QUICK use process isolated mount, if jobs are submitted to remote computing server, QUICK need to rebuild these mount-points on remote server, this is done by tool: xsub.

Directory Structure¶

Having unified directory structure has lots of benifit. Team members know where to find the files he is looking for, and where a new file he should put, the communication between members is reduced. On the other hand, QUICK has some builtin tools, which need to search and generate files, the pre-defined directory structure will help a lot.

Currently the pre-defined directory structure is as below:

<VOB>
+-- .bashrc.login
+-- .bashrc.project
+-- bin
+-- etc
+-- lib
|   +-- python
|   +-- perl
+-- resources
+-- resources_soft
+-- packages
|   +-- vhdl
|   +-- verilog
|   +-- sv
+-- units
    +-- <unit>
        +-- source
        |   +-- vhdl
        |   |   +-- rtl
        |   |   +-- tb
        |   +-- verilog
        |   |   +-- rtl
        |   |   +-- tb
        |   +-- sv
        |   |   +-- rtl
        |   |   +-- tb
        |   |   +-- uvm
        |   +-- upf
        +-- simulation
        |   +-- vcs
        |   +-- modelsim
        +-- spyglass
        +-- synthesis

Useful Aliases¶

According to the pre-defined directory structure, QUICK defined some useful aliases to switch directory conviniently.

cdvob: Switch to VOB root.
cdenv: Switch to $VOB/env directory.
cdpack: Switch to $VOB/env directory.
cdunits: Switch to $VOB/units/<unit> if with argument, otherwise switch to $VOB/units. For unit argument, auto-completion is supported, you need to type <TAB><TAB> to show the choices.
cdsrc: Switch to $VOB/units/<unit>/source if already under some unit. To change unit you need to specify a <unit> arguemnt, and this argument support auto-completion.
cdvhd: Switch to $VOB/units/<unit>/source/vhdl if already under some unit. To change unit you need to specify a <unit> argument, and this argument support auto-completion.
cdvlog/cdver: Switch to $VOB/units/<unit>/source/verilog if already under some unit. To change unit you need to specify a <unit> argument, and this argument support auto-completion.
cdsv: Switch to $VOB/units/<unit>/source/sv if already under some unit. To change unit you need to specify a <unit> argument, and this argument support auto-completion.
cdupf: Switch to $VOB/units/<unit>/source/upf if already under some unit. To change unit you need to specify a <unit> argument, and this argument support auto-completion.
cdsim: Switch to $VOB/units/<unit>/simulation if already under some unit. To change unit you need to specify a <unit> argument, and this argument support auto-completion.