오픈소스 LSF 기반의 스케쥴러 openlava

 

openlava는 LSF 코드를 기반으로 한 것인지.. 똑같게 흉내 낸 것인지는 모르겠지만.. LSF를 사용해 본 유저로서는 상당히 메리트가 있는 job 스케쥴러임.. 현재 사이트는 내려가 있는 상태인데.. 아마도.. IBM 측에서 항의를 한 듯..

openlava 빌드

http://www.openlava.org/download/download.html

openlava-2.2.tar.gz

1.30MB

% cd ~
% mkdir LSF
% cd LSF
% tar xvfz openlava-2.2.tar.gz
% cd openlava-2.2
% ./configure

Tcl 없이는 빌드할 수 없다 한다.

Tcl 설치

http://tcl.tk

% tar xvfz tcl8.5.17-src.tar.gz
% cd tcl8.5.17/unix
% ./configure --enable-threads
% make
% su
% make install
% exit

이제 다시 openlava를 빌드한다.

openlave 재빌드

% cd ~/LSF/openlava-2.2
% ./configure
% make
% su
% make install

openlave 셋팅

% cd ~/LSF/openlava-2.2
% cd config
% su
% cp lsb.hosts lsb.params lsb.queues lsb.users lsf.cluster.openlava lsf.conf lsf.shared openlava.* /opt/openlava-2.2/etc
% useradd -r openlava
% chown -R openlava:openlava /opt/openlava-2.2
% cp /opt/openlava-2.2/etc/openlava /etc/init.d
% cp /opt/openlava-2.2/etc/openlava.* /etc/profile.d
% chkconfig openlava on
% cd /opt/openlava-2.1/etc
% vim lsf.cluster.openlava

아래 부분의 호스트 이름을 수정한다.

Begin   Host
HOSTNAME          model          type  server  r1m  RESOURCES
# yourhost                  IntelI5      linux   1      3.5    (cs)
End     Host

 

위를 아래처럼..

Begin   Host
HOSTNAME             model          type  server  r1m  RESOURCES
ihmin                !              !     1       -       -
End     Host

다음 host를 추가한다.

% vim /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.0.6 ihmin localhost localhost.localdomain

 

openlava 데몬 실행

% . /etc/profile.d/openlava.sh
% vim /etc/init.d/openlava
 
# This is the root of openlava installation
# change it if installed somewhere else.
export OPENLAVA_TOP=/opt/openlava-2.2
# 아래의 라인을 추가
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
 
% service openlava start
Starting daemons...
lim started
res started
sbatchd started
 
% vim /etc/bashrc
 
# 끝 부분에 아래를 추가
export PATH=$PATH:/opt/openlava-2.2/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
 
% source /etc/bashrc
% bjobs
No unfinished job found

이제 bjobs가 제대로 실행된다.

submit 테스트

% vim test.job
find / -name "*.so"
 
# 아래와 같이 queue 이름과 CPU 갯수, 프로젝트 이름, 로그 파일 이름을 지정
% bsub -q normal -n 1 -P test -o test.log < test.job
Job <104> is submitted to queue <normal>.
$ bsub -q normal -n 1 -P test -o test.log < test.job
Job <105> is submitted to queue <normal>.
$ bsub -q normal -n 1 -P test -o test.log < test.job
Job <106> is submitted to queue <normal>.
$ bsub -q normal -n 1 -P test -o test.log < test.job
Job <107> is submitted to queue <normal>.
$ bsub -q normal -n 1 -P test -o test.log < test.job
Job <108> is submitted to queue <normal>.
 
$ bjobs
JOBID   USER    STAT  QUEUE      FROM_HOST   EXEC_HOST   JOB_NAME   SUBMIT_TIME
105     ihmin   RUN   normal     ihmin       ihmin       *me "*.so" Jan 16 11:51
106     ihmin   PEND  normal     ihmin                   *me "*.so" Jan 16 11:51
107     ihmin   PEND  normal     ihmin                   *me "*.so" Jan 16 11:51
108     ihmin   PEND  normal     ihmin                   *me "*.so" Jan 16 11:51

openlava를 통해서 스케쥴링이 잘 되는 것을 확인할 수 있다.
이 외에도 LSF와 호환되기에 LSF의 커맨드를 그대로 이용가능하다..

cd /opt/openlava-2.2/bin/
$ ls
badmin   bkill    bqueues   bsub     lam-mpirun  lsid       lsrcp
bbot     bmgroup  brequeue  bswitch  lsacct      lsinfo     lsrmhost
bchkpnt  bmig     brestart  btop     lsaddhost   lsload     lsrun
bhist    bmod     bresume   bugroup  lsadmin     lsloadadj  mpich2-mpiexec
bhosts   bparams  brun      busers   lseligible  lsmon      mpich-mpirun
bjobs    bpeek    bstop     esync    lshosts     lsplace    openmpi-mpirun

아래의 문서는 다른분이 쓴 글인데 참고...

OpenLava Installation _ gumdaeng.pdf

0.68MB

 

openlava 개발이 포기된 이유에 대한 아래의 글을 2024/6/28일 추가합니다.

Hello all,

this is David Bigagli the founder of OpenLava, I am writing on behalf of the OpenLava project. As some or most of you might have noticed the GPL2 OpenLava project is under attack by the IBM corporation. The github software repository have been shut down under the US DMCA law and now the OpenLava website http://www.openlava.org, hosted on Amazon S3, which provides the source code to the latest 4.0 and 3.0 version will be shut down in the next 24 hours unless the source code is removed.

IBM claims that the versions of OpenLava starting from 3.0 infringe their copyright and that some source code have been stolen from them, copied, or otherwise taken from their code base.

I have developed most of the OpenLava code and I have reviewed all contributions. All this development was done without access to any IBM code. All IBM claims regarding the source code are false and fabricated.

OpenLava is helping users in need of open source LSF compatible software to run their business. OpenLava brought the LSF experience to a wider community thanks to Platform Lava, to which anybody can contribute, help or get inspiration for other opensource projects.

IBM does not have a technical answer to OpenLava and to the benefit its users have. They cannot articulate why their software is better than OpenLava for the money they charge for it. IBM fears OpenLava because it does provide a better functionality than their own software and that's why it can only reply with a lawsuit by hiring: Kirkland & Ellis LLP.

We have no choice at this stage but to remove OpenLava 3.0 and 4.0 from our web site. We will bring back the original OpenLava 2.2 to github and http://www.openlava.org.

With your community support OpenLava is and will always remain a open source project under the GPL2 licens