(see article
in RIT News&Events Feb. 7, 2008)
On December 21st, 2007 Research Computing achieved its goal of harnessing the
power of 1000 processors by tapping into the unused computing cycles of hundreds of
computers at RIT. Computing power is being utilized in the Center for Imaging Science,
Computer Science, Research Computing, Information Technology Services, Information
Technology, Math, Electrical Engineering, Mechanical Engineering, Manufacturing and
Mechanical Engineering Technology, and Liberal Arts. And more will be joining the pool
in coming weeks.
Multiplying the power of a single desktop computer by 1000 or more is a tremendous
increase in personal computing capacity and extends the boundaries of research
problems that can be computed. Essentially, this is a campus computational grid. In
total, the current Condor system consists of more than 1.5 terabytes of memory, greater
than 67 terabytes of disk space, and about .75 teraflops of computing power.
Condor Clients by Operating System
What is Condor?
Condor is a project of the University of Wisconsin.
It is what is known as High
Throughput Computing, that is, the goal is to run as many jobs as possible at the same
time. The easiest way to do this is to tap into all the computers on campus when they
aren't being used. Condor senses when the computer is free according the policy on
that particular machine (the policy is set by the owner of each machine. e.g., "only use
for condor jobs between 11pm and 6am", etc.). The harvesting of these unused cycles
ensures that the investment in all that computer hardware is utilized to the maximum
extent possible.
Efficiency is not the only benefit from Condor. Researchers benefit by getting access to
a huge number of computer processors at essentially little or no cost. Certain types of
computer problems are ideal for the Condor environment. These jobs are ones that can
be separated into hundreds or thousands of little jobs that run independently of one
another but whose results can be aggregated at the end. An example of this is image
processing or video. Some analysis requires the processing of each individual pixel of
an image, or of a single frame of a movie. Each of these individual jobs can be run
separately and the results brought together at the conclusion of the job.
Condor at RIT - where did it come from?
In 1999 researchers in the Center for Imaging Science
needed more processing power
to analyze images with high resolution and hyperspectral bands using physics-based
models. Rolando Raqueno of the Digital Imaging and Remote Sensing (DIRS)
laboratory and Bob Krzaczek of the Laboratory for Imaging Algorithms and Systems
(LIAS) began to use Condor to meet these needs. A few years later, James Craig of
Computer Science started another condor cluster or "flock" and joined it to the CIS
flock.
In 2006, Rick Bohn (Research Computing) started to help support the Condor flock of
workstations in Imaging Science. By the summer of 2007, Paul Mezzanini (Research
Computing) started seriously deploying Condor on the 100 cpu high performance
cluster and the possibility of managing 1000 condor clients became an achievable goal.
In the Fall of 2007, Bill Hoagland (CIS/LIAS) developed new installers and Brent Strong
(ITS/DSS) and Jeremy Sieminski (ITS/DSS) installed Condor on Macintosh computers
in the Math Department. And by the end of 2007, over 1000 processors were available
on Windows, Macintosh, Solaris, and linux computers among all the flocks on campus
spread across eight departments. Condor1000 was a reality.
According to Raqueno, "The dissertation research of recent Ph.D. graduate, Captain Michael
Foster (USAF), using hyperspectral imagery fused with simulated airborne LIDAR data was
a prime example of a problem that could not have been solved in time without CONDOR."
...and where is it going? Grid Computing
With the success of the Condor1000 Project, RIT can not only support more
researchers on campus who will utilize the system, but RIT can now begin to
experiment with connecting its system with systems at other institutions to create pools
of thousands and tens of thousands of processors in size. The advantage of such a
large pool is threefold - larger jobs can be run, more users can run at the same time,
and backup and redundancy are available, should there be any outages on campus.
This is the benefit of a grid of computers: a shared cyberinfrastructure that increases
efficiency, scope and productivity. RIT's membership in the statewide NYSGrid and the
worldwide Open Science Grid will help enable such a scenario.
current Condor pool stats
Condor Wiki article
|
