NASA - SatCORPS Group - NASA Langley Cloud and Radiation Research

We are going to move the spinning disk archive from the WEB_BACKUP disk space to the PRD_ARCHIVE space. The e-mail that announced this change is below.

=================

Hi Everyone,

I'm e-mailing you because you have access to and know about the WEB_BACKUP disk on wbs128.

We are moving our data off the WEB_BACKUP disk and wbs128 immediately.

You also probably know that wbs128 has been having full disk issues with increasing frequency. The reason is that other groups now use the WEB_BACKUP disk and they are using it more and more.

We weren't originally part of the plan in terms of being on there, so our consumption of over half the disk has thrown a big wrench into the SA teams forecasting for that disk's growth. We do have a permanent spot (/SCF6/PRD_ARCHIVE on AMI or /noback/PRD_ARCHIVE on the clouds machines) so we've been asked to move ASAP.

I've started moving the NEXRAD to try and limit the impact to the website but will also migrate the cloud product domains and satellite imagery at some point.

This will impact all of us because while it is happening, files could be on wbs128 or have been moved to the PRD_ARCHIVE disk. Until I send the "The move is completed e-mail" this condition will exist. Once I send the e-mail saying the move is completely done, we will no longer be going to wbs128 to work on older files. Instead, use the AMI or /noback mounts.

So that everyone can keep up to date without being inundated with information, I'll use the message board so that you can ask questions or concerns and others can read and participate in the discussion. The thread is here.

Please check that you can access the message board, going forward, its probably going to be the best medium to move information between ourselves since there won't be as much e-mail CC'ing involved. If you don't have access, please let me know and I can fix.

We will be converting the noback disks 1-8 from their current setup (shared from the NAS devices as NFS exports) to a format called iSCSI. The reason for this is so that /noback/d1-9 can be mounted to all the servers.

It turns out, that having them mounted across the server platforms is something that would make people's lives easier with a minimum of effort.

So... d3 is first because I am the only person with data on it and am able to clear it out. Once d3 is set up as an iSCSI disk, we are going to copy data from one of the other disks (1,2,4,5 or 6) to d3 and then switch them. Once switched, the older drive will be reformatted as an iSCSI disk and the process will continue until all disks are converted.

As this is being done, you will probably notice that /noback/d1-9 will start appearing on cloudsway and gate machines. Depending on connectivity, it may also appear on ssai10/11/12.

Please do not use noback/d3. We are emptying /noback/d3 to convert it to an iSCSI disk.

Thus, it is NOT emptying so that others can use it.

This forum contains messages about the NAS disks (/noback) and the web archive disks.

Rabi -

You are right, the idea is to run on AMI. You are also right, files on enso cannot be seen as part of the file system on AMI processing nodes. I think if we use SCP to transfer the files necessary for job runs prior to executing a job from enso we will get the behavior we want.

Our other option is to have Pat send a request for more space on the system. I've asked for quota increases before and I've always gotten "if your PI sends a request for more space, we'll set it up" from the SA team. So, if Pat asks for more space, we'll get it, I guess. That would certainly make processing a little easier.

Rabi had the following questions/ comments about the data access:

Thad,

For the current deadline, we are running Terra/Aqua over continental USA for Oct, 2006. Doug sent out a file (8/5/2010) with the times the overpass fell between 25N-55N, 130W-65W.
I thought the idea was to run on AMI and utilize parallel processing capabilities. We have know for sometime that AMI does not allow nfs mounts from outside. What mechanisms are available for data to be shared inside AMI. How much space does just the CONUS 2006 occupy? I know we can ftp the data over but do we have the space on AMI?

To test my code, I would need access from cloud, spg10 & ssai11/ssai10.

Thanks

I wrote the following about the data locations and access methods:

It took most of the morning to *find* all the files and standardize the
directories so they all look the same. I've set up a "unionfs" mount
which is basically a whole bunch of drives joined together - thats why
the directories needed to be standardized.

The union mount is exported via NFS BUT, I need to add IP addresses for
the machines that want to mount the files. a "showmount -e enso" will
list the mount. If you're going to do an NFS mount, use: "mount
enso:/mount/union-drive place-to-mount-on-your-machine"

Finally, enso has a field account with the customary access so the files
can be accessed via ssh/scp through /mnt/union-drive

And, as if that weren't complex enough, they can be accessed via their
actual/real locations, on /mnt/union-n where n=0,1,2,3,4 (there are four
drives).

Note - the /mnt/union-2 drive is appearing flakey so I've been trying to
remove it from the system but need to copy the data off the drive.

If you can get me the IP addresses of the machines to add, I can set up
NFS mounting. Finally, we can't do NFS mounts to AMI - the AMI system
won't allow it as far as I know.

I wanted to get the info to you as quickly as possible, so this e-mail
is scattered, sorry.

Also, I discovered a problem in the retrieval of the reprocessed data -
the ordering thing isn't getting all the files from laads - I'm
investigating why. This means the reprocessed files aren't all here when
I had thought they were (discovered this on Saturday when I started to
look at why things were going so fast....)

If you look at the mount, everything is under "MODIS_data" so it looks like:

MODIS_data
terra-modis-data
2006
aqua-modis-data
2006
terra-modis-data-reprocessed
2006

Kirk wrote:

Dear Contrailers,

We've made some good progress on Task 079 so far but we need to keep the momentum. October 31 will be here very soon so it's imperative that we start processing data soon.
In regard to the action item list that I distributed after the last meeting, I'm aware of the following:

Robyn has made good progress restoring the Walters' tool functionality and continues to implement capabilities.
Rabi continues implementation of MODIS processing on x86 systems and developed a good flowchart documenting data and processing flow.
Kris has made good progress on the MERRA reader.
Doug generated overpass data to direct processing.
I requested an AMI account for Dave.

I'm sure other progress has been made as well.

Next week Dave, Konstantin, and Thad need to get together to look at ways to implement processing on AMI. The flowchart that Rabi provided should help guide this process.

Doug produced data files that indicate Terra and Aqua overpasses. His message is:

I made data files for Oct 2006 that indicate the Terra and Aqua 5-min
granule times needed to cover the CONUS domain 25-55N, 65-130W. The
files are on the SAN at /nfs/xsan/AWIN/data/doug/modis_orbdat. The
.gran.times.dat files have the 5-min times and the .times.dat files have
entry and exit times for this domain. This procedure takes into account
the MODIS swath, not just sub-satellite track.

Rabi Prepared a processing flow diagram and it is here:

http://louis-pc.larc.nasa.gov/site/wiki/index.php/File:Accri_4aug2010.pdf

His description from his e-mail is:


Attached is a processed processing and dataflow diagram for our next ACCRI delivery(MODIS, over CONUS, Oct, 2006) . Hopefully I have captured everything we discussed. To meet the current deadline, I have left the input/output formats as flat binary so existing processing units can be used without too much addition of the code. The post processing step will take the flat binary or ascii files and combine with the VISST/SIST retrievals into one netcdf. Once this is out of the way, we have to combine some intermediate flat binary files into one netcdf files. This will also help in archival and storage.

The processes in slide 1 can be run on the AMI system as they don't require mcidas- only IDL, HDF/netcdf libraries....

Kirk prepared the meeting notes and they are here:

http://louis-pc.larc.nasa.gov/site/wiki/index.php/File:ARRA_079_jul272010_notes.pdf

This thread contains messages about ARRA Task 079. The Wiki Page is here: http://louis-pc.larc.nasa.gov/site/wiki/index.php/ARRA_Task_079_CONUS_Contrail_Climatology

This forum is to discuss projects that the group is doing.

This thread deals with with installation and other setup requirements of the ordering tool.

This forum is for discussions related to the prototype CERES ordering tool being worked on louis-pc.

Matt had a few questions and I replied with the following e-mail:
=============================================

Hi Matt,

I recently completed the build for McIDAS 2008 and got the source from Doug. I believe I have a copy stored on Columbia and can try to transfer that to the new system if it would help.

As far as install requirements, do you mean space or software? I know that the GCC/GFortran suite needs to be available beforehand. If you have any additional questions, I can try and field them for you.

Finally, since this is probably going to be a noteworthy conversation, I've placed our trail into the message system here:



Thank you,

Thad

Ayers, Jeffrey Kirk (LARC-E302)[SCIENCE SYSTEMS & APPLICATIONS INC.] wrote:
>
> Matt,
>
> I'm forwarding your request to Doug and Thad. They recently did an install on a similar system at Ames and Doug is our resident McIDAS expert.
>
> I'll come up with some disk names.
>
> Doug and Thad, Please get Matt what he needs.
>
> Kirk
>
> -----Original Message-----
> From: Allen, Matthew S. (LARC-E302)[SCIENCE SYSTEMS & APPLICATIONS INC.]
> Sent: Fri 1/23/2009 9:02 AM
> To: Ayers, Jeffrey Kirk (LARC-E302)[SCIENCE SYSTEMS & APPLICATIONS INC.]
> Subject: linux system
>
>
>
> Kirk,
>
>
>
> I need to check out the install requirements and such for McIDAS, to prepare to put it on the new Linux system. Can you please download the required files for 64-bit Linux and let me know where they are? If there is a separate 32-bit version, please grab that one too. I would do this myself, but they require a login for the McIDAS-X software.
>
>
>
> There are 3 disks that I've temporarily named disk1, disk2, and disk3. If you prefer a different name please let me know what you want to use.
>

This thread is to memorialize the work done to instal McIDAS-X 2008 on a linux machine.

"Marc" from case.edu sent in a trouble alert to Pat this afternoon reporting trouble with the AVHRR GAC web page.

It turns out that the disk "SATGIF1" on louis-pc had failed. This caused data loss on the drive.

Louis came into the office and attempted repair to the disk. He eventually determined the drive could not be restarted and instead replaced the disk.

Once replacement was complete, Louis began the process of restore from tape.

Once the replacement is complete, we can attempt to rebuild any missing data.

Processing should not be affected and data files going forward should appear as normal. If not, then we need to fix/troubleshoot.

E-Mail chain is below:
===============================
Dear Pat and Marc,

Louis troubleshot the problem and the disk drive doesn't look like it is functional. He is restoring data to a replacement drive and expects it to take a few days to reload the replacement disk.

Once we have everything back from the tape, we can then try to rebuild anything that might have been lost.
That said, going forward, the processing will probably continue so we should probably start to see regular data appear at the normal rate - so, Jan 23 files should appear tomorrow on their regular schedule.

I will continue to monitor and directories to see if there is any processing that needs adjustments.


Thank you,

Thad


Patrick Minnis wrote:
>
>
> Begin forwarded message:
>
>> *From: *"marc" <mdr6@case.edu <mailto:mdr6@case.edu>>
>> *Date: *January 22, 2009 1:43:18 PM EST
>> *To: *<p.minnis@larc.nasa.gov <mailto:p.minnis@larc.nasa.gov>>
>> *Subject: **HRPT Portal*
>>
>> Can’t seem to get any data off http://www-angler.larc.nasa.gov/avhrr/ Are you down?
>>
>
> ********************************************************************************
> Patrick Minnis, PhD
> MS 420
> NASA Langley Research Center
> Hampton, VA 23681-0001
>
> phone: 757-864-5671 fax: 757-864-7996 email: patrick.minnis-1@nasa.gov <mailto:patrick.minnis-1@nasa.gov>
> homepage:http://www-pm.larc.nasa.gov/
> ********************************************************************************
>
>

Ben will be performing the SSHd upgrade and will work on the Apache once the dependancy is cleared up:

E-Mail dated Jan 12, 2008 is below:
==========================================
Ben,

Thank you for the heads up about the update, holding off until we have the SSHd sounds like a very good idea all things considered. If there is any kind of assistance we can provide you with, please let us know!

Thank you,

Thad

Loyall, Benjamin Farragut (LARC-E302)[SCIENCE SYSTEMS & APPLICATIONS INC.] wrote:
> Thad,
>
> I just checked with John and he wants me to do the SSHd upgrade on Angler. Given that Apache 2.2.11 ties into the SSL libs, I'd like to hold off on rolling that into production until I have the SSHd work done. Most likely it will break Apache requiring it to be recompiled.
>
> I will be working on the SSHd stuff on Tues.
>
> ben

This is the origin of the apache version upgrade. Louis and I have been investigating the use of the "mod_rewrite" module for apache and a version update would accomplish this.

I wrote a quick e-mail to Ben inquiring about updating and the mod_rewrite module and this is his reply:

==============================================
Thad,

This should not be much of an issue on angler. I'll have to get the root password from Frank. Will likely be either Friday or Monday before I'm able to roll on it though.

ben

Thad Chee wrote:
> Hi Ben,
>
> Is it possible for us to upgrade the apache server on angler from the 1.3 series of apache to the 2.2/4 series?
> For a number of reasons, we would like to have the mod_rewrite module installed on angler (and enabled on louis-pc) and if we are going to do that, we could also update the version in the process.
>
> One of the biggest reasons we would like to have mod_rewrite is to allow us to repackage complicated URL's into simpler ones. The best example of this is our showdoc interface.
>
> We would also like to use mod_rewrite as a tool to detect and phase out pages that have been superseded by later products but still allow the use of the older URLs. We have had a few trouble reports in the past few months where users were using very old versions of tools and we'd like to have them redirected into the newer versions of the tools.
>
>
> Thank you,
>
> Thad

--
=============================================
Words to ponder:
Anyone who has the power to make you believe
absurdities has the power to make you commit
atrocities.
--Voltaire
=============================================
Benjamin F Loyall, SSAI
Telephone : (757)-864-3217
Fax : (757)-864-7790
email : Benjamin.F.Loyall@nasa.gov

Mailing address:
Mail Stop 420
21 Langley Boulevard
NASA Langley Research Center
Hampton, VA 23681-2199
=============================================