China Sahana/VM Deployment–Day 7


This was an intense day. Gang Chen reported that the China team wants to accelerate plans to deploy the VM (Volunteer Management) module in Chengdu. As Xin S, China team project manager, put it, “people management is important…so you can fully speed up on volunteer management.” The priority for now is to finish translation of VM into Chinese.

Gang called us (Trishan, Giovanni, Antonio, and the me) at 11:30 AM with a new list of bug reports and feature requests. He reported that the China team had made changes to Sahana’s Disaster Victim Registry (DVR) that seem to have impacted the VM module. Given the independence between the various modules, the only way that changes in one module should cause bugs in another is if the changes involved some underlying functionality. From Gang’s initial report it sounded like the China team had made changes to the Sahana database.

The changes involved the Sahana’s geographical location system. Sahana’s location model is hierarchical: (country, state, city) or (country, province, city, locale) and so on. It is completely customizable, meaning that the user can specify the name and number of the levels in the hierarchy. This is usually done by admin when the system is initialized. Admin can set the top level of the hierarchy to a very broad (country) or very narrow (neighborhood) location, and any number of hierarchical levels can be specified:

defaultlocation.jpg

 

locationlevels.jpg

Sahana’s UI is well designed to allow great flexibility in setting up the location system. However, it is not well documented and it is easy to see that an inexperienced Sahana user might have trouble figuring out how to use it effectively.

It seems that the Chengdu team was not able to set up an appropriate location hierarchy. This was evident from the bug reports we received. The following screen shots show that although a 8-level location hierarchy was created (first image), victim locations are input (second image) using a single drop-down menu plus a text field:

locationhierbug.jpgvic_reg-1.jpg

After much discussion, it became evident that the China team wanted to flatten the hierarchy to a single level (hence the single drop-down) plus some free-form textual information. This was confirmed in a subsequent phone conference (see below).

After much discussion we drafted a message to the China team explaining what we think went wrong and how it could be fixed using Sahana’s built-in location infrastructure. Our primary concern was that the changes made to the location infrastructure in the DVR were the cause of the location errors in the VM and would cause similar errors in other Sahana modules. We recommended rolling back the changes and reconfiguring the location hierarchy (as a single layer) using Sahana’s built-in functionality. Gang Chen requested a conference call with the China team for later in the day.

We then went through the seven bug reports that Gang Chen received from the Sahana team, several of which had nothing to do with the location issues. Fortunately, most were straightforward to figure out. We went through each report in turn. Gang would translate the report from Chinese and then try to reproduce it using the demo system installed on IBM-China’s server. We would try to reproduce using our demo versions, one located on Giovanni’s server and one on my laptop. Our versions had been updated on Friday from files sent by Gang. One problem from this indirect approach is that it not always possible to determine if we are looking at the same version of the system. However, we were able to determine the causes for all the bugs. Here’s a rough transcript of our team analysis: Buganalysis.0524.txt.

The conference call ended at around 2:00 PM. After that Trishan, Giovanni, Antonio, and I met at Trinity to work on the bug fixes together. We synchronized our systems to our local VPN repository. We fixed two or three bugs by 5:00 PM when I had to leave for a BBQ. When I got back home at 11:30 PM the rest of the non-location bugs were fixed and Giovanni had transmitted our most recent build to Gang Chen who committed it to the IBM-China server. Here’s a summary of the fixes: Bugfixes.0524.txt.

At 11:30 PM or so I joined a conference call that Gang set up with the China team. Gang reported that the China team did not make any changes to the data model (database structure) but only to the data. That’s good. As we expected, they implemented a flattened, 1-level location hierarchy, as can be seen by comparing the 4-level hierarchy we use in our demo system:

// MySQL dump of location table.
INSERT INTO `location` (`loc_uuid`, `parent_id`, `opt_location_type`, `name`, `iso_code`, `description`) VALUES
('w50dlc-1', 'NULL', '1', 'USA', 'NULL', 'Country'),
('w50dlc-2', 'w50dlc-1', '2', 'Connecticut', 'NULL', 'The nutmeg state'),
('w50dlc-3', 'w50dlc-2', '3', 'Hartford', 'NULL', 'New England''s rising star'),
('w50dlc-4', 'w50dlc-3', '4', 'Trinity College', 'NULL', 'Soon to be D3 baseball champs');

with the 1-level hierarchy used in the China demo, in which all locations have a NULL parent and a 0 level:

INSERT INTO `location` VALUES ('110000','NULL','0','北京','BJ','UY');
INSERT INTO `location` VALUES ('110100','NULL','0','北京市市辖区','BJSSXQ','UYYYLA');
INSERT INTO `location` VALUES ('110101','NULL','0','北京东城区','BJDCQ','UYAFA');
INSERT INTO `location` VALUES ('110102','NULL','0','北京西城区','BJXCQ','UYSFA');
INSERT INTO `location` VALUES ('110103','NULL','0','北京崇文区','BJCWQ','UYMYA');
INSERT INTO `location` VALUES ('110104','NULL','0','北京宣武区','BJXWQ','UYPGA');

The China data are inherently hierarchical but the hierarchy is encoded in the last field. Thus UY is the code for a province and UYYYLA is a code for a city within the province. Given the current status of the location table, his hierarchy is not reflected in the Sahana parent and level fields. This would not be a problem as long as the MySQL field_options table were set up properly. To conform to this flattened hierarchy, it should contain a single entry for the opt-location-type parameter with a value 0. However, it was encoded as follows:

INSERT INTO `field_options` VALUES ('opt_location_type','6','镇');
INSERT INTO `field_options` VALUES ('opt_location_type','5','乡');
INSERT INTO `field_options` VALUES ('opt_skill_type','csk1','自定义技能-技能1');
INSERT INTO `field_options` VALUES ('opt_location_type','4','县');
INSERT INTO `field_options` VALUES ('opt_location_type','3','市');
INSERT INTO `field_options` VALUES ('opt_location_type','2','省');
INSERT INTO `field_options` VALUES ('opt_location_type','1','国家');
INSERT INTO `field_options` VALUES ('opt_location_type','7','村');
INSERT INTO `field_options` VALUES ('opt_location_type','8','组');

These table entries explain the 8-level hierarchy we observed (above) when we were discussing the VM bug reports. This conflict between the field_options and location tables acconts for the location bugs in the VM module.

During the conversation with the China team, they reported that they had not made any structural changes to the Sahana database. However, they did change the Sahana location library, specifically shn_location_add() function in the lib_location.inc file. The reason for this change was that the China team was convinced that they had discovered a “bug” in the Sahana library. The “bug” was that they were not able to have two locations with the same name. They modified the shn_location_add() function so they could add, say, Sichuan city and Sichuan province as separate geographic locations.

Despite our best efforts to convince the China team–with Gang Chen working as a real time translator–that this was not a “bug” but rather a consequence of the fact that their Sahana location hierarchy had not been properly configured, we were unable to do so. In their defense they have to use location data given to them by the Chinese government. These data look hierarchical to us but rather than process these data to fit Sahana’s hierarchical scheme–we estimate this would take a few hours of scripting–they inserted it as 1-level data into the location table. They are under pressure to deploy the victim registry (DVR) module in a few hours. The DVR module appears to be working properly now with their modified the library. So it’s not hard to see why they don’t want to change anything at this point.

Of course, we pointed out that the change in the library would impact other modules, as it had the VM, but to no avail. We offered to help fix the location conflicts. By running tests on our demo system, Giovanni was able to establish that it in fact is possible to have two locations, at the same level, with the same name. And Giovanni created the following SQL scripts, which would bring their MySQL tables into compliance with Sahana’s design:

update location set opt_location_type = 1;
delete from field_options where field_name = 'opt_location_type' and option_code <> 1;

But they didn’t want to change anything, with the DVR deployment looming.

This was pretty frustrating. We asked the China team if there were any representatives from the Sahana core team in Chengdu. They reported that there was but not a technical person. Despite our efforts throughout the day to reach the Sahana core team in Colombo, we were unable to do so. Our concern is that if the Sahana system is not configured correctly, it may fail to function properly. Although the DVR module is working now, will the library changes cause bugs to proliferate as other modules are brought online?

We spent the rest of the call (until 2:00 AM) fixing four or five new VM bugs that came to light. Gang Chen promised that we could all sleep late on Sunday morning!

It looks like the situation in Chengdu is chaotic and there is a rush to get the DVR deployed. I guess our role is to support the VM module as best we can under the circumstances.

  1. #1 by Gang Chen on May 25, 2008 - 8:01 pm

    I slept well to 11 AM May 25th, and hope rest of the team do the same:-) After then, good news is that the first Sahana China version is up and running officially.

(will not be published)