Reflections: RMH Homebase

I had a really great time working with the RMH homebase. It gave me a chance to work in some both familiar and unfamiliar territory. Over the course of the semester I have been working on another project in Drupal. Drupal, is written in PHP like the RMH homebase, and requires a SQL database to be set up for testing. With Drupal I learned how to use LAMP in linux. By the time I needed to use it in the RMH homebase, I was ready. Now there were some new unfamiliar experiences that I enjoyed tackling.

  • SimpleTest- It seemed many of my classmates, as well as myself, have not seen nor used SimpleTest before for unit testing in PHP/SQL. Thanks to advice in my classmates’ blogs I was able to set it up and get it to run for my LAMP server.
  • Refactoring– Many classes at the college go over errors and syntax in code, and they go over documentation and style, but they never teach us how to adapt. When I sat down to look at the RMH homebase code, I got a chance to try and familiarize myself with another programmer’s code. Not just that, I had to refactor it. Refactoring on such a large scale was difficult at first, but the netbeans IDE has a neat feature that allows you to search all the files within a project for certain terms. Also simply trying to understand an unfamiliar complex system quickly was a challenge. By reading the comments thoroughly, and exposing myself to as much code as possible, I was able to really familiarize myself with the key aspects of the code.
  • Considering all angles– Lastly, complex systems, like RMH homebase, taught me that you must look at every aspect of a system, every detail, every inch, and every bit to fully understand the problem and address all possible issues. For example, the login manager completed its task correctly, but it did not complete it with good security. Finishing a task does not mean the task is complete.

These experiences helped to make me a better programmer and software engineer in more ways than one. I am glad to have had the opportunity to work on the homebase and look forward to taking these lessons into my future career.

RMH homebase UPGRADED!

This week we finally got to upgrade our RMH homebase code to 2.0. This version includes some extra documentation, organization, functionality, and database tables. The following exercises refine this functionality and focus on SQL implementation.

 

7.1 Find examples of where the following criteria are broken in the dbDates.

 

>Every entry in the table has exactly one value of the appropriate type.

68 $shifts=$d->get_shifts();

69 foreach ($shifts as $key => $value) {

70 insert_dbShifts($d->get_shift($key));

 

$shifts is stored into value when it contains several different values and data types. This problem adds to the complexity of updating the values in the table, since you do not always know what types are being put into the table and in what order.

 

>No attribute in the table is redundant with the primary key.

 

58 $query=”INSERT INTO dbDates VALUES

59 (\””.$d->get_id().”\”,\””.

60 get_shifts_text($d).”\”,\””.$d->get_chef_notes().”\”,\””.$d->get_mgr_notes().”\”)”;

 

These values are all subsets of the primary key $shift. This can make it much more complex to search. If the primary key was only id, the search would be much easier.

 

7.2 Develop the functions get_shift_month, get_shift_day, get_shift_year, get_shift_start, and get_shift_end.

 

For this exercise I will only show the get_shift_month method, in order to prevent redundancy in this post:

 

function get_shift_month($id)

$shiftMonth = explode("-",$id);

return $shiftMonth[0];

 

This getter extracts the month from an array within the shift. Shifts follow the format MM-DD-YY-start-end. By creating an array separated at each dash, we can extract every value in an array created using explode() and return it to the parent function.

 

7.3 Design, implement, and test the calendar month view from the back of the book. This feature would allow the user to view all the shifts for an entire month. This feature requires us to add in a new module known as dbMonths. The simplest design I could use is code reuse. By using the code in dbWeeks and replacing the weeks with months. Below is the implementation of months:

 

include_once(‘Month.php’);

include_once(‘dbinfo.php’);

include_once(‘dbDates.php’);

 

/**

* Drops the dbMonths table if it exists, and creates a new one

* Table fields:

* [0] id: mm-dd-yy

* [1] dates: array of RMHDate ids

* [2] status: “unpublished”, “published” or “archived”

* [3] name: name of the month

* [4] end: timestamp of the end of the month

*/

function setup_dbMonths() {

connect();

mysql_query(“DROP TABLE IF EXISTS dbMonths”);

$result=mysql_query(“CREATE TABLE dbMonths (id CHAR(8) NOT NULL, dates TEXT,

status TEXT, name TEXT, end INT, PRIMARY KEY (id))”);

if(!$result)

echo mysql_error();

mysql_close();

}

 

/**

* Inserts a month into the db

* @param $m the month to insert

*/

function insert_dbMonths($m) {

if (! $m instanceof Month) {

die (“Invalid argument for dbMonths->add_week function call”);

}

connect();

$query = “SELECT * FROM dbMonths WHERE id =\””.$m->get_id().”\””;

$result = mysql_query ($query);

if(mysql_num_rows($result)!=0) {

delete_dbMonths($m);

connect();

}

$query=”INSERT INTO dbMonths VALUES

(\””.$m->get_id().”\”,”.get_dates_text($m->get_dates()).”,\””.

$m->get_status().”\”,\””.

$m->get_name().”\”,\””.

$m->get_end().”\”)”;

$result=mysql_query($query);

mysql_close();

if (!$result) {

echo (“unable to insert into dbMonths: “.$m->get_id(). mysql_error());

return false;

}

else foreach($m->get_dates() as $i)

insert_dbDates($i);

return true;

}

 

/**

* Deletes a week from the db

* @param $m the week to delete

*/

function delete_dbMonths($m) {

if (! $m instanceof Month)

die (“Invalid argument for delete_dbMonths function call”);

connect();

$query=”DELETE FROM dbMonths WHERE id=\””.$m->get_id().”\””;

$result=mysql_query($query);

mysql_close();

if (!$result) {

echo (“unable to delete from dbMonths: “.$m->get_id(). mysql_error());

return false;

}

else foreach ($m->get_dates() as $i)

delete_dbDates($i);

return true;

}

 

/**

* Updates a week in the db by deleting it and re-inserting it

* @param $m the week to update

*/

function update_dbMonths($m) {

if (! $m instanceof Month)

die (“Invalid argument for dbMonths->replace_week function call”);

if (delete_dbMonths($m))

return insert_dbMonths($m);

else return false;

}

 

/**

* Selects a week from the database

* @param $id week id

* @return mysql entry corresponding to id

*/

function select_dbMonths($id) {

if(strlen($id)!=8) {

die (“Invalid month id.”);

}

else {

$timestamp = mktime(0,0,0,substr($id,0,2),substr($id,3,2),substr($id,6,2));

$dow = date(“N”,$timestamp);

$id=date(“m-d-y”,mktime(0, 0, 0, substr($id,0,2), substr($id,3,2)-$dow+1, substr($id,6,2)));

}

connect();

$query = “SELECT * FROM dbMonths WHERE id =\””.$id.”\””;

$result = mysql_query ($query);

if (!$result) {

echo ‘Could not run query: ‘ . mysql_error();

$result_row = false;

}

else

$result_row=mysql_fetch_row($result);

mysql_close();

return $result_row;

}

 

/**

* retrieves a Month from the database

* @param $id = id of the week to retrieve

* @return the desired week, or null

*/

function get_dbMonths($id) {

$result_row=select_dbMonths($id);

if($result_row) {

$dates=$result_row[1];

$dates=explode(“*”,$dates);

$d=array();

foreach($dates as $i){

$d[]=select_dbDates($i);

}

$m=new Month($d, $result_row[2], $result_row[3], $result_row[4], $result_row[5]);

}

return $m;

}

 

/**

* the full contents of dbMonths, used by addMonth to list all scheduld weeks

* @return mysql result array of weeks

*/

function get_all_dbMonths() {

connect();

$query=”SELECT * FROM dbMonths”;

$result=mysql_query($query);

mysql_close();

return $result;

}

 

/**

* generates a string of date ids

* @param $dates array of dates for a week

* @return string of date ids, * delimited

*/

function get_dates_text($dates){

$d=”\””.$dates[0]->get_id();

for($i=1;$i<7;++$i) {

$d=$d.”*”.$dates[$i]->get_id();

}

return $d.”\””;

}

 

?>

 

After creating the module, I made the new module testdbMonths to help make assertions on the values of the database. The new code produced the correct results from the database and no bugs were overtly present.

 

These exercises were a great chance to try out SQL and PHP on open source software. I look forward to the final leg of the RHM source code exercises in chapter 8!

Challenges of the Software Age

This week, we got a chance to delve into some open software news sources. The task was to take two articles from opensource.com and post our personal response and reflect upon the articles argument or thesis. For my first article, I chose one about Louis C.K., the comedian. I am a lover of comedy, and Louis C.K. is one of my favorites. To be honest, I was little surprised to see an article about him even covered in a software magazine, but once I read the article I fully understood.

The article states its premise outright, “The answer to stabilizing content and price is letting artists retain greater control of their work.” This claim is not unheard of and is a sound premise. It is based off of Louis C.K. providing a download of his stand up special for five dollars a pop. This model was used instead of the production model to prove that ease of access will generate revenue, and that the current system of production is antiquated. Using this method, he spent approximately 250,000 dollars. Surprisingly, he generated 1,000,000 dollars in revenue. The author of the article is arguing that this electronic method provided better user-developer relations, and allowed Louis to make better quality comedy because he had control over the entire production.

This argument needs to be heard many times over, and not just in comedy. Software can benefit from this method of thinking as well. Over and over we hear of software losses due to theft through pirating software. However, as the author of the article is saying, the issue can be solved simply through ease of access and a reasonable pricing model. Another great example of this thought is in mobile application development. The rise of easy to afford, easy to install, and mobile apps demonstrates this key principle: price and production affect piracy. The current structure of the software world promotes attacking individuals for sharing files, and punishing paying users with inconvenient protection measures. In the end, removing this methodology helps customer relations by making the paying user feel less punished for choosing the right way.

Also mentioned in the article is price. Price for software can reach upwards of millions of dollars. How much of this cost is purely administrative? How much comes from unnecessary costs such as advertising and publishing? When you go to the store and look at the sixty dollar game, I can tell you a significant chunk of that price is going straight to the publisher, not the developer. By removing these middle men in the modern internet era, we can reduce the cost of software to a point where it is almost non-existent (open source anyone?). We can create better software by fostering a more direct relationship between the end user and the developer. We can create better software for a better price by being in greater control of our development process. This point is what I construed from the article. Developers must always be agile in the field of fast-paced technology. So why not start adapting now?

The second article I chose was called “A cure of the common troll”. By troll, they are referring to patent trolling. With the rising boom in software comes new technology and innovations. These new technologies can all be patented in order to protect the developer’s copyright. Some companies will arise and have risen, whose sole purpose is to collect patents and then sue infringers. This is the art of the patent troll. The results of this trolling are adverse. For one, patent trolling restricts innovation by preventing smaller companies from developing without being sued out of existence. Another notable reason is that many of these patents have been bought, sold, and traded. These patents are not the true inventors but people who buy or acquire these patents from the inventors for profit. By choosing to do so, they are going against the whole concept of a patent to begin with: to protect the inventor. Lastly, many of the patents are dealt with in an archaic manner. A common analogy is that it is like patenting the door knob or the wheel. These are basic universal components that just cannot be patented because they are so basic and necessary to software development.

The article suggests a way to deal with these trolls of the modern age:

“First, create a compulsory licensing mechanism for patents whose owners are not making competitive use of the technology in those patents. Patent owners should be required to declare the areas or products that incorporate the patented technology. All other non-practiced areas should be subject to a compulsory license fee. (A non-practiced “area” would be a market or technology sector or activity in which the patent owner is not using or licensing the invention rights, though the owner may be using the patent in other “areas.”) Licensing rates for patents could be set by patent classification or sub-classification based on industry average licensing rates for each such technology. Again, this would only apply to applications where the patent is not being practiced or voluntarily licensed by the patent owner.
Given the vast number of patents issued, an accused party should have a reasonable, set time after receiving notice of a patent within which to pay for the license going forward. Compulsory licenses are authorized by the treaties we have entered into, and we have significant experience with compulsory licensing of copyrighted works from which to develop an analogous patent mechanism. Uniform rates could be set.
Second, cap past damages for trolls at $1 million per patent and eliminate the possibility of obtaining injunctive relief for infringement of patents that are not in use, or are not used commercially, by the patent owner.
Third, a mandatory fee shifting provision should be put in place where the plaintiff is required to pay the defendant’s reasonable defense fees if the plaintiff does not obtain a better recovery than what was offered by the defendant. (Presently, there is such a cost shifting mechanism in place; however, the relevant costs typically are a tiny fraction of the legal fees in a case.)
Fourth, for U.S. domestic defendants, require that suits be brought in the venue where the defendant’s primary place of business is located.
Fifth, if a party wants more than limited discovery from the opposing side, particularly for electronically stored information (ESI), the requesting party should pay the cost of production. For large technology companies, ESI production alone can cost into the seven figures.”

I am a big supporter of all these concepts. I would also add to the list, that patents cannot be bought or sold, only inherited or renounced (made open to all). By doing so, patent companies would be insolvent and inviable. Each of the other suggestions from the author are great ideas and should be considered in updating our current system of patent application and distribution.

These two articles discussed some hot button issues in not just open source development, but also in all forms of software development. I particularly enjoyed this assignment and found the articles to be both informative and interesting. I look forward to reading more!

Exercises in Unit Testing

Today’s exercise is based of of chapter five in our software development book. I have to look at three exercises at the end of the chapter. Based on what I have seen and read, these exercises will be applications in testing and documentation within a FOSS project. Let’s get started.

5.1 Examine the RMH Homebase release 1.5 code base and accompying documentation. Identify at least one instance of the following:

a. Long Method
b. Too Few Comments
c. Data Clumps

This method comes from personEdit.php. It is over 140 lines long. The purpose of the function is to refine all the forms entered. As you can see in the code below. The code is effecting over 14 variables in this part of the code alone which could cause some hard to find side effects. There are few comments relating what each of these blocks effect, and the method tries to accomplish too much in this one function. There is so much going on that it could be its own php file. Also it appears every piece of data extraction is occurring in this function on the variable id. The constant use of id in so many different extractions leads to data clumping.

function process_form($id) {
//step one: sanitize data by replacing HTML entities and escaping the ‘ character
$first_name = trim(str_replace(‘\\\”,”,htmlentities(str_replace(‘&’,’and’,$_POST[‘first_name’]))));
$last_name = trim(str_replace(‘\\\”,’\”,htmlentities($_POST[‘last_name’])));
$address = trim(str_replace(‘\\\”,’\”,htmlentities($_POST[‘address’])));
$city = trim(str_replace(‘\\\”,’\”,htmlentities($_POST[‘city’])));
$state = trim(htmlentities($_POST[‘state’]));
$zip = trim(htmlentities($_POST[‘zip’]));
$phone1 = trim(str_replace(‘ ‘,”,htmlentities($_POST[‘phone1’])));
$clean_phone1 = ereg_replace(“[^0-9]”, “”, $phone1);
$phone2 = trim(str_replace(‘ ‘,”,htmlentities($_POST[‘phone2’])));
$clean_phone2 = ereg_replace(“[^0-9]”, “”, $phone2);

$private_notes = trim(str_replace(‘\\\”,’\”,htmlentities($_POST[‘private_notes’])));
$public_notes = trim(str_replace(‘\\\”,’\”,htmlentities($_POST[‘public_notes’])));
$my_notes = trim(str_replace(‘\\\”,’\”,htmlentities($_POST[‘my_notes’])));

$background_check = ”;
$shadow = ”;
$interview = ”;
if($_POST[‘background_check’]==’yes’) $background_check = ‘yes’;
if($_POST[‘interview’]==’yes’) $interview = ‘yes’;
if($_POST[‘shadow’]==’yes’) $shadow = ‘yes’;

$convictions = trim(str_replace(‘\\\”,’\”,htmlentities($_POST[‘convictions’])));
$wherelived = trim(str_replace(‘\\\”,’\”,htmlentities($_POST[‘wherelived’])));
$experience = trim(str_replace(‘\\\”,’\”,htmlentities($_POST[‘experience’])));
$motivation = trim(str_replace(‘\\\”,’\”,htmlentities($_POST[‘motivation’])));
$specialties = trim(str_replace(‘\\\”,’\”,htmlentities($_POST[‘specialties’])));
d. Speculative Generality

As for speculative generality, thew helpfooter.inc file clutters the code, and could have easily been made into a txt file for the user. Or better yet it could have been added to another method that deals with all footers, instead of being a repetitive include file.

5.2 For each of these “bad smells” refactor the code to reduce the size of the code base.

a. Long Method
b. Too Few Comments
c. Data Clumps

For these three I had to do a good chunk of work. I separated the process_form into different functions. This made the code more manageable and alleviated the Long Method and Data Clumps issues. This yielded four methods: process_address, process_notes, process_meeting, and process_background. Then I had to add comments to each explaining what they did and why they were there. This de-cluttered the code and turned a long method into a reasonable one.

d. Speculative Generality- I combined the helpfooter.inc with the footer.inc to remove one include file, and changed the reference to helpfooter.inc within the help.php file.

5.3 Perform unit tests

I ran unit tests on personEdit.php and help.inc that yielded all passes. Overall I am satisfied with the results of my refactoring though I will need to add in more test cases later to make sure I have checked absolutely everything necessary.

This weekend I and my team will be working on setting up a project schedule for the rest of the semester. See you all then!

Bug Exercises Part II: Patching

Today’s exercises involve one of the more basic skills that all programmers should know: patching. In the open source world patching has become pretty standardized, except for certain minutiae in formatting. So for the exercise my job was to create some test patches and understand the test process using the Unix terminal.

7.2.1

This exercise was creating a basic diff output using none other than the diff command. The output of this diff corresponded to the books results and is reprinted below.

steve@ubuntu:~$ diff -u hello.c hello.c.punct

— hello.c 2012-02-09 23:50:18.659577870 -0500

+++ hello.c.punct 2012-02-09 23:51:37.780125196 -0500

@@ -5,6 +5,6 @@

#include

int main() {

– printf(“Hello, World.\n”);

+ printf(“Hello, World!\n”);

return 0;

}

7.2.2
Then the next exercise was to examine the differences made by not having a -u. By not having the -u we see that the formatting of the output has changed. There is less information and is more bare representation.

steve@ubuntu:~$ diff hello.c hello.c.punct

8c8

printf(“Hello, World!\n”);

7.8
This exercise was the creation of patch file containing the word bar. First I created the file “foo” containing “bar” and used diff on it with the null file provided by Unix. The resulting output was created in a file named “foopatch.patch”. Redundant I know. Just did not want to lose the foo.c

— foo.txt 2012-02-09 23:58:58.439593808 -0500
+++ /dev/null 2012-02-09 23:48:23.021831997 -0500
@@ -1 +0,0 @@
-bar

7.9
Finally this exercise had me making a patch file using a real program. The program is called caultelis. I took the echo.c file and changed a small snippet of code to compare to echo.c.reverse. The following output was created, which matches the output the book says I will get.

— src/echo.c.reverse 2012-02-10 00:06:29.018333108 -0500
+++ src/echo.c 2012-02-10 00:10:10.938330675 -0500
@@ -258,14 +258,14 @@
}
else
{
– while (argc > 0)
+ while (argc > 0)
{
– fputs (argv[0], stdout);
argc–;
– argv++;
+ fputs (argv[argc], stdout);
if (argc > 0)
putchar (‘ ‘);
}
+
}

if (display_return)

These exercises were a great experience. Considering we have our first bug fix due Monday I was glad to get this practice in for making a patch to my team’s Drupal project. This weekend I will update the results of creating my first official patch for Drupal.

Bug Exercises Part One

While working on the team bug to be submitted Monday, each of us needed to complete some exercises in our open source online textbook. These exercises correlate directly with our team’s project and will prove valuable in becoming better at managing open source projects.

6.4 Find the oldest bug that’s still open in your chosen project. Write a blog entry describing the problem, with a theory about why the bug hasn’t been resolved yet. (Bonus points if you can actually resolve the bug.)

There is a bug that is 6 years and 27 weeks old. The bug is with the bug reporting system itself within a drupal project. Each bug can be classified in various ways. When it is classified as “patch (ready to be committed)” the bug is removed from the issues list. The bug was resolved in the comments but left open because of other issues discussed within the comments.

6.5 Figure out how to create a new account on the bug tracker of your chosen project. You’ll need that account very soon.

I had actually created an account a few days ago to post about a bug I saw. The name of the account is steveo1490.

6.6 Go through your project’s bug tracker and find a bug that you think you might be able to reproduce — and then try to reproduce it in the latest build. Take careful notes. Report your experiences as a comment to the bug. If you can reproduce the bug, great! Give as much information as you can. If you can’t reproduce the bug, great! Give as much information as you can, and ask the original reporter if there are other steps you might be able to take to reproduce the bug.

A bug I was working on this past weekend I was able to successfully reproduce. When in the administrative theme, the edit, add, revision, and delete pages will not come up properly if you change the lettering into anything other than all lowercase. I tried quite a few combinations such as Edit, EdIT, and EDIT. Each version did not show the administrative theme which shows the bug’s existence.

6.7 Find five bug reports in the newstate, and attempt to triage them according to the rules above. Your goal is to do as much as you possibly can, in a short period of time, to make those bug reports as useful as possible to the developer to whom they are assigned. (Note: be sure to follow any triage rules that your project may have defined. If there are no set triage rules, be sure to announce your intentions on the project’s mailing list, so that developers can provide you some guidelines if they choose.)

After reviewing some of the newer bugs. I noticed most of the bugs are triaged into the correct categories. The only slip ups I noticed that might make development and bug fixing more difficult is prioritizing. The Drupal bug tracker has four priorities: normal, minor, major, critical. Almost all new bugs were categorized as normal and some of the major bugs I looked at were either dupes, which I commented on and reduced the priority on, or were just not a functional issue. Being a nonfunctional bug lowers the priority down to at least the normal to minor level. I manged to change the priority on a few which I hope will give developers the right glimpse at what bugs need to be fixed now rather than later.

These exercises gave me a significant amount of experience with the bug tracker, as well as bug tracking organization. The triage techniques were also useful to learn, so I can use them in my future computer science projects that involve code maintenance. This Friday I will be posting about even more exercises that help foster my bug fixing talents.

Freeciv Experimentation

Today’s assignment was to do a simple assignment in our open source online book. Appearances can be deceiving however. For this assignment we had to build a program called freeciv from source code using a step by step guide. Now there were two apparent problems when I first looked at the guide. For one the guide is two years old so a lot could of changed with freeciv and its dependencies since then. Also the guide is a loose guide in that it only accounts for major problems not small ones. Instead the author encourages you to just google answers instead of providing advice on them himself.

So I downloaded the files using subversion to my machine with no problems. I opened the install file and read the requirements for the program. So I started to go through these requirements one by one while looking at the author’s instructions. I check for the gcc package using rpm and it says no package found. Well that is no problem I can just use yum to install it. I type in the yum install command and it says there is no such package as well… how peculiar. After that I double check my commands and authority to make sure I am root. I then start searching the net for the issue. Unfortunately no one seemed to have this problem. I then saw that there is a way to verify ‘make’ being in your packages. Using ‘make -v’ I see I have make. I try to duplicate the command with gcc: ‘gcc -v’. Presto! There is a version of gcc on my system, but why didn’t yum or rpm see it? I decide to ignore it and keep going for now. As I try to fulfill more requirements I keep getting the no package found error. Yum could not find anything.

At this point the frustration kicks in. I scour the internet for problems with yum and find nothing relating to my problem. After two hours of searching, I review the instructions and think back to what I noticed on the various forums I visited: red hat and fedora. Don’t tell me… wait for it…. Yep! I did not realize yum is for fedora users only while I am on an Ubuntu system. So that whole time I should have been using apt-get to install all my necessary packages not yum. Time to try again. I start succeeding with a few packages (autoconf and gettext) but then I see some are not found again! Knowing I am in a different system I start looking up these packages for Ubuntu. They are under different names of course! So with haste I start looking up each missing package and installing.

Finally I have all the packages. Time to to use autogen.sh. Success for the first few minutes. It is finally building! After a few minutes I sigh dejectedly as I see another failure. The author even mentions the problem, gtk2 needs to have its dependencies installed using a develop version. Back to Google. In the following few minutes I found the package I needed and restarted my autogen.sh. This time to my elation there were no failures. I finally saw the “type make” line. I slowly key in make in anticipation. After hours of work the source code is finally built. I push the enter key. The make begins and with this step I had completed the assignment.

I am confident the lessons learned in this assignment will assist me in building our team’s project code this weekend.