REMnux

REMnux is a lightweight Linux distribution for assisting malware analysts with reverse-engineering malicious software. The distribution is based on Ubuntu and is maintained by Lenny Zeltser.

John H. Sawyer March 22, 2012

In my last blog about Linux Live Environments, I mentioned REMnux, an environment specifically built for malware analysis. I’d spent a little time with REMnux when it first came out, but decided to take the latest version (3.0) for a test drive.

Since I just received the new “Practical Malware Analysis” book from No Starch Press, the detailed lab exercises seemed like a perfect way to test out the tools included in REMnux. While most of the tools in the book are Windows-based, there are Linux-based equivalents found on REMnux.

The first task was downloading the lab files linked from http://practicalmalwareanalysis.com/labs and extracting them.

If you have issues try https://sourceforge.net/projects/labs-encryptzip/ for an encrypted zip download.

WARNING: The lab binaries contain malicious code and you should not install or run these programs without first setting up a safe environment.

Compatibility: The labs are targeted for the Microsoft Windows XP operating system. Many of the labs work on newer versions of Windows, but some of them will not. The labs are designed to mimic realistic malware. Some of them are well-written code that runs reliable and some of them (just like real malware) are poorly written code that may crash, contain memory leaks, or otherwise behave unexpectedly.

Practical Malware Analysis Labs Web Site

My plan to solely use REMnux was immediately thwarted by the self-extracting Windows executable that contained the lab files. There was a EULA (end user license agreement) wrapper as part of the executable that had to be accepted before the files could be extracted — a problem solved with a quick boot of a Windows XP virtual machine. Ideally, the authors will replace or supplement the self-extracting executable with a standard zip file.

After a quick and easy read through the first few chapters, I started to dig into the lab examples using REMnux’s tools. Chapter 1’s labs want you to upload the example binaries to VirusTotal to see whether any antivirus products detect them as possible malware. While I could have used Firefox to upload the files, I chose to use pyew’s “vt” plugin that searches VirusTotal using the MD5 hash of the file instead of uploading the actual file.

Searching just using the MD5 could have backfired if no one had uploaded the file to VirusTotal yet, but given the popularity of the book, the lab files had already been uploaded many times. It doesn’t appear that REMnux currently comes with a command-line tool to upload files to VirusTotal, so here are a couple of options (#1 and #2) that simply require that you get a free API key from VT first.

A few other tasks in the first few “Practical Malware Analysis” labs include looking at executables’ import/export functions, compile date, and packer identification. Pescanner works pretty well to figuring out those answers, but not all of them. For example, pescanner identifies suspicious import functions but does not list them all, and it incorrectly identified some of the executables as having been packed when they weren’t. Pyew faired better by being able to list all imports and exports and correctly identifying the packer, but it did not have an obvious way to show the compile date. But when used together, you can get the exact answers you need.

The last lab in Chapter 1 asks you to use Resource Hacker to look for resources that are stored in the file. To accomplish the same task on Linux, a command-line tool can be used called hachoir-subfile. Running hachoir-subfile against the Lab 1-4 executable will extract the embedded in PE file.

I’m looking forward to digging into the book more and using REMnux further to see just what I can do without having a Windows virtual machine. It’s not that I have anything against Windows — I just like to find alternative Linux-based tools that do the same thing as Windows-based tools. There’s definitely an advantage to having both platforms available for analysis with the plethora of tools to use.

I highly recommend taking a look at the “Practical Malware Analysis” if you’re interested in the topic, as it is one of the best books I’ve seen on subject and the labs are great. I suspect I’ll have more blogs in the future about the book and tools as I spend more time with it and find alternative tools to use for the analysis.

John Sawyer is a Senior Security Analyst with InGuardians. The views and opinions expressed in this blog are his own and do not represent the views and opinions of his employer. He can be reached at johnhsawyer@gmail.com and found on Twitter @johnhsawyer.

 

Bootable USB Flash Drive

There are many ways to create a live USB drive carrying an operating system like Ubuntu, but the method I will describe further is mainly based on using SUN’s VirtualBox. While the method described on the Ubuntu documentations implies installing a Live CD image on a USB flash drive, which would then need to extract and load the operating system in the RAM, the method that I will describe on this page implies installing a fresh operating system on a bootable flash drive that will work the same way as from a real HDD (except the speed, of course). Thus, you should have a good bootable USB 2.0, with decent I/O data processing speeds, with at least 4GB (considering that the operating system itself weighs ~2GB, Karmic Koala).
Divide your USB flash drive into two partitions

In order to separate the operating system from the documents you would like to save on the flash drive, it is advisable that you divide your USB flash drive into two partitions. Only do this if your USB flash drive has more than 2GB of space and you do not need to save changes you make inside the operating system. In order to achieve this, you need to have GParted installed (or at least this is what i prefer). If you are not following this tutorial on a Linux machine, then you’ll have to use whatever software you best know that works with your operating system (on Windows I recommend Acronis Disk Director and Partition Magic). To get GParted type the following command in a terminal:

sudo apt-get install gparted

Now backup all data you have on your USB flash drive, because we will need to format it and create two partitions. Haven’t backed up your data? You’re playing with fire!

http://mintarticles.com/read/operating-systems-articles/how-to-install-portable-linux-ubuntu-on-a-bootable-usb-flash-drive-from-sun-virtualbox,13641/


Advanced Topics And Tools
Grub4DOS
Super Grub Disk
Guide for MultiBoot USB-stick with boot.ini Menu

UNetbootin (Universal Netboot Installer) is a cross-platform utility that can create live USB systems and can load a variety of system utilities or install various Linux distributions and other operating systems without a CD.

This installation mode creates bootable USB flash drives and bootable USB Hard Disk Drives; it is a Live USB creator.

Multiple installs on the same device are not supported.

It is worth noting that UNetbootin’s meta-data is very out of date. For example, the latest version of Linux Mint offered in the drop-down menu is version 10, whilst the latest official release is version 14 (at time of writing, February 2013). However, UNetbootin can still be used to write a bootable Mint 14 ISO file onto a USB device, if the user first downloads the ISO file manually.


Utility And Rescue Bootable USB Flash Drive

These can contain just about any tool you want: anti-virus, OS boot cd’s, OS repair / recovery discs, programs, etc.) This is a slightly more difficult section depending on exactly what you want on your flash drive. This can also be time consuming.

Different programs and bootable Windows and Linux ISO’s require different boot parameters. Which is why some things work with one program and not another.

SARDU, XBOOT, and YUMI can create a multiboot utility flash drive but each officially support different programs / ISO’s.
EasyBCD can create multiboot flash drives but requires you to PAY ATTENTION when configuring.
You’ll have to find which actually work best for you.
I haven’t found one that does everything I would like it to (do all the programs below and work) so I’ve got 2 utility / rescue flash drives.

Places to find help for the above programs.
1 – See the links on those programs home pages.
2 – Reboot.Pro
3 – 911CD

Additions Tested :
– Windows 7 Recovery Discs (32 & 64-bit)
– Windows Vista Repair Discs (32 & 64-bit)
– UBCD4Win (SP3 slipstreamed pre-build, nlited to add drivers and update packs)
– Hiren’s Boot CD 14
– openSUSE 11.4 LiveCD (KDE)
– Linux Mint LiveCD (Gnome & KDE) (if Linux Mint works then Ubuntu should too)
– AVG
– Avira
– Kaspersky
– Microsoft Standalone System Sweeper
– Acronis True Image Home 2011
– Acronis Disc Director 11

Below are some notes on each program as of this writing.
They are not intended to bash anyone, they are just the results I came up with.

SARDU ( 2.0.3 beta 6)
– Do not rename ISO’s.
– openSUSE does not work.
– Hiren’s Boot CD support removed AFAIK due to it’s questionable legality. (download v2.0.2c if you need this)
– UBCD4Win does not work in this version. (download v2.0.2c if you need this)
– Microsoft Standalone System Sweeper supported.
– To add Acronis to this see here.

Make sure all your ISO’s are in one folder.
Do not rename the ISO’s.
If you haven’t already downloaded them. Click the button next to the check box will take you to the download page.
Plug in your preformatted flash drive
Click the CD/ISO picture (upper left) to load the ISO folder.
ISO’s already in the folder will be preselected.
Click the Search USB button on the right to find your flash drive.
Click the picture of the flash drive below it to start the process of making your bootable flash drive.

SARDU creates a multiboot USB drive, a multiboot DVD or multiboot CD (all-in-one) for free (personal and non commercial use, read the license). Hard disks (internal and external), SSD, USB flash drive and all removable memory disk and media are supported.
The multi bootable device can include comprehensive collections of antivirus rescue CD, utilities and popular Linux live distributions. Windows PE can also be included, as well as recovery disks and install media for Windows XP (Professional, Home and 64 Bit), Windows Vista, Windows Seven and Windows Eight.

How To Install Linux With Ease Using UNetbootin
http://www.makeuseof.com/tag/install-linux-with-ease-using-unetbootin/

Create a Bootable Linux Flash Drive in Three Easy Steps
http://www.pcworld.com/article/249870/create_a_bootable_linux_flash_drive_in_three_easy_steps.html


Yumi (Pendrivelinux.com) offers to be the successor of the MultibootISOs program.

I also found Sardu (sarducd.it), it is MORE GUI then the GUI of Yumi, and also offers more implicit support for Windows-based ISO’s.

Both programs come with download link and a list of confirmed distributions that work.

Placing all ISO’s in Yumi’s executable folder should add them all in once, but I got adding multiple iso’s only working with Sardu.


Best 20 Methods To Create a Bootable USB Flash Drive [Windows, Linux]

A search led to a recommendation that I try XBOOT. Another source suggested that SARDU and XBOOT both might be more robust than YUMI.

Were there other possibilities? An AlternativeTo webpage listed Universal USB Installer (of which UNetbootin was apparently a clone) and WinToFlash as much more popular than SARDU, XBOOT, or YUMI, but these did not appear to be multiboot solutions. That is, they would load only one program onto the USB drive. At this point, Wikipedia’s list of tools to create live USB systems did not distinguish multiboot from single-boot tools — but it did make clear that there were many single- or multiboot tools out there. One source offered a way to use UNetbootin to create a multiboot flash drive, but it, too, sounded complicated. A search suggested that EasyBCD was another possibility, but it appeared that it was a boot manager that would let you decide whether to boot from, say, a hard drive partition containing Windows 7 rather than another partition containing Linux.

So I took it as a choice among YUMI, SARDU, or XBOOT. A search led to a thread with several user reports that tended to favor YUMI. As I had also found, one comment recommended formatting within YUMI rather than formatting the USB drive via Windows Explorer. One blogpage, written in spring 2011, seemed to find little difference in capabilities, between SARDU and YUMI, except that SARDU had the advantage of allowing the user to burn a CD or DVD containing one (or possibly more) installer. Two other webpages praised SARDU, but without offering specific comparisons against alternatives like YUMI. The XBOOT webpage seemed to indicate, as others had done, that XBOOT was preprogrammed to accept far fewer programs and distributions than YUMI; the same had also seemed to be true of SARDU.


YUMI (Your Universal Multiboot Installer), is the successor to MultibootISOs. It can be used to create a Multiboot USB Flash Drive containing multiple operating systems, antivirus utilities, disc cloning, diagnostic tools, and more. Contrary to MultiBootISO’s which used grub to boot ISO files directly from USB, YUMI uses syslinux to boot extracted distributions stored on the USB device, and reverts to using grub to Boot Multiple ISO files from USB, if necessary.

Aside from a few distributions, all files are stored within the Multiboot folder, making for a nicely organized Multiboot Drive that can still be used for other storage purposes.

Creating a YUMI Multiboot MultiSystem Bootable USB Flash Drive
YUMI works much like Universal USB Installer, except it can be used to install more than one distribution to run from your USB. Distributions can also be uninstalled using the same tool!


XBOOT is yet another neat little Multiboot ISO USB Creator. It is a Windows based application that can be used to create a Live Multiboot USB or even a Multiboot ISO file that can then be burnt to a CD/DVD. XBOOT supports many Linux Distributions and Utilities, and allows you to use your choice of a Grub or Syslinux bootloader. Also included is a built in QEMU emulator (enabling you to boot an ISO from within Windows).

 

XBOOT – Multiboot ISO/USB Creator

Authors Website: http://sites.google.com/site/shamurxboot/


Gujin is a GPLed bootloader for the PC.

This software boots your PC and analyse your filesystems. It displays a graphical menu for you to select which system to boot. Intended to replace LILO and Loadlin, written in C with GCC, fully real mode.

Users

Download gujin boot/system loader files

 

Project detail and discuss

Get support

What can Gujin do?

It can read FAT12, FAT16, FAT32, ext2, ext3, ext4 (with constant inode size) and ISO 9660 filesystems.

It has a graphical user interface with mouse support, and can be installed on any media: floppy, hard/USB disk partitions, hard/USB disk MBR, CD/DVDROM, DVD-ram (i.e. FAT with 2048 bytes/sectors). It can also use a serial port as input/ouput instead of the screen and keyboard.

Gujin can chain-load other bootloader, load Linux kernel, has an unfinished loader of multiboot specification, but more importantly for our current interest it can load standard ELF files (more exactly GZIP compressed ELF files).

There is two major ELF variant on the PC: ELF32 (with 32 bits load address, sizes and entry point) and ELF64 (with 64 bits fields), gujin loads any of them and switch the processor in protected mode to jump to the ELF entry point.

Gujin will not try to set-up memory paging at all, that is the job of the kernel to select which kind of paging it wants – so even the 64 bits ELF files will have to handle the transition from 32 bits to 64 bits themselves (because you need paging to go to 64 bits mode).

Same for interrupts, Gujin switches to protected mode but does not re-enable interrupts, that is the job of the kernel being booted to set-up the Interrupt Decriptor Table and handle each interrupts, because BIOS will not help the kernel any more.

Gujin can also relocate the ELF file if it contains relocation information, see option –emit-relocs of the “ld” linker of the “binutils” toolchain.

Because most applications will want to collect BIOS informations before the switch to protected mode, Gujin can call in real-mode a function of the ELF file, and if this function returns zero – continue the loading process – else display an error message (for instance: “trying to execute a 64 bits application on a processor without this feature!”).

Moreover, in most cases the kernel can decide to return to the Gujin bootloader (if it did not erased it from memory), without forcing a reboot.

The Gujin bootloader is built using standard Linux tools, it does not need Linux to run (only a not-too-buggy BIOS) – but we have to assume you are running Linux to use the GNU toolchain: GCC, binutils, GZIP… to produce the ELF file for the kernel. Also, Gujin installer needs either Linux 32 bits or Linux 64 bits to run and install the Gujin bootloader on a device.

Because Windows uses another executable format, you cannot install cygwin/MinGW on windows and use the compilation toolchain directly, in this case you would have to generate a cross compiler toolchain to produce ELF files, and that is out of scope for this description (but not that difficult).


So enough text description, let’s try it!

The floppies being out-dated, let’s say we want to use a USB stick as our test media. In some cases, we may want to use a SD card, basically the process is the same.

We first need to check that the target PC will be able to boot that USB stick or SD card, when Gujin is installed on it.

To have increased chances of success, we will ask the Gujin installer to reformat completely this USB disk or SD card, so first backup any interresting file you have in some other place.

Then, download this gujin executable if you are using a 32 bits Linux, and that gujin64 executable if you are using a 64 bits Linux:

If you do not trust anybody, download the source file gujin-*.tar.gz, extract it in a directory and type “make” – that will produce an executable named “gujin” (even on a 64 bits Linux).

Remember to visit http://gujin.org to check if there is a newer version of Gujin, and to increase Gujin author counters and keep him happy.

Then, we reformat that dedicated USB stick as a bootable FAT filesystem, erasing all its content: first go in “root” by typing “su” or “sudo” (distribution dependant), then get the device name of you USB stick (let’s say it is /dev/sdg) (sometimes /dev/mmcblk0 for SD cards), and type:

 ./gujin /dev/sdg --disk=BIOS:0x00,auto

Depending on the size of the USB stick, that will have created either a FAT16 or a FAT32 (or even a FAT12) filesystem, but that point is not really important.

You then unplug this device, and replug it: most distribution will automatically mount the filesystem and display a window of its content: only a single file which is the bootloader itself.

To check that this filesystem is correctly created, you can type:

 /sbin/fsck.vfat /dev/sdg

With the Gujin installer parameters we used, that would have created a “superfloppy” format on our USB stick, that is currently the format most PC will be able to understand and boot from.

That does not mean your own PC will 100% boot it, due to BIOS bugs – so you need to test now that this USB stick is bootable by your PC or not: umount the USB stick, plug it in the test PC and power it on, see if Gujin is started (you will notice easily).

If it is not started try to check:

 - that the boot order in the BIOS is set to boot USB devices first
 - try the different USB devices if your BIOS has switchable items.
 - try to tell the Gujin installer to use the Extended BIOS instead by typing (--disk=EBIOS:0x00,auto is the default):
 ./gujin /dev/sdg
 - try to tell the Gujin installer to generate a real disk and not a superfloppy by:
 ./gujin --mbr /dev/sdg --disk=BIOS:0x00,auto
 - try the two previous options together:
 ./gujin --mbr /dev/sdg --disk=EBIOS:0x00,auto
 - try to use another (smaller) USB stick, some BIOS will only accept to boot from a FAT16 superfloppy

By now you should know a lot more about your BIOS, and have a bootable USB stick.


Then, we want to generate this ELF kernel – let’s try to generate a “hello world”: Create a file with that content:

 const char msg1[] = "Hello protected-mode text world! please reboot ..."; 
 #define STACKSIZE 64 * 1024 
 static unsigned stack[STACKSIZE / 4] __attribute__ ((aligned(32))); 

 void _start (void) 
 { 
 	/* We are flat non-paged memory and interrupt disabled */ 
 	asm (" mov %0,%%esp " : : "i" (&stack[STACKSIZE / 4])); 
 	volatile unsigned short *video_array = (volatile unsigned short *)0xB8000; 
 	unsigned cpt1; 

 	video_array += 10 * 80; /* few empty lines */ 
 	/* We want blue background color and lightgray foreground, so 0x1700: */ 
 	for (cpt1 = 0; cpt1 < sizeof(msg1) - 1; cpt1++) 
 		video_array[cpt1] = 0x1700 + msg1[cpt1]; 

 	while (1) 
 		continue; 
 }

Then compile it like (you may need to add “-fno-stack-protector” too, distribution dependant):

 $ gcc -m32 -Wall -O2 -s -static -nostartfiles -nodefaultlibs -Wl,-Ttext=0x110000 hello.c -o hello.elf

and compress it by:

 $ gzip -9 hello.elf -c > hello.kgz

You just need to copy that hello.kgz file into the USB stick, and reboot with this USB stick, you will get a menu with “hello.kgz” displayed.

If you click on this filename, you will have “Hello protected-mode text world! please reboot …” displayed (Because this hello-world do not manage graphic modes, you have to “force start kernel in text mode” in Gujin configuration).

If you want a bit more complex “hello world” applications, you should download Gujin install*.tar.gz pack .

you will find there few KGZ files, like previous example but with some addresses displayed to show the exact position of the application in memory.

You will see hello_32bits.kgz and hello_32bits_r.kgz to show the effect of relocation, and hello_64bits.kgz and hello_64bits_r.kgz to show 64 bits applications.

To get the source code, just download the file gujin*.tar.gz, and extract the corresponding c files.

To see the command used to compile them, just search the string “hello_32bits.kgz:” (with semicolon) in the file Makefile.

If you just want a real mode “hello world”, you can do that too with an ELF file where the load address is null for the code segment.

There is a limit of 64 Kbytes for real mode ELF file.

Just create a file which contains:

 /* hello_bios.c - public domain - no support whatsoever */

 /* Constant entry block, max 64 KB code+data+stack, do not touch: */
 asm (
 "	.code16gcc				n"
 "	.section .init,"ax",@progbits		n"
 "	.global _start				n"
 "_start:					n"
 //  "int $3 # if debugging with borland td.exe, started by tiny.exe	n"
 "	pushw	%ds				n"
 "	pushw	%es				n"
 "	pushw	%fs				n"
 "	pushw	%gs				n"
 "	pushfl					n"
 "	pushal					n"
 "	movw	%ss,%cs:1f+1			n"
 "	movw	%sp,%cs:2f+2			n"
 "	movw	%cs,%ax				n"
 "	movw	%ax,%ds				n"
 "	movw	%ax,%es				n"
 "	movw	%ax,%fs				n"
 "	movw	%ax,%gs				n"
 "	movw	%ax,%ss				n"
 "	movl	$0,%esp				n"
 "	cld					n"
 "	calll	main				n"
 "	movl	%eax,%cs:3f+2			n"
 "1:	movw	$0,%ax				n"
 "	movw	%ax,%ss				n"
 "2:	movl	$0,%esp				n"
 "	popal					n"
 "	popfl					n"
 "	popw	%gs				n"
 "	popw	%fs				n"
 "	popw	%es				n"
 "	popw	%ds				n"
 "3:	movl	$0,%eax				n"
 "	lretw					n"
 "	.previous				n"
 );

 /* For information about asm() construct, see GCC manual:
 http://gcc.gnu.org/onlinedocs/gcc-4.5.0/gcc/Extended-Asm.html
   For information about BIOS services, see
 http://stanislavs.org/helppc/
 */
 enum standard_color {
 	black,		blue,		green,		cyan,
 	red,		magenta,	brown,		lightgray,
 	darkgray,	lightblue,	lightgreen,	lightcyan,
 	lightred,	lightmagenta,	yellow,		white
 	};

 /* This assumes 16 colors modes, text or graphic - i.e. not 256 colors: */
 static inline void
 VGA_writestring (const char *string, unsigned char row, unsigned char col)
 {
 	unsigned short page_attribute = 16 * blue + lightgray, strlen_string;
 	const char *endstring = string;

 	/* We cannot rely on libc to provide strlen(): */
 	while (*endstring)
 		endstring++;
 	strlen_string = endstring - string;

 	asm (
 "	xchgl	%%ebp,%4		n"
 "	int	$0x10 # writestring	n"
 "	xchgl	%%ebp,%4		n"
 		: : "a" (0x1301), "b" (page_attribute),
 		"c" (strlen_string), "d" (((unsigned short)row << 8) | col),
 		"r" (string)	/* in fact %es:%bp, but "B" (string) doesn't work */
 		);
 }

 static inline void BIOS_wait (unsigned nb_microsecond)
 {
 	unsigned short status;

 	/* Use volatile when there is at least one result, but even if the result
 		is not used, the code has to be inserted (status is the result): */
 	asm volatile ("	int	$0x15	"
 		: "=a" (status)
 		:  "a" (0x8600), "d" (nb_microsecond & 0xFFFF), "c" (nb_microsecond >> 16)
 		);
 }

 int main (void)
 {
 	VGA_writestring ("Hello BIOS world! Please wait...rn", 0, 0);
 	BIOS_wait (3 * 1000 * 1000); /* 3 seconds before returning to Gujin*/
 //	return 0x80000000; /* to return to Gujin menu */
 	return 0xA0000000; /* to return to Gujin menu, do not display "press a key to continue" */
 }

And then compile it by:

 $ gcc -m32 -Wall -Os -s -static -nostartfiles -nodefaultlibs -Wa,-R -Wl,--section-start,.init=0 hello_bios.c -o hello_bios.elf

and compress it:

 $ gzip -9 hello_bios.elf -c > hello_bios.kgz

you can copy this hello_bios.kgz to the USB stick and run it, I hope the different functions of the source file are obvious.

The Gujin bootloader is willing to let the real mode and protected mode kernel to access the bootloader internal data (like what is the current video mode, how to display strings…) as long as the kernel is licensed under the GPL license – i.e. when the GZIP kernel contains a GZIP comment describing its license. That is illustrated by cleandisk.kgz compiled form this cleandisk.c source code and linked with this linker file.

The main point about the GPL license is that it is not possible to draw a line in between Gujin possible bug and a closed source application: it is no allowed to modify Gujin data while in real mode – but there isn’t any enforcement by hardware.

Obviously, Gujin will enable you to run an ELF file with a real mode part AND a protected mode part, just have a look at the example hello_gpl.kgz and its source code hello_gpl.c.

See Also

Articles

Threads

External Links

  • Gujin at sourceforge.net

Anon

Index of /files

programming class

Questions Addressed:

How Programming Languages Evolved
How the computer stores data
Numbering systems the computer likes
Different data types
Different programming Syles
Procedural
Functional
Object Oriented
What does it mean when a language is “strongly” or “weakly” typed
What is compiling and do I need to do it?
When I would do it, and when I would not
Why, what advantage does it provide
Modern Computer Languages Overview
Bash
Perl
Python
Ruby
C
C++
Java
Vala
C#/Mono
Programming methodologies
Waterfall
RAD
Summary of Graphical Programming Libraries
GTK
QT
FLTK
SDL
And finally, Programmers tools:
Eclipse
NetBeans
Anjuta

This is a lot of topics, and this week will be an overview. It should get you enough information to recognize the labels on the map, even if you are not 100% sure where the map will lead you. That will be the task for the following weeks.

eXtreme Programming

Uno de los problemas fundamentales con las metodologías de desarrollo, de hecho, con cualquier esfuerzo de normalizar un proceso entre personas, es que el deber ser en un sentido moral idealista obscurece el es. eXtreme Programming es un enfoque contra intuitivo para aumentar la productividad de los programadores.

A pesar de los esfuerzos heroicos del equipo de mercadotecnia y de la necesidad de los usuarios de mantener los costos bajos, un programador es productivo alrededor de 2 a 4 horas diarias en promedio. Un monstro en el closet pero una realidad. Esto anuado al hecho de la programación es una arte en al que unos pocos virtuosos pueden realizarla con soltura, 5% de los programadores (o menos) hacen 95% del trabajo (o más). Por eso los beneficios de programación en pares en realidad no implican un costo en productividad. Antes al contrario, probablemente un equipo de 2 de programadores trabajando bajo el esquema de programación extrema sea 2 a 3 veces más productivo que los mismos programadores trabajando de manera aislada.

El énfasis en diseño y pruebas es simplemente una realidad del ciclo de desarrollo:

  • Un defecto en codificación es un defecto, aunque corregirlo puede generar más defectos.
  • Un error en la fase de diseño produce más de 10 defectos en código
  • Un error en la fase de levantamiento de requerimientos produce más de 100 defectos en código

Por eso el esfuerzo de desarrollo debe concentrarse en el análisis y realizar iteraciones cortas donde rápidamente la funcionalidad del sistema sea aparente al usuario final y este pueda dar la retroalimentación necesaria para mantener las cosas en la dirección correcta de manera eficaz.

El esfuerzo de desarrollo debe seguir aproximadamente la siguiente ponderación:

  • 40 % análisis y diseño
  • 5 % codificación
  • 30 % pruebas y soporte
  • 25 % más análisis, diseño, pruebas.

Referencias

diagrama xtrem programming

Technorati Tags: ,,,,,,,,,,,,,,,

Los operadores de copia e igualdad.

Una de las trampas de la orientación a objetos en lenguajes como C# son los operadores de copia e igualdad.

En C# al usar el operador = o ==, la igualdad entre objetos referenciados solo se da si en realidad es el mismo objeto y de manera similar al hacer una copia nos podemos llevar una sorpresa si no tenemos cuidado de estar copiando la referencia o el contenido. Por eso la interfaz ICloneable es controversial porque su significado es ambiguo

Referencias:

Implementar ICloneable mediante serialización

ICloneable Interface

Should we Obsolete ICloneable (The SLAR on System.ICloneable)

IClonable deep vs shallow, best practise

Creating and Using Attributes in your .NET application

IEquatable Generic Interface

Libreria empresarial del grupo de patrones y practicas de Microsoft

La última versión de la Enterprise Library del grupo de Patterns and Practices de Microsoft se libero en mayo del 2007 y es compatible con .Net 2.0 y 3.0.

Información actualizada y material didactico se puede localizar en el sitio comunitario de la libreria empresarial.

Enterprise Library: La evolución de los .NET Application Blocks de patterns & practices

Entlib – Enterprise Library es la evolución de los Bloques Aplicativos .NET que han sido desarrollados por el Grupo PAG (Microsoft Platform Architecture Guidance) dentro de Microsoft. Este grupo genera guías y arquitecturas de referencia, patrones de diseño, y código fuente desarrollado con la implementación de diversos escenarios tecnológicos.

Los desarrolladores en su momento puden usar la guía para comprender las mejores prácticas referenciadas y sugeridas por Microsoft para aplicaciones .NET; o incorporar el bloque aplicativo como tal dentro de sus desarrollos, en su formato original y/o extendido.

Los “Bloques Aplicativos .NET” que en su momento fueron liberados son los siguientes:

Por la forma gradual en que fueron desarrollados dichos bloques aplicativos, los mismos estaban desintegrados y la experiencia de utilización e extensibilidad eran diferentes entre si. Además que la utilización de cada uno de dichas piezas de software obligaba a la instalación de componentes de software independientes.

Con estas áreas de oportunidad la nueva versión de los “.NET Application Blocks” se integró con la nueva etiqueta de Enterprise Library. El grupo de PAG ha anunciado lo siguiente:

Entlib es una librería de activos de software reutilizable que atenderá los retos comunes en el desarrollo del software empresarial.

Entlib está focalizado en la consistencia, extensibilidad, fácil utilización e integración de los diversos bloques aplicativos existentes y futuros.

Es importante aclarar que Enterprise Libray no es un producto como tal, sino que es un componente de software que es proporcionado como está, pero del cual se puede contratar soporte directamente de Microsoft, tratado bajo un esquema parecido al código escrito por los usuarios.

Recursos relacionados:

  1. Web Service Software Factory
  2. Smart Client Software Factory
  3. Web Client Software Factory
  4. Guidance Automation Extensions and Guidance Automation Toolkit

Requerimientos:

  • Sistema oprativo: Windows Server 2003; Windows Vista; Windows XP

Note: If you already have the Enterprise Library 3.0 installed, you must uninstall it before installing the Enterprise Library 3.1. However, you can install the Enterprise Library 3.0 or the Enterprise Library 3.1 when 2.0 is already installed.

  • Microsoft .NET Framework 2.0 or 3.0. You need .NET Framework 3.0 for the
    Application Block Software Factory and the WCF adapters for the Validation
    Application Block and Exception Handling Application Block
  • Microsoft Visual Studio 2005 development system (any of the following
    editions):

    • Microsoft Visual Studio 2005 Standard Edition
    • Microsoft Visual Studio 2005 Professional Edition
    • Microsoft Visual Studio 2005 Team Edition for Software Developers
    • Microsoft Visual Studio 2005 Team Edition for Software Testers
    • Microsoft Visual Studio 2005 Team Edition for Software Architects
    • Microsoft Visual Studio 2005 Team Suite

  • To use the Application Block Software Factory and the Strong-Naming
    Guidance Package, you need the Guidance Automation Extensions (GAX). To
    modify these guidance packages, you also need the Guidance Automation
    Toolkit (GAT).
  • Some blocks and samples require the use of Microsoft SQL Server or other
    database products.
  • Visual Studio Team System or NUnit 2.2 is required if you want to
    execute unit tests.

Referencias:

Enterprise Library 3.1 – May 2007

Juan Carlos Lozada’s WebLog

Acropolis

More Information

 

  Enterprise Library
The patterns & practices Enterprise Library is a library of reusable and extensible application blocks designed to assist developers with common enterprise development challenges. Enterprise Library 3.0 contains application blocks for Caching, Cryptography, Data Access, Exception Handling, Logging, Policy Injection, Security and Validation.
  Caching Application Block
The Caching Application Block is a component of Enterprise Library which provides a flexible and extensible caching mechanism for use in client and server-side .NET development projects.
  Smart Client – Composite UI Application Block
Are you building applications with complex user interfaces? Do you want to take full advantage of the power of the Microsoft Windows desktop? Check out this recently released application block that provides guidance on building world-class, enterprise ready, client applications. Available both in C# and Visual Basic .NET.
  Cryptography Application Block
The Cryptography Application Block is a component of Enterprise Library which makes it easier to include cryptographic functionality in .NET applications. The block provides a simple interface to DPAPI, symmetric encryption and hashing, and uses the Enterprise Library configuration tool to simplify key management.
  Data Access Application Block
The Data Access Application Block is a component of Enterprise Library which reduces the amount of custom code that you need to create, test, and maintain when building data access layers in .NET applications.
  Exception Handling Application Block
The Exception Handling Application Block is a component of Enterprise Library that makes it easier to implement consistent exception handling policies at logical tiers in an application. Exception policies can be configured to perform tasks such as logging exceptions, wrapping or replacing exception types.
  Logging Application Block
The Logging Application Block is a component of Enterprise Library that allows developers to instrument their applications with logging and tracing calls. Log and trace messages can be filtered, formatted and routed to a choice of trace listeners, including the event log, text files, database or WMI.
  Policy Injection Application Block
The Policy Injection Application Block is a component of Enterprise Library which allows developers to specify the crosscutting behavior of objects in terms of a set of policies. Crosscutting concerns are the necessary tasks, features, or processes that are common across different objects. Examples are logging, authorization, validation, and instrumentation.
  Security Application Block
The Security Application Block is a component of Enterprise Library that builds on the capabilities of the Microsoft .NET Framework to help you perform authentication, authorization, check role membership and access profile information.
  Validation Application Block
The Validation Application Block is a component of Enterprise Library which provides a common approach to defining validation rules for your business objects that allows them to be reused across different layers of your application.
  Web Service Facade for Legacy Applications
This guide discusses best practices for interfacing with legacy applications by using Microsoft® ASP.NET Web services and the Microsoft .NET Framework. The .NET Framework provides the foundation for creating a Legacy Application Interface solution using Microsoft technologies. This guide provides a sample solution using a Microsoft FoxPro® database as the legacy application and connecting it to a .NET-based application using ASP.NET Web services and SOAP. The specific technologies involved are ASP.NET, C# or the Microsoft Visual Basic® .NET development system, the .NET Framework, XML, Visual Basic, COM, and ADO.

La reutilización de código

Ahora, como antes, más que antes, como siempre, la reutilización de código se presenta como un valor fundamental en el desarrollo de sistemas. Uno de sus aspectos es la interoperabilidad de los códigos. Por decirlo de alguna manera, la compatibilidad de una aplicación con diferentes versiones de un sistema operativo y con diferentes sistemas operativos.

Por un lado Microsoft, por otro los demás. El imperio contra los rebeldes republicanos y los feudos vecinos. Pero dentro del mismo imperio se hablan distintas lenguas y los rebeldes tienen diferentes agendas.

Una aplicación que trabaja en Windows 95 no necesariamente funciona en Windows XP, Visual Basic 6 y Visual Basic .Net son animales distintos. Una aplicación Linux que funciona en la distribución Red Hat no necesariamente funciona en la distribución SuSe. La frase platform independent source en la practica marca una prueba de iniciación para hechiceros.

En el caso de Microsoft, algunas de estas incompatibilidades son de origen mercadológico. ¿Cuál es la diferencia entre Windows XP Home Edition y Windows XP Pro? Limitaciones artificiales en la versión casera con respecto a la versión profesional. Desde el punto de vista de Microsoft este modelo funciona, Vista no tiene 2 versiones distintas sino n, cada una definida por un segmento de mercado. Las utilidades de MS aumentaron 65% con respecto al año pasado y podemos esperar más de lo mismo por lo menos en el corto plazo.

En el caso del movimiento Open Source las incompatibilidades son de origen sociocultural. Distintos grupos trabajan con combinaciones distintas de herramientas y enfoques metodológicos. Estos herramentales se yuxtaponen unos con otros y las combinaciones son infinitas. La versión de gcc pude ser la diferencia clave para que un paquete se construya correctamente.

Una iniciativa que no termino de entender es Mono. El concepto es bueno, pero ya va un par de veces que trato de construir una aplicación .Net para fallar miserablemente. Al revisar la letra chiquita del readme aparece que la aplicación es Mono ¿Cuál es el caso de incluir archivos de solución y proyecto de Visual Studio si VS no puede construir la aplicación? ¿Si se requiere reproducir el ambiente de trabajo del desarrollador con librerí­as y variables de entorno porque no documentar esas dependencias? Entiendo que son pecadillos del bien intencionado pero se me escapa la motivación fundamental del chango.

http://www.go-mono.com/docs/
http://www.codeproject.com/cpnet/hellomono.asp

Un aspecto problemático del desarrollo í­nter plataforma son las interfaces graficas de usuario (GUI). Cada sistema operativo tiene su look-and-feel caracterí­stico y el manejo eficiente de ventanas requiere el uso del API nativo correspondiente.

Un enfoque que se puede tomar es agregar una capa intermedia entre la aplicación y el sistema operativo que abstraiga la interacción entre la capa lógica y la interfase grafica a cambio de una penalización en el rendimiento. Algunos problemas que se pueden presentar con librerí­as de este tipo:

  • El uso de una capa intermedia adicional disminuye el rendimiento de la aplicación.
  • La librerí­a necesaria para soportar la funcionalidad adicional de múltiples sistemas operativos aumenta le tamaño de las aplicaciones más alla de lo que se justifica con la funcionalidad de la aplicación misma. Por lo mismo el soporte para plataforma móvil no es adecuado
  • La apariencia de la aplicación no corresponde a la de una aplicación nativa y los diálogos son distintos a los que los usuarios usan normalmente.
  • La necesidad de definir un mí­nimo común denominador hace que se pierda la oportunidad de usar las características más avanzadas de un sistema operativo en particular.
  • El uso de librerí­as fuera de la esfera de influencia del sistema operativo anfitrión saca a la aplicación del ciclo de vida del mismo y dificulta el proceso de mantener alineadas las actualizaciones de la aplicación con cambios en el sistema operativo.

La tabla siguiente muestra un comparativo de librerí­as para desarrollo ínter plataforma.

Liberarí­a Tamaño (MB) Tamaño comprimido (MB)
Java 30+ 15
GTK+ 9+ 4
QT 4+ 2
wxWidgets <1 <.5

Java es una norma abierta que funciona bien como propuesta í­nter plataforma. La maquina virtual de java (JVM) aísla la aplicación del sistema operativo anfitrión y esta disponible normalmente en todas partes. Sin embargo las aplicaciones de java tienden a ser chupa recursos. Si revisas los procesos en una estación XP con Firefox instalado, Firefox es normalmente el campeón en memoria utilizada.

GTK+ es un grupo importante de bibliotecas o rutinas para desarrollar interfaces gráficas de usuario (GUI) para principalmente los entornos gráficos GNOME, XFCE y ROX de sistemas Linux. GTK+ es la abreviatura de GIMP toolkit (conjunto de rutinas para GIMP). Es software libre (bajo la licencia LGPL), multiplataforma y parte importante del proyecto GNU. Inicialmente fue creado para desarrollar el programa de manejo de imágenes GIMP, sin embargo actualmente es muy usada por muchos otros programas en los sistemas GNU/Linux. Cabe mencionar que Qt es una alternativa a GTK que también es muy utilizada (en el entorno KDE, por ejemplo).

GTK+ se ha diseñado para permitir programar con lenguajes como C, C++, Java (Sun), Perl o Python.

GTK ha sido portada a Windows pero el look-and-feel no es nativo.

Qt es una biblioteca multiplataforma para desarrollar interfaces gráficas de usuario. Fue creada por la compañía noruega Trolltech. Qt es utilizada en KDE, un entorno de escritorio para sistemas como GNU/Linux o FreeBSD, entre otros. Utiliza el lenguaje de programación C++ de forma nativa y además existen bindings para C, Python (PyQt), Java (Qt Jambi), Perl (PerlQt) y Ruby (QtRuby) entre otros. El API de la biblioteca cuenta con métodos para acceder a bases de datos mediante SQL, así­ como uso de XML y una multitud de otros para el manejo de ficheros, además de estructuras de datos tradicionales. Inicialmente Qt apareció como biblioteca desarrollada por Trolltech (en aquel momento “Quasar Technologies”) en 1992 siguiendo un desarrollo basado en el código abierto, pero no libre. Se usó activamente en el desarrollo del escritorio KDE (entre 1996 y 1998), con un notable éxito y rápida expansión.

Qt cuenta actualmente con un sistema de doble licencia: una GPL para el desarrollo de software de código abierto (open source) y software libre, y otra de pago para el desarrollo de aplicaciones comerciales. Las librerí­as Qt son también liberadas bajo licencia GPL para Windows y Mac.

wxWidgets son unas bibliotecas multiplataforma, freeware/Open Source para el desarrollo de interfaces gráficas programadas en lenguaje C++. Es una librería pequeña que encapsula en una interfase común llamadas al API nativo de cada sistema operativo.

wxWidgets usan una licencia GPL, concretamente la licencia L-GPL, similar a la GPL con la excepción de que el código binario producido por el usuario a partir de ellas, puede ser propietario, permitiendo desarrollar aplicaciones empresariales sin coste.

Las WxWidgets proporcionan una interfaz gráfica basada en las bibliotecas ya existentes en el sistema (nativas), con lo que se integran de forma óptima y resultan muy portables entre distintos sistemas operativos. Están disponibles para Windows, MacOS, UNIX/Linux, OpenVMS y OS/2. También pueden ser utilizadas desde otros lenguajes de programación, aparte del C++: Java, Javascript, Perl, Python, Smalltalk, Ruby