C++ - LIBUMEM Best Simple Memory Leak Detection Tool Ever

A Simple usage of libumem
  1. export LD_PRELOAD=libumem.so
  2. export UMEM_DEBUG=default
  3. export UMEM_LOGGING=transaction
  4. Execute the program collect the core
  5. Debug using core file

bash-3.00# mdb core

Loading modules: [ libumem.so.1 libc.so.1 libuutil.so.1 ld.so.1 ]
> $G

C++ symbol demangling enabled
> ::findleaks
> BUFCTL $<bufctl_audit


BUFCTL is the address outputted by the ::findleaks

  • http://developers.sun.com/solaris/articles/libumem_library.html
  • http://blogs.sun.com/amith/entry/detecting_memory_corruption_with_libumem
  • http://docs.sun.com/app/docs/doc/816-5041/casestudy-31?a=view
  • http://blogs.sun.com/hema/entry/libumem_to_detect_modify_after
  • http://docs.sun.com/app/docs/doc/816-5041/6mb7ae3ld?a=view


Labels:

Posted by - at 9:57 pm | 0 comments read on

C++ - Object Heap Marker to Detect Memory Leaks

In this post I have a program which generates core and from the core file we can find out how many objects for a particular class has been created. This helps in isolating memory leaks related issues. Using this method we can analyze release mode core dumps so as to see the values within each objects.

The basic idea is to put a heap marker string along with each object so that we can identify the object start and end by going through the core dump. What I am doing here is create an array within the class which will have a predefined string so as to identify the object. Now whenever we create that object it gets persisted into heap and at any time we dump the heap we can see that string in heap thereby identifying the start and end of an object. I have tested all these in redhat linux 8.0.

Here is the Class which has markers



#define MARKERSIZE 5
#define BEGMARKER "<A>"
#define ENDMARKER "</A>"
#include <iostream>
class A
{
char beginA[MARKERSIZE ];
int i;
char b;
char endA[MARKERSIZE];

public:
A() : i(0xabcd), b('X')
{
std::cout << "CTOR" << std::endl;
strcpy(beginA,BEGMARKER);
strcpy(endA,ENDMARKER);
}

~A()
{
std::cout << "DTOR" << std::endl;
}

void displayme()
{
printf("ADDR BEG = %p\n",beginA)
printf("ADDR I = %p\n", &i);
printf("ADDR B = %p\n", &b);
printf("ADDR END = %p\n",endA)
}
}


Now the main function which create 10 such objects and generates a core. After which we will analyze the core file to say how many objects for class A got created


#include <sys/types.h>
#include <unistd.h>
#include <signal.h>
#include <A.hpp>

int main()
{
for(int i=0 ; i<10 ; ++i)
{
A *a = new A();
//a->displayme();
//delete a;
}

//send a SIGSEGV signal to generate core
pid_t pid = getpid();
kill(pid,11);
}

Before running the program make sure that you have set the ulimit properly to generate core dumps.
 
-bash-2.05b# ulimit -c unlimited
-bash-2.05b# ./a.out > /dev/null
Segmentation Fault (core dumped)
-bash-2.05b# od –c core.1233| grep "< / A >"|wc –l
11


Here the count is showing as 11 because core also has the string literal entry for ENDMARKER. So 11 – 1 = 10. Just by going through the core file we are able to tell how many objects of A reside in memory.

Now using od itself you can analyze the objects and figure out the values which got populated into the object. While going through the values within the object make sure you take care of the endianess.

Labels: , , ,

Posted by - at 9:05 pm | 0 comments read on

C# - Command line Tools n Modularizing IF-ELSE

I have been writing some of the command line tools which handles lots of functionality. I used to end up writing lots of IF-ELSE which are non reusable code. What I think we should do to make such tools really extensible is defining the commands within an XML file.

<command name="displayallnames" class="[class and assembly name]" />

Once a person enters a particular option we traverse through the input XML file to pick the required class and call necessary methods.
An interface ICommand like defines what we expect from such commands.

namespace code.grep.in.commandline
{
enum ExecResult
{
int SUCCESS = 0;
int FAILURE = 1;
int INVALID_USAGE = 2;
}
interface ICommand {
ExecResult execute(Arguments args);
void usage();
}
}

Now what we can do is in the command line tools is to read the XML file and using reflection call the corresponding classes execute method.

I will post the complete implementation soon!

Labels: ,

Posted by - at 7:48 pm | 3 comments read on

SharePoint - SSP Personalization Services Permissions

In /ssp/admin/manageservicepermissions.aspx datasources are used to map these permissions. So it seems it gets stored inside a table within SQL

<ItemTemplate>
<asp:LinkButton runat="server"
ForeColor='<%#DataBinder.Eval(Container.DataItem, "Color")%>'
OnClientClick='<%#DataBinder.Eval(Container.DataItem, "Script")%>'
Text='<%# DataBinder.Eval(Container.DataItem, "UserName")%>'
OnClick="EditUsers"
ToolTip='<%# DataBinder.Eval(Container.DataItem, "UserName")%>'/>
</ItemTemplate>



SSP service permissions are stored as a text field under dbo.MIPObjects of SSP's database. It uses 6 bits to store Manage Analytics, Manage Audiences, Manage User Profiles, Personal Features, Personal Site, Set Permissions and i can go through enabling each to find the flags.

<object>
<field name="PersistedXml" type="string">
<?xml version="1.0" encoding="utf-16"?>
<Acl>
<Ace IdentityName="NT AUTHORITY\Authenticated Users"
DisplayName="NT AUTHORITY\Authenticated
Users" SID="***"
Rights="3" />

<Ace IdentityName="NT AUTHORITY\NETWORK SERVICE"
DisplayName="NT AUTHORITY\NETWORK SERVICE"
SID="**"
Rights="63" />
<Ace IdentityName="fbamembers:admin"
DisplayName="admin"
SID=""
Rights="63" />

<Ace IdentityName="fbaroles:users"
DisplayName="Users"
SID=""
Rights="3" />
</Acl>
</field>
</object>

Posted by - at 9:58 am | 0 comments read on

Windows - 013 Useful Windows Commands

Useful Windows XP commands

  1. Merge two files into another - copy smalefile1 + smallfile2 largefile
  2. Decompress Files - extract
  3. pushd and popd same as that for bash.
  4. Associate a folder to a drive - subst X: C:\windows
  5. Tree structure for directory - tree
  6. Find files or find string with in a file - find/findstr
  7. Hiding files within another file(NTFS Filesystem Specific) - type 1.jpg > 1.txt:hidden.jpg
  8. Alias a command - doskey ls=dir
  9. Starting Add/Remove Programs - appwiz.cpl
  10. Show listening process and port - netstat -no|findstr "LISTEN"

Labels: ,

Posted by - at 11:13 am | 0 comments read on

Linux - 013 Linux Bash Tips

0xa Linux Bash Commands which helps me most in my daily work.


1. cd -, swaps take you to the last cd last location

2. cd, Change to home directory

3. pushd, pushes the current directory path into stack and later you can use popd to go back to that location, use dirs to display the stack

4. fg, mainly for programmers who are editing and compiling the from command line. Eg. Start vim make some changes to the file use :w to save the contents press Ctrl-Z that process will get stopped. Now compile the code then use fg to bring back that to foreground.. Use jobs to see all stopped process and use bg to run those in background.

5. Ctrl-L, to clear the screen.

6. Ctrl-R to reverse search the previous commands

7. !xyz will execute the last xyz command with all the previous arguments.

8. Ctrl-A to go to the beginning for the line, Ctrl-E to the end of the line,

9. Ctrl-U erase all characters backward, Ctrl-W erase a word backward

10. shopt -s cdspell, will correct minor errors in cd command.

Labels:

Posted by - at 11:12 am | 0 comments read on

SharePoint - Permissions

SharePoint Permissions
Using Object Model (OM) we can get all resources and its heirachy details. OM provides RoleAssigment property for each SPWeb/SPList/SPListItem object which will list down the role assignments. Now using this we can collect all the RoleAssignment for all Web->List->ListItems.


Sample RoleAssigment property
XML representation of RoleAssigment for web resource (/sites/mysite) is given below. Each permission has a member/group id associated and a OR’ed mask which signifies the role (limited access, full control, read, design, contribute). For detailed description about each mask go through the SPBasePermission section.
Code

Console.WriteLine(list.RoleAssignments.Xml);


Output


<permissions>
<permission memberid=3 mask=9223372036854775807 />
<permission memberid=4 mask=756052856929 />
<permission memberid=5 mask=1856436900591 />
<permission memberid=7 mask=206292717568 />
<permission memberid=8 mask=206292717568 />
</permission>

RoleAssignment property internals
I used the following code to retrieve RoleAssigments for SPList

foreach (SPRoleAssignment ra in list.RoleAssignments)
{
foreach (SPRoleDefinition role in ra.RoleDefinitionBindings)
{
// check ra.Member.Name, role.Name , role.BasePermissions
}

}





XML Description
<permission memberid=3 mask=9223372036854775807 /> “mysite Owners” has “Full Control”
memberid = 3 “mysite Owners” group
mask = 0x7FFFFFFFFFFFFFFF FullMask
<permission memberid=4 mask=756052856929 /> "mysite Visitors" group has "Read" access
memberid = 4 “mysite Visitors” group
mask = 0xB008431061 ViewListItems
| OpenItems
| ViewVersions
| ViewFormPages
| Open
| ViewPages
| CreateSSCSite
| BrowseUserInfo
| UseClientIntegration
| UseRemoteAPIs
| CreateAlerts
<permission memberid=5 mask=1856436900591 /> "mysite Members" group has "Contribute" access
memberid = 5 "mysite Members" group
mask = 0x1B03C4312EF ViewListItems
| AddListItems
| EditListItems
| DeleteListItems
| OpenItems
|ViewVersions
|DeleteVersions
|ManagePersonalViews
|ViewFormPages
|Open
|ViewPages
|CreateSSCSite
|BrowseDirectories
|BrowseUserInfo
|AddDelPrivateWebParts
|UpdatePersonalWebParts
|UseClientIntegration
|UseRemoteAPIs
|CreateAlerts
|EditMyUserInfo
<permission memberid=7 mask=206292717568 /> "ABCD\\killer" has "Limited Access"
memberid = 7 "ABCD\\killer"
mask = 0x3008011000 ViewFormPages
|Open
|BrowseUserInfo
|UseClientIntegration
|UseRemoteAPIs
<permission memberid=8 mask=206292717568 /> "ABCD\\police" has "Limited Access"
memberid = 8 "ABCD\\police"
mask = 0x3008011000 ViewFormPages
|Open
|BrowseUserInfo
|UseClientIntegration
|UseRemoteAPIs


SPBasePermission

SPBasePermissions (enum) Mask (hex) Description
EmptyMask 0x0000000000000000 Grant no permissions.
FullMask 0x7FFFFFFFFFFFFFFF Grant all permissions.
ViewListItems 0x0000000000000001 Allow viewing of List Items in Lists, Documents in Document Libraries, and Web Discussion comments.
AddListItems 0x0000000000000002 Allow addition of List Items to Lists, Documents to Document Libraries, and Web Discussion comments.
EditListItems 0x0000000000000004 Allow editing of List Items in Lists, Documents in Document Libraries, Web Discussion comments, and to customize Web Part Pages in Document Libraries.
DeleteListItems 0x0000000000000008 Allow deletion of List Items from Lists, Documents from Document Libraries, and Web Discussion comments.
ApproveItems 0x0000000000000010 Allow approval of minor versions of a List Item or Document.
OpenItems 0x0000000000000020 Allow viewing the source of Documents with server-side file handlers.
ViewVersions 0x0000000000000040 Allow viewing of past versions of a List Item or Document.
DeleteVersions 0x0000000000000080 Allow deletion of past versions of a List Item or Document.
CancelCheckout 0x0000000000000100 Allow discard or check in of a Document which is checked out to another user.
ManagePersonalViews 0x0000000000000200 Allow creation, change, and deletion of Personal Views of Lists.
ManageLists 0x0000000000000800 Allow creation and deletion of Lists, addition or removal of Fields to the schema of a List, and addition or removal of Public Views of a List.
ViewFormPages 0x0000000000001000 Allow viewing of Forms, Views, and application Pages, and enumerate Lists.
Open 0x0000000000010000 Allow access to the items contained within a Site, List, or Folder.
ViewPages 0x0000000000020000 Allow viewing of Pages in a Site.
AddAndCustomizePages 0x0000000000040000 Allow addition, modification, or deletion of HTML Pages or Web Part Pages, and editing the Site using a Windows SharePoint Services compatible editor.
ApplyThemeAndBorder 0x0000000000080000 Allow application of a theme or borders to the entire Site.
ApplyStyleSheets 0x0000000000100000 Allow application of a style sheet (.css file) to the Site.
ViewUsageData 0x0000000000200000 Allow viewing of reports on Site usage.
CreateSSCSite 0x0000000000400000 Allow creation of a Site using Self-Service Site Creation, an implementation-specific capability of Windows SharePoint Services.
ManageSubwebs 0x0000000000800000 Allow creation of Subsites within the Site or Site Collection.
CreateGroups 0x0000000001000000 Allow creation of a group of Users that can be used anywhere within the Site Collection.
ManagePermissions 0x0000000002000000 Allow creation and modification of permission levels on the Site and assigning permissions to Users and Site Groups.
BrowseDirectories 0x0000000004000000 Allow enumeration of Documents and Folders in a Site using [MS-FPSE] and WebDAV interfaces.
BrowseUserInfo 0x0000000008000000 Allow viewing the information about all users of the Site.
AddDelPrivateWebParts 0x0000000010000000 Allow addition or removal of personal Web Parts on a Web Part Page.
UpdatePersonalWebParts 0x0000000020000000 Allow updating of Web Parts to display personalized information.
ManageWeb 0x0000000040000000 Allow all administration tasks for the Site as well as manage content.
UseClientIntegration 0x0000001000000000 Allow use of features that launch client applications; otherwise, Users must work on Documents on their local machines and upload changes to the WFE.
UseRemoteAPIs 0x0000002000000000 Allow use of SOAP, WebDAV, or [MS-FPSE] to access the Site.
ManageAlerts 0x0000004000000000 Allow management of alerts for all Users of the Site.
CreateAlerts 0x0000008000000000 Allow creation of email alerts.
EditMyUserInfo 0x0000010000000000 Allow a user to change his or her own User information, such as adding a picture.
EnumeratePermissions 0x4000000000000000 Allow enumeration of permissions on the Site, List, Folder, Document, or List Item.

Posted by - at 12:43 am | 0 comments read on

Readable Code - Remove Checking null

Readable Code – Checking NULL

We must have come across these lines of code many times

//JAVA
bool func(String name)
{
if ( (name != null) && (name.equals("true") ) {
...
} else {
...
}
}


A better way to code this would be


//JAVA
bool func(String name)
{
if ( "true".equals(name) ) {
...
} else {
...
}
}

Labels: ,

Posted by - at 8:36 pm | 0 comments read on

SharePoint - Windows SharePoint Services (WSS) Overview and Architecture

Intend
Off lately I have started working on technologies based on C# and ASP.NET. Initially I used to concentrate on scalability and performance optimization in C#. Now I have started looking into SharePoint technology. While I was searching for the materials on SharePoint there were a lot of materials which describes the functionality part but I was not able to find any single document that helped someone new to C# and ASP.NET in understanding SharePoint. In this article I am trying to cover the basics of SharePoint so as to enable someone new with these technologies all of the content which I am putting down over here are collected from various sites given inline and in Reference sections so all courtesy to them.

  • Overview of ASP.NET Applications
  • SharePoint Technology Stack
  • Windows SharePoint Services v2 vs v3
  • Windows SharePoint Services Architecture


Overview of ASP.NET Application
All most all of us are aware about server side scripting languages where the processing of the code takes place in the server and final HTML output is send back to the client for browser rendering. Similarly MS has a technology which is similar to that which is ASP.NET. ASP.NET has a very rich set of controls and features that enables us to write scalable and extensible web applications. So before we proceed to I would like to touch upon how ASP.NET is being handled by MS web server named Internet Information Server (IIS).



Within IIS we have a mapping list for extensions which map *.aspx to a particular DLL name which will be processing the request for resources of type *.aspx (eg: http://code.grep.in/sample.aspx). Now whenever a request is received by server it redirects the request to corresponding DLLs which are called Internet Server API (ISAPI) extensions. Now similarly they have ISAPI filters, ISAPI extensions are end points where they process the most part of the request. ISAPI filters can be cascaded in the sense we can have multiple ISAPI filters through which the request can go. There are specific hook point where we can plug-in these ISAPI filters. So for processing *.aspx request we have extensions which does all the ASP.NET file parsing and response building. ASP.NET also has something like ISAPI filters and ISAPI extensions and they are HTTP Modules and HTTP Handlers respectively.

SharePoint Technology Stack

Figure 2 shows the technology stack related to SharePoint. Light blue components are the basic foundation on which SharePoint is built upon. What SharePoint returns back by consuming those fundamental services are colored dark blue and dark red.We don’t have to understand all of it you will be able to co-relate few of them by the end of this article.



There are rich set of feature supplied by Core Workspace Services which can be referred to as Windows SharePoint Services (WSS, the blocks in dark blue) and the Applications/Portals can be referred to as MOSS (Microsoft Office SharePoint Server, the blocks in dark red).MOSS is built on top of WSS technology so it extends the WSS features and provides much more new features and tighter integration with Microsoft Office Applications like Word, Excel etc.

Windows SharePoint Services V2 vs V3

Before we jumpstart into architecture let me tell you few of the interesting design changes. In v2 WSS used to have a ISAPI filter which sits in IIS and which sees the Inclusion and Exclusion list to figure out if it is a ASP.NET Page or WSS requestas shown in Figure 3. Once that is done WSS Extension used an unmanaged code to interact with ASP.NET.So the integration was tighter with IIS rather than ASP.NET and they had problems with WSS code getting ran before ASP.NET context gets initialized. Another problem was that WSS had implemented its own ASP.NET parser for handling the ASP.NET pages residing in databases as there was not mechanism to do so in ASP.NET 1.1. This particular WSS ASP.NET parser lacked lot many features supported by classic ASP.NET parser. There was always non-sync with parser features with this implementation.



In order to avoid all the issues and conflicts they redesigned WSS on using ASP.NET. Now WSS it completely written using ASP.NET and now they have HTTPModules and HTTPHandlers to serve the request. ASP.NET 2.0 provided a mechanism to fetch pages from databases using HTTPModules which is called Virtual Path Provider; clients can extend the Virtual Path Provider to implement their own ways to fetch ASP.NET pages. WSS has one called SPVirtualPathProvider within SPRequestModule which does the processing for database persistent ASP.NET pages.

Windows SharePoint Services Architecture

From the technology stack diagram we know the features provided by WSS. Figure 4 shows the internal components and a sample flow of request.Wildcard mapping (*) enables the request to be routed to ASP.NET extension; another way to look at WSS is serving all types of resource like *.doc, *.ppt, *.aspx.




Within ASP.NET runtime they create a worker request for the same incoming request and we have a application factory which manages all the applications like SharePoint application (SPHttpApplication). The entire application specific configuration like external DB connector details all resides in web.config which get reread on modification enabling us to do only the fly changes without affecting older application instances. Older application instances will be still alive if it is in use but destroyed once it has processed all the assigned requests. With ASP.NET we can have multiple modules as show like Auth Modules, Session Modules, SPRequestModule etc. SPRequestModule is one which fetches the URL residing in databases.We can see various HttpHandlers like SPHttpHandler, ASP.NET Page handler, ASP.NET Web service handler etc. SPHttpHandler is the HttpHandler responsible for WSS related handling.



WSS is a kind of operating system for a web – a network built on the HTTP and HTML protocols. We have Administrative Object Models to mine the data within WSS in an organized object model hierarchy.Given below is the general WSS architecture that we can think off.
From the diagram itself it is evident that WSS is an user-friendly system of content creation, storage and management. WSS has various databases associated with it like
  • Configuration DB
    • Metadata
    • Configuration settings
    • Information about every single IIS Web site that has been SharePoint-extended in the Web farm
    • A Web farm can have only single configuration DB and multiple content DB
  • Content DB
    • Actual site content
    • User Information
  • Admin DB
    • Central Administrator tool


References
With this I guess I have provided some basic overview about what WSS is.

Upcoming articles which I plan are based on:
  • MOSS Architecture
  • User profile management in WSS and MOSS
  • Overview of Administrative Object Model

Labels: , ,

Posted by - at 7:20 pm | 2 comments read on

Archives

Links