Author Archives: rprasad

About rprasad

Notes while working in a large academic organization.

using the Django admin to copy an object

Posted on by
Reply

I just ran across this snippet from a hasty project last year.  For a course database, an administrator needed an easy way to copy semester details via the admin.

Ideally, the user would:

(1) Check selected semester objects
(2) Choose “copy semester” from the dropdown menu at the top

These options are displayed in the image below.

The basics of admin actions are well described in the Django documentation.

For the screen shot above, here is the code used to copy the object, including ForeignKey and ManyToMany relationships.  (Notes about the code follow.)

Location: admin.py

from django.contrib import admin
from course_tracker.course.models import SemesterDetails
import copy  # (1) use python copy

def copy_semester(modeladmin, request, queryset):
    # sd is an instance of SemesterDetails
    for sd in queryset:
        sd_copy = copy.copy(sd) # (2) django copy object
        sd_copy.id = None   # (3) set 'id' to None to create new object
        sd_copy.save()    # initial save

        # (4) copy M2M relationship: instructors
        for instructor in sd.instructors.all():
            sd_copy.instructors.add(instructor)

        # (5) copy M2M relationship: requirements_met
        for req in sd.requirements_met.all():
            sd_copy.requirements_met.add(req)

        # zero out enrollment numbers.  
        # (6) Use __dict__ to access "regular" attributes (not FK or M2M)
        for attr_name in ['enrollments_entered', 'undergrads_enrolled', 'grads_enrolled', 'employees_enrolled', 'cross_registered', 'withdrawals']:
            sd_copy.__dict__.update({ attr_name : 0})
 
       sd_copy.save()  # (7) save the copy to the database for M2M relations

    copy_semester.short_description = "Make a Copy of Semester Details"

Below are several notes regarding the code:

(1) Import python’s ‘copy’ to make a shallow copy of the object

(2) Make the copy.  Note this will copy “regular” attributes: CharField, IntegerField, etc.  In addition, it will copy ForeignKey attributes.

**(3) Set the object id to None.  This is important.  When the object is saved, a new row (or rows) will be added to the database.

(4), (5) For ManyToMany fields, the data must be added separately.

(6) For the new semester details object, the enrollment fields are set to zero.

To hook this up to the admin, it looks something like this:

Note “copy_semester” is added to the “actions” list.

class SemesterDetailsAdmin(admin.ModelAdmin):
    actions = [copy_semester]   #  HERE IT IS!
    save_on_top = True    
    inlines = (SemesterInstructorQScoreAdminInline, SemesterInstructorCreditAdminInline, CourseDevelopmentCreditAdminInline )
    readonly_fields = ['course_link','instructors_list', 'course_title', 'instructor_history', 'budget_history', 'enrollment_history', 'q_score_history', 'created', 'last_update']
    list_display = ( 'course', 'year', 'term','time_sort','instructors_list', 'last_update','meeting_date', 'meeting_time', 'room', 'number_of_sections',  'last_update')
    list_filter = (  'year', 'term', 'meeting_type', 'course__department__name', 'instructors' )
    search_fields = ('course__title', 'course__course_id', 'instructors__last_name', 'instructors__first_name')
    filter_horizontal = ('instructors', 'teaching_assistants', 'requirements_met', )
admin.site.register(SemesterDetails, SemesterDetailsAdmin)

And that’s it!

 

populate a newly made subclass

Posted on by
1

This is a bit of a hack but I created a subclass* named FacultyMember and wanted to back-link it with existing records in the Person class: (* This subclass falls under Multi-table inheritance.)

class Person(models.Model):
    lname = models.CharField('Last Name', max_length=75)
    fname = models.CharField('First Name', max_length=75)
    affiliation = models.ForeignKey(PersonAffiliation)
    appointment =  models.ForeignKey(AppointmentType)
    # (20+  other fields)
"""
Newly made subclass.
"""
class FacultyMember(Person):
    research_statement = models.TextField()
    journal_articles = models.ManyToManyField(JournalArticle)
    # (approx 6 fields, several of which are FKs)

(Note, the original FacultyMember model was made 2 years before the Person model. The subclassing is a bit of clean-up.)

There are 581 Person objects, 43 of which are faculty members. It wasn’t obvious to me how to create FacultyMember objects from the existing Person objects. One hack was to do the following.

>>> # (1) retrieve a person who is a faculty member
>>> p = Person.objects.get(lname='Smith', fname='John')
>>> p
>>> <FacultyMember: Smith, John>
>>>
>>> # (2) copy the Person object's dictionary
>>> p_dict = p.__dict__.copy()
>>> p_dict
{'grad_program_id': None, 'title_id': 203L, '_state': , 'second_phone': u'',
 'visible': True, 'appointment_id': 22L, 'long_title': u'', 'id': 1607L,
'minitial': u'', 'primary_lab_id': 48L, 'slug': u'john-smith', 'affiliation_id': 11L,
 'room': u'Room 123', 'phone': u'123-456-7890', 'alt_search_term': u'', 'lname': u'Smith',
 'office_id': None, 'second_email': u'', 'fname': u'John', 'grad_year_id': None, 
 'building_id': 1L, 'email': u'jsmith@university.edu'}
>>>
>>> # (3) remove the '_state' key/value from the dictionary
>>> p_dict.pop('_state')
>>>
>>>
>>> # (4) use the dictionary to make/start creating a new FacultyMember
>>> f = FacultyMember(**p_dict)
>>>
>>> # (5) link the Person object to the new FacultyMember via the 'ptr' attribute
>>> f.ptr = p
>>> f.save()
>>> f
>>> <FacultyMember: Smith, John>

That’s it.  I made the test above into a short script and it worked.  Initially I tried to instantiate the FacultyMember using the “ptr” attribute or the associated id.  I may try that again…

 

python: string to a datetime object

Posted on by
Reply

The task of converting strings to date or date/time objects arises fairly often.  For example, a colleague just had a request for a script to parse a log file so that it may be stored in  a database.  The log file looks something like this:

MACHINE001,SOME_USER,9/24/2010 5:03:29 PM,Logged on
MACHINE001,SOME_USER,9/24/2010 5:450:43 PM,Logged off

To convert the date/time section (boldfaced above) to a usable python object, use the datetime module’s strptime function.  For example:

>>> from datetime import datetime
>>> dt_str = '9/24/2010 5:03:29 PM'
>>> dt_obj = datetime.strptime(dt_str, '%m/%d/%Y %I:%M:%S %p')
>>> dt_obj
datetime.datetime(2010, 9, 24, 17, 3, 29)

To go in reverse, use datetime’s strftime.  Following up on the example above:

>>> dt_obj.strftime('%A, %B %d, %Y at %H:%M hours')
'Friday, September 24, 2010 at 17:03 hours'

For full documentation, including definitions for the date directives (%Y, %m, %d, etc.) use the following link:

8.1.7. strftime() and strptime() Behavior

(scroll down for directive defintions.  e.g. “%A” is “Locale’s full weekday name.”, etc)

 

 

 

Evernote, actually saving time

Posted on by
Reply

I’m a bit slow with new technology.  At one point in 2002, I was one of the few people in NYC without a cell phone. (But I could use a pay phone at any time:)

Last week a co-worker announced he was leaving to work at Evernote.  I looked up the company, thinking, oh, this is just an old idea, a variation on Mirror Worlds.  But then I downloaded Evernote and this weekend experimented with using it for a (non-programming) side project.  It was fantastic!

The information on the walls pictured below (as well as hundreds of digital files) are making their way to Evernote.   Information included journal articles*, transcripts, photos, a handwritten diagram, and a note on the back of an ATM receipt**.

I was able to organize and synthesize large and small chunks of information in a non-linear way.  The software considerably cut down the time it was taking to find information–letting me focus on my project.

I’ll use Evernote a little longer and then I’ll probably start using it for work projects.  So far Evernote hasn’t gotten in the way and I’m consciously not becoming crazy and trying to catalog everything.

(More importantly: the faster those papers are taken off the wall, the less I’ll be in trouble at home:)

*I’m also using Mendeley for research articles–now if someone could combine Evernote and Mendeley tagging:)

** This year I’m trying to get by without making notes on the back of the ATM receipts.  It’s my default organization method: jot down a quick note, put it in pocket, put the pants (including note), through the washing machine…

django: debugging sql queries in a template

Posted on by
1

This link to a 3-step Django snippet created in 2007 still works for showing queries within a template:

http://djangosnippets.org/snippets/93/

It’s helpful for debugging pages.  For example, adding the template snippet from the link above will show something like this at the bottom of your pages:

 

 

 

 

 

 

For step 3 in the link above, “Use RequestContext when rendering the current template”, include the following in your view file:

from django.template import RequestContext
from django.shortcuts import render_to_response

def view_news_page_by_slug(page_slug):
    # build your page, etc
    lu = { 'title': 'hooray', 'story': 'blah, blah, blah' }
    return render_to_response('your_template.html', lu, context_instance=RequestContext(request))

The template code from the django snippet is fairly straightforward.

To see queries from the python shell, see the django documentation for “How can I see the raw SQL queries Django is running?“

django urls: list a project’s url names

Posted on by
Reply

Below is quick hack to pull out an array of all the url names in a project.

The script iterates through the “urlpatterns” defined in your settings file as the ROOT_URLCONF.

From the “python manage.py shell” prompt:

from django.conf import settings

URL_NAMES = []
def load_url_pattern_names(patterns):
    """Retrieve a list of urlpattern names"""
    global URL_NAMES
    for pat in patterns:
        if pat.__class__.__name__ == 'RegexURLResolver':            # load patterns from this RegexURLResolver
            load_url_pattern_names(pat.url_patterns)
        elif pat.__class__.__name__ == 'RegexURLPattern':           # load name from this RegexURLPattern
            if pat.name is not None and pat.name not in URL_NAMES:
                URL_NAMES.append( pat.name)
    return URL_NAMES

root_urlconf = __import__(settings.ROOT_URLCONF)        # access the root urls.py file
print load_url_pattern_names(root_urlconf.urls.urlpatterns)   # access the "urlpatterns" from the ROOT_URLCONF

There’s probably an easier way to this, but I needed the url names for an error checking script.

Django Admin: Resizing Form Fields (for TabularInline)

Posted on by
2

Recently, one “small”* project in particular required the resizing of fields for displaying Inline Models in the admin.

For example, one inline model, “ElectroporationConditionsRecord” needed to display 11 fields.


(click image to see it full sized)

Squeezing the 11 fields into a readable row meant updating the “size” attribute of each input box.  A “default sized” input box in the admin does not include the “size” attribute, as in ‘size=”7″‘ below:

<input type=”text” size=”7” value=”0″ name=”id_days_of_drug_selection” id=”id_days_of_drug_selection”>

One way to add the “size” attribute, or any other attribute, is by making a form and connecting it to the admin.  For this example, three files are involved in making this happen:

my_appp/models.py      # definition of model ElectroporationConditionsRecord
       /forms.py
       /admin.py

(1) models.py

Here is a snippet of the ElectroporationConditionsRecord model, from the models.py file:

class ElectroporationConditions(models.Model):

    example_target_model = models.ForeignKey(ExampleTargetModel)  # example FK
    construct_name = models.CharField(max_length=40)
    drug_selection = models.CharField(max_length=40)
    days_of_drug_selection= models.IntegerField(default=0)
(etc..)

(2) forms.py

The inline model for the ElectroporationConditionsRecord in the screenshot above uses the following form, located in the forms.py file:

from django import forms

class ElectroporationConditionsForm(forms.ModelForm):
    '''ElectroporationConditionsForm.  Used to size the text input boxes'''

    class Meta:
        widgets = { 'construct_name': forms.TextInput(attrs={'size': 20})
                    , 'drug_selection': forms.TextInput(attrs={'size': 20})
                    , 'days_of_drug_selection': forms.TextInput(attrs={'size': 7})
                    , 'drug_concentration': forms.TextInput(attrs={'size': 7}) 

                    , 'dna_quantity': forms.TextInput(attrs={'size': 7})
                    , 'dna_concentration': forms.TextInput(attrs={'size': 7})
                    , 'linearized_by': forms.TextInput(attrs={'size': 7}) 

                    , 'passage': forms.TextInput(attrs={'size': 7})
                    , 'peak_voltage': forms.TextInput(attrs={'size': 7})
                    , 'time_constant': forms.TextInput(attrs={'size': 7})
                 }
# examples of other form widgets: PasswordInput, HiddenInput, Select, DateInput, etc.

Note, each key in the widgets dictionary above (e.g. ‘construct_name‘, drug_selection‘, etc.) is the name of a field in the model ElectroporationConditionsRecord.

You could also substitute the “size” attribute for a css class, as in:

              , 'time_constant': forms.TextInput(attrs={'class': 'input_sm_number'})

where the css might be something like:

             input.input_sm_number { width:6opx; }

(3) admin.py

To connect the new ElectroporationConditionsForm to the model ElectroporationConditionsRecord, a change is made in the admin.py:

# For this TabularInline Admin model, use the form that sets the size attributes
#
class ElectroporationConditionsAdminInline(admin.TabularInline):
    model = ElectroporationConditionsRecord   # from models.py
    form = ElectroporationConditionsForm      # from forms.py, sets the size attributes for the input boxes
    extra=0

# example of using the TabularInline Admin model defined above
#
class ExampleTargetModel(admin.ModelAdmin):
    # note: the ElectroporationConditionsRecord has an FK to this model, ExampleTargetModel
    inlines = (  ElectroporationConditionsAdminInline,)

Basically, that’s it.

In summary, to define the field width  in the admin model:

(a) Make a form similar to the ElectroporationConditionsForm, as in step (2) above.

(b) In your admin.py file, connect the form (ElectroporationConditionsForm) to your model, similar to the ElectroporationConditionsAdminInline model shown in step (3) above.

There are also other ways to do this, as seen on StackOverflow.com.

* The project is still small/low maintenance meaning it’s just models and the Django admin. It has 39 database tables, not including the “built-in” django tables.

Highcharts in the Django Admin

Posted on by
2

(Note, this assumes that you have a Django project installed and running. This example is for Django 1.3, the official version.)

Below are some notes for adding Highcharts to a Django admin page. In this case, I want to chart the enrollment in a science class over time.

Specifically, I have a model name “Course” and I want to see the chart on the “Course” change form in the admin.

(1) Download Highcharts, unzip it, and place in a directory under your Django MEDIA_ROOT directory, as specified in your Django settings.py.

(2) Make a blank template file named “change_form.html” file as well as the appropriate directories for your model. In the case of my Course model, the location is:

../templates/admin/course/course/change_form.html

Notes on the path above:

“templates” – a directory specified in your django settings.py file

“admin” – indicates a template for the django admin

” course” – name of my app. (The Course model is defined in ‘../course/models.py’)

“course” – name of the model, Course, in lowercase

“change_form.html” – name of an existing file in the django source code (/django/contrib/admin/templates/admin/change_form.html)

(2) Add the following lines of code to your change_form.html file.

{% extends "admin/change_form.html" %}
{% block extrahead %}{{ block.super }}
{% url 'admin:jsi18n' as jsi18nurl %}
<script type="text/javascript" src="{{ jsi18nurl|default:"../../../jsi18n/" }}"></script>
{{ media }}
<!-- start: two new lines for highcharts -->
<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.6.1/jquery.min.js" type="text/javascript"></script>
<script src="{{ MEDIA_URL }}highcharts/js/highcharts.js" type="text/javascript"></script>
<!-- end: two new lines for highcharts -->
{% endblock %}

Most of the code above is taken from the “extrahead” block in the original “change_form.html” file. Two new lines of javascript have been added, as indicated by the comments above.  (Again, this is Django 1.3, the official version. If you’re using a different version, copy the appropriate code from the change_form.html template included with django.)

The 1st line calls in the JQuery library and the 2nd line connects to highcharts.js.

(3) Test. Go to the the Django admin and attempt to add a new instance of your model, or edit an existing one. View the source in your browser:

(a) Make sure that the new lines show up and that

(b) the javascript files are being loaded. (e.g., Click on the links to the jquery and highcharts js files to make sure they load.)

Note: Django 1.3 has JQuery built-in but it has it’s own namespace. Using the built-in JQuery requires editing highcharts.js and changing “JQuery” to “django.Jquery”.  Because of maintenance, changing the highcharts.js is not advisable.

(4) For my Course model, I’m adding a function in the models.py file named “enrollment_chart”.  For example purposes, I’m throwing test data into the function.  (In reality, the function calls a queryset that is passed to the template.)

    def enrollment_chart(self):
        lu = { 'categories' : [ 'Fall 2008', 'Spring 2009','Fall 2009', 'Spring 2010', 'Fall 2010', 'Spring 2011'],\
             'undergrad' : [18, 22, 30, 34, 40, 47],\
             'grad' : [1, 2, 4, 4, 5, 7],\
             'employee' : [2, 3, 0, 1, 1, 2] }
        lu['total_enrolled'] = [sum(a) for a in zip(lu['undergrad'], lu['grad'],lu['employee'])]

        return render_to_string('admin/course/course/enrollment_chart.html', lu )
    enrollment_chart.allow_tags = True

(5) Make the “enrollment_chart.html” file referred to in the function above.  Note, the file goes in the same directory as the “change_form.html” file in step (2).

(6) In the “enrollment_chart.html” file, add the following code:

Note 1:  This code is from the highcharts example graph with some changes to the labels and the addition of Django template tags.

Note 2: the “renderTo” attribute must match the “id” of the <div ..> at the bottom of this code.

<script type="text/javascript">
var chart;
jQuery(document).ready(function() {
	chart = new Highcharts.Chart({
		chart: {
			renderTo: 'enrollment_container',
			defaultSeriesType: 'line',
			marginRight: 130,
			marginBottom: 30
		},
		title: {
			text: 'Enrollment by Semester ',
			x: -20 //center
		},
		subtitle: {
			text: '',
			x: -20
		},
		xAxis: {
			categories: [ '{{ categories|join:"','" }}']
		//	categories: ['Fall 2008', 'Spring 2009','Fall 2009', 'Spring 2010', 'Fall 2010', 'Spring 2011']
		},
		yAxis: {
			title: {
				text: 'Number of Students'
			},
			plotLines: [{
				value: 0,
				width: 1,
				color: '#808080'
			}]
		},
		tooltip: {
			formatter: function() {
	                return ''+ this.series.name +''+this.x +': '+ this.y;
			}
		},
		legend: {
			layout: 'vertical',
			align: 'right',
			verticalAlign: 'top',
			x: -10,
			y: 100,
			borderWidth: 0
		},
		series: [{
			name: 'Total Enrolled',
			//data: [1, 27, 34, 39, 46, 56]
			data: [{{ total_enrolled|join:"," }}]
		}, {
			name: 'Undergrads',
            data: [{{ undergrad|join:"," }}]
			//data: [18, 22, 30, 34, 40, 47]
		}, {
			name: 'Grads',
			data: [{{ grad|join:"," }}]
			//data: [1, 2, 4, 4, 5, 7]
		}, {
			name: 'Employees',
			data: [{{ employee|join:"," }}]
			//data: [2, 3, 0, 1, 1, 2]
		}]
	});

});
</script>
<div id="enrollment_container" style="width: 700px;height: 500px"></div>

(7) Update your admin.py file to connect the enrollment_chart() function in step (4) to your admin screen.

In the admin.py file, add enrollment_chart to:

(a) the list of readonly_fields, as well as

(b) a field in your fieldsets array

(Location of the admin.py file for the Course model: ../course/admin.py)

Below is an example where enrollment_chart has been added:

class CourseAdmin(admin.ModelAdmin):
    list_display = ( 'course_id', 'title', 'catalog_number', )
    search_fields = ( 'course_id', 'title', 'catalog_number', )
    list_filter = ('status',)

    readonly_fields = ('semester_details', 'enrollment_chart' )
    fieldsets = [
     ('Course', { 'fields':  [  'course_id', 'title', 'catalog_number', \
                    'department', 'course_type', 'status', ]}), \

    ('Semester Details', { 'fields':  [  'semester_details',  ]}),\
    ('Enrollment', { 'fields':  [  'enrollment_chart',  ]}),\
                    ]
admin.site.register(Course, CourseAdmin)

Summary

The explanation above is long, but, if you’re django instance is already running, the steps are fairly quick.

(a) Download Highcharts, unzip it, and place in a directory under your Django MEDIA_ROOT directory (step 1)

(b) Add two admin templates:

change_form.html  – call the JQuery and the highcharts.js libraries  (steps 2 and 3)

enrollment_chart.html – or another template to put the chart’s javascript + containing <div..> (steps 5 and 6)

(c) Create a function accessible by your model that passes chart data  (step 4)

(d) Modify the admin.py to link the function to the django admin and a template (step 7)

Once the basics are going, you can start customizing your chart:)