Course CS/SE 6332

Professor: Dr. Wei Yang
Term: Fall 2018
Meetings: ECSS 2.306 Friday 10:00AM-12:45PM

Professor’s Contact Information:
Office Location: ECSS 2.701
Email Address: [email protected]
(Please type the prefix [CS 6332] on the subject of the email)
Office Hours: TBD

General Course Information

Pre-requisites, Co-requisites, & other restrictions
Suggested pre- or co-requisite: CS 3340; CS 3376; CS 4348

Course Description

CS 6332 is a graduate level, research oriented, systems and software security class. The goal of this course is to understand the low-level details of the real software implementations by using program analysis techniques; examine the state of the art attacks, such as memory exploits (e.g., ROP); design practical systems defense (e.g., automatic vulnerability signature generation); design program analysis to reverse engineer the bytecode/binary code.

Learning Outcomes

In general, through this course, students should be able to parse undocumented bytecode/binary code, quickly deciphering the code’s function and purpose. Specifically, students shall be able to understand and know:

Automated program analysis for the reverse engineering of program bytecode/binary code.

Static binary code analysis. Dynamic Binary code instrumentation. Data flow analysis, pointer analysis, and control flow analysis. Program slicing.

Vulnerability Discovery, Memory Exploits, and system defense.

Understand the common software vulnerabilities such as buffer overflow, format string, integer overflows. Understand how to develop exploits against each vulnerability, and understand how to bypass the state-of-the-art defense

Required Texts & Materials

[CHR] Ed Skoudis; Tom Liston. “Counter Hack Reloaded: A Step-by-Step Guide to Computer Attacks and Effective Defenses”, Second Edition
[AOE] Erickson, Jon. “Hacking: The Art of Exploitation”, 2nd Edition
[SRE] Eldad Eilam. “Reversing: Secrets of Reverse Engineering”

Suggested Texts, Readings, & Materials

[CSAPP] Randal E. Bryant and David R. O'Hallaron. “Computer Systems: A Programmer's Perspective, 2/E”
[TSH] Kozoil, Jack. “The Shellcoder's Handbook: Discovering and Exploiting Security Holes”
[PPA] Principles of Program Analysis. Springer.

Assignments & Academic Calendar

Week 1Course Overview
Week 2Understanding the Program Representations
Week 3Static Analysis Tool Introduction
Week 4Projects, Empirical Studies and Challenge Programs
Week 5Principles of Program Analysis
Week 6Applications of Program Analysis
Week 7Symbolic Execution
Week 8Automatic Exploits Generation
Week 9Buffer Overflow, Integer and Heap Overflow
Week 10Return Oriented Programming and HeapSpray
Week 11Fighting for Malware: Unpacking, Disassembling, Decompilation
Week 12Mobile Malware Analysis
Week 13Obfuscation and Cloaking
Week 14Trending Security Topics (1)
Week 15Trending Security Topics (2)
Exam Date(s), Time(s): TBD

Course Policies

Grading (credit) Criteria

Projects: 60%
Participation: 10%
Exam: 30%

Grading Curve

under 60F

Make-up Exams

Make-up exams will be granted only for exceptional conditions, as approved by the instructor.

Extra Credit

Students can earn up to 30% credits by conducting empirical studies and programming challenges.

Late Work

30% reduction in grade for late project submissions up to 24 hours. Assignments will not be accepted past due 24 hours unless there are extenuating circumstances.

Class Attendance

You are expected to attend class. By CS Dept policy, missing three (3) consecutive classes results in a letter grade drop and missing four (4) consecutive classes is an automatic failure for the class.

Classroom Citizenship

Cell phones shall not be used in the classroom during sessions. Place them on mute. If you receive a call, leave the room.

Student Conduct and Discipline

The University of Texas System and The University of Texas at Dallas have rules and regulations for the orderly and efficient conduct of their business. It is the responsibility of each student and each student organization to be knowledgeable about the rules and regulations which govern student conduct and activities. General information on student conduct and discipline is contained in the UTD publication, A to Z Guide, which is provided to all registered students each academic year.

The University of Texas at Dallas administers student discipline within the procedures of recognized and established due process. Procedures are defined and described in the Rules and Regulations, Board of Regents, The University of Texas System, Part 1, Chapter VI, Section 3, and in Title V, Rules on Student Services and Activities of the university’s Handbook of Operating Procedures. Copies of these rules and regulations are available to students in the Office of the Dean of Students, where staff members are available to assist students in interpreting the rules and regulations (SU 1.602, 972/883-6391).

A student at the university neither loses the rights nor escapes the responsibilities of citizenship. He or she is expected to obey federal, state, and local laws as well as the Regents’ Rules, university regulations, and administrative rules. Students are subject to discipline for violating the standards of conduct whether such conduct takes place on or off campus, or whether civil or criminal penalties are also imposed for such conduct.

Academic Integrity

The faculty expects from its students a high level of responsibility and academic honesty. Because the value of an academic degree depends upon the absolute integrity of the work done by the student for that degree, it is imperative that a student demonstrate a high standard of individual honor in his or her scholastic work.

Scholastic dishonesty includes, but is not limited to, statements, acts or omissions related to applications for enrollment or the award of a degree, and/or the submission as one’s own work or material that is not one’s own. As a general rule, scholastic dishonesty involves one of the following acts: cheating, plagiarism, collusion and/or falsifying academic records. Students suspected of academic dishonesty are subject to disciplinary proceedings.

Plagiarism, especially from the web, from portions of papers for other classes, and from any other source is unacceptable and will be dealt with under the university’s policy on plagiarism (see general catalog for details). This course will use the resources of, which searches the web for possible plagiarism and is over 90% effective.

Email Use

The University of Texas at Dallas recognizes the value and efficiency of communication between faculty/staff and students through electronic mail. At the same time, email raises some issues concerning security and the identity of each individual in an email exchange. The university encourages all official student email correspondence be sent only to a student’s U.T. Dallas email address and that faculty and staff consider email from students official only if it originates from a UTD student account. This allows the university to maintain a high degree of confidence in the identity of all individual corresponding and the security of the transmitted information. UTD furnishes each student with a free email account that is to be used in all communication with university personnel. The Department of Information Resources at U.T. Dallas provides a method for students to have their U.T. Dallas mail forwarded to other accounts.

Withdrawal from Class

The administration of this institution has set deadlines for withdrawal of any college-level courses. These dates and times are published in that semester's course catalog. Administration procedures must be followed. It is the student's responsibility to handle withdrawal requirements from any class. In other words, I cannot drop or withdraw any student. You must do the proper paperwork to ensure that you will not receive a final grade of "F" in a course if you choose not to attend the class once you are enrolled.

Student Grievance Procedures

Procedures for student grievances are found in Title V, Rules on Student Services and Activities, of the university’s Handbook of Operating Procedures.

In attempting to resolve any student grievance regarding grades, evaluations, or other fulfillments of academic responsibility, it is the obligation of the student first to make a serious effort to resolve the matter with the instructor, supervisor, administrator, or committee with whom the grievance originates (hereafter called “the respondent”). Individual faculty members retain primary responsibility for assigning grades and evaluations. If the matter cannot be resolved at that level, the grievance must be submitted in writing to the respondent with a copy of the respondent’s School Dean. If the matter is not resolved by the written response provided by the respondent, the student may submit a written appeal to the School Dean. If the grievance is not resolved by the School Dean’s decision, the student may make a written appeal to the Dean of Graduate or Undergraduate Education, and the deal will appoint and convene an Academic Appeals Panel. The decision of the Academic Appeals Panel is final. The results of the academic appeals process will be distributed to all involved parties.

Copies of these rules and regulations are available to students in the Office of the Dean of Students, where staff members are available to assist students in interpreting the rules and regulations.

Incomplete Grades

As per university policy, incomplete grades will be granted only for work unavoidably missed at the semester’s end and only if 70% of the course work has been completed. An incomplete grade must be resolved within eight (8) weeks from the first day of the subsequent long semester. If the required work to complete the course and to remove the incomplete grade is not submitted by the specified deadline, the incomplete grade is changed automatically to a grade of F.

Disability Services

The goal of Disability Services is to provide students with disabilities educational opportunities equal to those of their non-disabled peers. Disability Services is located in room 1.610 in the Student Union. Office hours are Monday and Thursday, 8:30 a.m. to 6:30 p.m.; Tuesday and Wednesday, 8:30 a.m. to 7:30 p.m.; and Friday, 8:30 a.m. to 5:30 p.m.

The contact information for the Office of Disability Services is:

The University of Texas at Dallas, SU 22
PO Box 830688
Richardson, Texas 75083-0688
(972) 883-2098 (voice or TTY)

Essentially, the law requires that colleges and universities make those reasonable adjustments necessary to eliminate discrimination on the basis of disability. For example, it may be necessary to remove classroom prohibitions against tape recorders or animals (in the case of dog guides) for students who are blind. Occasionally an assignment requirement may be substituted (for example, a research paper versus an oral presentation for a student who is hearing impaired). Classes enrolled students with mobility impairments may have to be rescheduled in accessible facilities. The college or university may need to provide special services such as registration, note-taking, or mobility assistance.

It is the student’s responsibility to notify his or her professors of the need for such an accommodation. Disability Services provides students with letters to present to faculty members to verify that the student has a disability and needs accommodations. Individuals requiring special accommodation should contact the professor after class or during office hours.

Religious Holy Days

The University of Texas at Dallas will excuse a student from class or other required activities for the travel to and observance of a religious holy day for a religion whose places of worship are exempt from property tax under Section 11.20, Tax Code, Texas Code Annotated.

The student is encouraged to notify the instructor or activity sponsor as soon as possible regarding the absence, preferably in advance of the assignment. The student, so excused, will be allowed to take the exam or complete the assignment within a reasonable time after the absence: a period equal to the length of the absence, up to a maximum of one week. A student who notifies the instructor and completes any missed exam or assignment may not be penalized for the absence. A student who fails to complete the exam or assignment within the prescribed period may receive a failing grade for that exam or assignment.

If a student or an instructor disagrees about the nature of the absence [i.e., for the purpose of observing a religious holy day] or if there is similar disagreement about whether the student has been given a reasonable time to complete any missed assignments or examinations, either the student or the instructor may request a ruling from the chief executive officer of the institution, or his or her designee. The chief executive officer or designee must take into account the legislative intent of TEC 51.911(b), and the student and instructor will abide by the decision of the chief executive officer or designee.

Off-Campus Instruction and Course Activities

Off-campus, out-of-state, and foreign instruction and activities are subject to state law and University policies and procedures regarding travel and risk-related activities. Information regarding these rules and regulations may be found at Additional information is available from the office of the school dean.

These descriptions and timelines are subject to change at the discretion of the Professor.