This Formative Evaluation Report covers the second grant funding year for Project Venture (June 1, 1999 to June 8, 2000). Sections within the report provide a Project Description, the Evaluation Approach, Findings and Recommendations, the results from Evaluation Instruments, and an Appendix.

Project Venture’s second project year was one of organization and growth as staff was hired and trained, procedures were adopted, and training and mentoring of teachers began in earnest. Also, during this project year, a number of initiatives related to technology integration were implemented at the state and national level. For example, the state of Arizona worked toward revising technology standards for students and the School Facilities Board provided funds to lower the student to computer ratio and to train teachers to use technology.

Project Venture is a Technology Innovation Challenge Grant program that provides comprehensive teacher and staff development in the project’s consortium of five diverse consortium members consisting of seventeen school districts. Three overall objectives guide the work of Project Venture:

1. To increase the number of teachers trained to use technology for teaching and learning by 25% annually,
2. To develop and implement a curriculum that is in alignment with State Content Standards and includes the integration of technology, and
3. To develop and implement an ongoing evaluation protocol that assists with project refinement, implementation and ensures sustainability and replication by the end of the project.

Project Venture has an opportunity to identify those aspects of a training model that are effective in the types and sizes of schools that compose the Project Venture consortium and to build a replicable model for others to utilize. The five local education agencies (LEAs) represent K-12 teachers and students in rural, suburban, and urban settings; socio-economic ranges from below poverty to affluent; one room schools to large suburban enrollments; and pedagogical and philosophical differences.

The project’s individualized training model impacts consortium classrooms by providing assistance to help teachers learn and practice the skills necessary to integrate technology into classroom learning activities. Technology Mentor Teachers (TMTs) provide both technology training classes and individual classroom mentoring and just-in-time training with Project Venture teachers in the classroom to improve technology integration and to develop lessons and units that meet Arizona State Content and Technology Standards.

Project Venture staff and consortium members have accomplished many foundational tasks during this project year despite changes in personnel at both the project level, within the consortium, and on the external evaluation team. The project has continued to refine and solidify its goals and objectives as they reflect upon the success of their activities. The Project Description section of this report describes the many and varied activities undertaken by the project that have led to its current successful position.

A Logic Map has been created for the project that defines the project inputs and the intermediate goals and terminal goals or outcomes. The Formative Evaluation Approach and Data Source Matrix, which are presented in following sections of this report, served as a guide to the measurement and evaluation of the intermediate goals. The project Plan of Operation is also included in this report and shows the alignment of the Activities to the Formative Evaluation Approach.

A series of evaluation instruments were utilized during the project year that serve to inform Project Venture about each of the components of the project. The training model being refined by the project appears to be a very successful model for providing timely and appropriate training to classroom teachers about the integration of technology into their classrooms. The Technology Mentor Teacher role will likely develop into the aspect of the project that determines the final success of the project. This role has been successfully implemented in each of the diverse consortium school districts.

Teacher reflections and classroom observations point to greater student access to technology and indicate greater student motivation to be involved with their learning environments. In years three, four and five, Stanford 9 testing data will be used to compare student achievement between Project Venture and non Project Venture classrooms as appropriate.

Overall, Project Venture is well placed to continue its progress toward successful completion of its grant requirements and sustainability of the project into the future. Year Three of the project looks to the exciting task of working closely Arizona School Services Through Educational Technology (ASSET) (http://www.asset.asu.edu/) to lead in the development of lesson plans and units that support the new Arizona State Technology Standards. Other local, state, and national initiatives for technology integration in schools are in place to support and enhance Project Venture as well. Entering the third year of Project Venture, consortium members feel fortunate to have the resources necessary to assist with the integration of technology for teaching and learning.

Table of Contents *

Table of Tables *

Table of Figures *

Project Description *

Logic Map *

Project Venture Plan of Operation *

Formative Evaluation Approach *

Findings and Recommendations *

Evaluation Procedures and Instruments *

Appendix *

Figure 1: Project Venture Logic Map *

Project Venture is a Technology Innovation Challenge Grant program that provides comprehensive teacher and staff development in the project’s consortium of five diverse LEAs. An individualized training model to assist teachers with the integration of technology into classroom learning activities is a primary goal of the project. Technology Mentor Teachers (TMTs) in each district provide technology classes and work individually with teachers in the classroom to enhance technology integration.

The second project year was one of organization and growth as Project Venture TMTs were hired and trained, procedures were adopted and implemented, and training and mentoring of teachers began in earnest. During this project year, a number of initiatives related to technology integration were implemented at the state and national level. For example, the state of Arizona worked toward revising technology standards for students. The School Facilities Board Student First project provided funds to lower the student to computer ratio and to train teachers to use technology. Furthermore, E-Rate monies have allowed school districts to continue to connect classrooms and the International Society for Technology in Education (ISTE) has released updated technology standards for teachers. Entering the third year of Project Venture, consortium members feel fortunate to have the resources necessary to assist with the integration of technology for teaching and learning.

Training Model

Project Venture is refining a training model, originally piloted by Creighton Elementary School District, which provides five levels of technology skill and classroom technology integration training based on the assessment of a teacher’s prior knowledge and skill. Level I teachers are beginning to use technology and are learning basic computer survival skills, Level II teachers begin to take advantage of the full use of technology and make use of all the tools technology has to offer, Level III teachers possess the skills necessary to begin integrating technology into curricular areas through project based learning, Level IV teachers are promoters of technology integration, they are masters with computers, curriculum development, and management issues related to the use of technology with students. Level V teachers will result from the project; these teachers are trainers of Level I through Level IV teachers — they are masters of technology, curriculum, assessment, classroom management, and have the ability to guide others towards seamless integration. Level V is not addressed in the training objectives at this point in time, and therefore, will not be discussed in this report. The Project Venture Advisory group will work to develop the outcomes that will determine Level 5 readiness during Year Three.

Training objectives have been developed for four levels of the five levels and are consistent throughout consortium member districts. The training objectives may be found in the Appendix and on the Project web site (http://www.creighton.k12.az.us/projectventure). Representative skills at each of the four levels within each learning category (Desktop Basics and Troubleshooting, Word Processing and Desktop Publishing, Internet, Spreadsheets, Databases, Instructional Software, and Multimedia) are documented and form the basis for the technology training that is provided by Project Venture. Technology integration is introduced in Level II and is a major part of all Level III and Level IV training.

The Technology Mentor Teachers in each district select, develop, and provide training that corresponds to the four training levels as formal training courses, individual training or coaching on an as-needed basis, model teaching, team-teaching, or mentoring. In addition, teachers who have been selected as Level III and IV participants in the grant are provided with in-class mini-labs of one computer per five students (average of 5 computers per classroom), a printer, and a presentation device. Internet connectivity and appropriate instructional and management software are also included. Having technology readily available in a classroom allows teachers to integrate technology use into the curriculum in a seamless manner.

Project Venture Consortium

The 17 districts that comprise Project Venture represent almost all educational and demographic possibilities. The local education agencies represent the following variables:

• K-12 teachers and students in rural, suburban, and urban settings,
• Socio-economic ranges from below poverty to affluent,
• One room schools to large suburban enrollments
• Pedagogical and philosophical differences.

Project Venture has an opportunity to identify aspects of the training model that are effective in all types and sizes of schools in this diverse consortium. Table 1 describes the consortium members.

Table 1 Project Venture Consortium Member School Districts

This table illustrates that the districts are in different stages of development. While some were ready to work with teachers in classrooms, the Maricopa County Small School District Consortium needed to build the technological infrastructure in order to support Level III Project Venture Teacher classrooms.

Project Administration

The Project Venture grant director is employed; with grant funds, by Creighton Elementary School District, the fiscal agent for the project. The director also supervises, in collaboration with building principals, the six TMTs in that district. The Technical support/web developer position for the grant is also housed at Creighton, but supports each consortium member as needed. Each consortium member district has appointed an administrator to participate in Project Venture and oversee its administration in their respective districts. This Advisory team holds regularly scheduled meetings to develop policies and procedures, define and revise evaluation instruments, assure consistency in the implementation of the project, and work with the External Evaluator to ensure appropriate timely evaluation of the project as well as to assess progress toward the project goals. The Advisory group will ensure the success of the project through ongoing assessment and the implementation of identified revisions and refinements to project activities.

Project Activities

The Advisory Team has worked during the last project year on many project activities. The results of these activities show an organization that is working together and accomplishing much. These activities fall into five general categories that will be reported below: the training model, evaluation instruments, the TICG Database for project reporting, professional development, and consortium web site.

Training Objectives

During the summer of 1999 the consortium Advisory Team and the TMTs became familiar with the Project Venture training model and developed technology training areas with learning objectives for training levels I through IV. This intensive activity resulted in a detailed listing of skills that each teacher should master within each training level in the following categories: Desktop Basics and Troubleshooting, Word Processing and Desktop Publishing, Internet, Spreadsheets, Databases, Instructional Software, and Multimedia. These objectives focus on both application skills and the integration of applications in teaching Arizona content and Technology Standards. This document is in the Appendix and may also be found at the Project Venture web site http://www.creighton.k12.az.us/projectventure. The objectives are used by the TMTs to select and develop technology training courses and opportunities for their districts.

These training objectives in turn have helped to define the evaluation instruments being used for Project Venture. They are directly connected to the Technology Self-Evaluation instrument, the Technology Training Course Evaluation instrument, the TMT Weekly Time Report, and the selection of Level III and IV Project Venture classroom teachers.

Project Venture Teacher Selection Criteria

Consortium members have collaborated to develop a selection process for Level III and IV classroom teachers that takes place in the spring of each year. Each district has modified the process to meet their own specific needs, but there are common components: an application that includes examples of current classroom technology use (teacher and/or student), an integrated lesson or unit, a technology skill self-assessment, a principal recommendation, and a commitment letter or agreement. The agreement or commitment letter for each district also contains common elements, with each Project Venture teacher agreeing to participate with the project on four levels as shown in Table 2.

Table 2 Project Venture Teacher Agreements.

Self Evaluation Report

The Advisory Team and the TMTs played a major role in defining the Technology Self Evaluation Report instrument, which assesses individual skills for all the training categories (see Training Objectives above) and can be used by TMTs to assist in determining skill levels and therefore, the training needs of individuals and schools. This instrument is directly matched to the training objectives. This instrument may be found in the Appendix and on-line at the EdCare lab Project Venture site at Arizona State University (http://research.ed.asu.edu/projectventure/ ). The self-assessment is available to all teachers, staff, administrators, parents and community members of the consortium districts. This instrument was planned as an annual assessment to place an individual in the appropriate level for training and mentoring purposes, track an individual’s growth in skill, and advise TMTs about the training needed at specific levels.

Course Evaluation

The Advisory Team and the TMTs also defined the Training Course Evaluation instrument in order to consistently evaluate the technology courses prepared and delivered by TMTs. This instrument is also web-based and is hosted on the EdCare server at Arizona State University. After evaluations are complete for a course the responses are stored in a master database and the TMT’s supervisor receives copy by email. This instrument is currently being revised to be reliably available online, to deliver email as expected, and to allow TMTs to see a summary of participant comments immediately after all evaluations have been submitted on-line. Additionally the database will allow printing of the results as courses are completed, and provide a copy to the trainer as well as to their supervisor.

TMT Time Report

This on-line report tracks TMT weekly activities by documenting the amount of time spent in specific tasks. This data provides information about daily activities, successes, challenges, and new learning. Weekly activities are summarized in categories that relate to the job duties of TMTs (Level I-IV classroom activities, technology training classes, individual training, advisement, TMT professional development, administrative, travel, other). This information proved to be useful in defining the effectiveness of the project by quickly pointing out successes and highlighting issues of concern. The resulting description of TMT duties and time commitments should be helpful to others desiring to replicate the project. The report may be found in the appendix and on-line at http://research.ed.asu.edu/projectventure/tmttime/TMT.html .

Classroom observations

The Advisory Team and the TMTs also assisted with refinement of the classroom observation instrument that was being piloted by the external evaluators. A pilot test of the instrument was held during March and April 2000. The instrument was used during the observation of Project Venture classrooms. See the Appendix for a copy of this form.

Lesson or Unit Publication Process

The consortium members met during the last few months of the project year to formulate a lesson or unit publication process. The Advisory Team is in the process of finalizing the publication process and, in the meantime, each district continues to submit lessons and units for inclusion in the project database and posting on the Project Venture web site. 100 lessons or units have been created, 75 of which are ready for publication in the Project Venture Database.

All consortium members contributed to the completion of the Project Report submitted via the TICG Database on June 8, 2000. This database was created expressly for all Technology Innovation Challenge Grantees and forms a comprehensive database about TICG grants that is available to all reporting agencies and other stakeholders. The database was not available for data entry until May 2000, and all information was required to be complete on June 8, 2000. The consortium members were able to provide the required data on very short notice at a difficult time in their school calendars. The TICG database will be available for future years and can be utilized as a common area for entering project data in an ongoing fashion throughout each project year.

Technology Mentor Teachers and the Advisory Team have participated in the following activities that have enhanced their leadership and mentoring skills:

• Savvy Cyber Teacher TrainingÒ — a twenty-hour condensed version of the Train-the-Trainer model that provides skills and techniques for integrating the Internet into curriculum. This training is a product of Stevens Institute of Technology, a consortium partner.
• Book Talks — two opportunities to read and discuss the following books: Growing Up Digital: The Rise of the Net Generation by Don Tapscott and Teaching with Technology: Creating Student Centered Classrooms by Judith Haymore Sandholtz, Cathy Rengstaff, and David Dwyer . These books focused on current issues in educational technology.
• Facilitative LeadershipÒ — a three day, twenty-four hour, training course focusing on development of leadership skills related to facilitating meetings, consensus building, and team development. This course is offered by Interaction Associates.
• Cognitive CoachingÓ — by Art Costa and Robert Garmstrom, a seven day training course offered by Judy Gottschalk,that develops skills for coaching and mentoring teachers. TMTs completed the first three of these days and are finishing the course in year three.

Evaluation Web Site

Cisco Systems, a grant partner, provides an evaluation web site for Project Venture located at and maintained by EdCare Labs at Arizona State University in Tempe, Arizona (http://research.ed.asu.edu/projectventure/ ). The actual evaluation instruments are coded by Cisco software engineers and updated as necessary. The Technology Self-Evaluation, Technology Training Course Evaluation, and TMT Time Report are currently available on-line. At the current time evaluation data is only available to the external evaluators as file transfers of the database. The evaluators are working with Cisco Systems to arrange time for an engineer to update the instruments and to provide more consistent access to the data for evaluators and Project Venture staff. Recent changes to the course evaluation and TMT time report make these instruments more user-friendly. In addition, TMTs will soon be provided with the ability to print summaries of course evaluations and their TMT time reports.

Project Web Site

Project Venture hosts a web site at Creighton Elementary School District that supports consortium members. The site also informs viewers of project activities, disseminates resources and project tools, and provides links to districts and partners (http://www.creighton.k12.az.us/projectventure). Screen prints of a few sample pages are included in the Appendix. The Project maintains listservs for consortium members to communicate activities and disseminate information on a regular basis.

Inputs

The initial components necessary to achieve intermediate and terminal goals for Project Venture success are listed as Inputs on the logic map (Figure 1). They are the Project Venture Consortium, Classroom Teachers, Technology Mentor Teachers (TMTs), and Equipment and Infrastructure.

Intermediate Goals

Inputs lead to various intermediate goals that result from the interactions of the TMTs and classroom teachers. The numbers in parentheses relate to the numbers of the Intermediate Goals on the Logic Map. TMTs receive professional development (1) that enhances their mentoring and technology skill so that they, in turn,, can mentor and train classroom teachers (3). These training (4) and mentoring activities (2) lead to a database of published lessons and units aligned with state content and technology standards (5) an increased number of teachers who are trained to use technology (6), and increased teacher technology integration skills (7). With the addition of equipment and infrastructure inputs, the intermediate goals of increased technology integration in the classroom (10) and into the lessons and units (8) lead to an intermediate goal of increased student access to integrated technology learning opportunities (9).

Each of the intermediate goals is measured by one or more evaluation instruments, which are identified in the Formative Evaluation Plan.

Terminal Goals

Terminal goals, including worldwide access to technology-integrated lessons and units (11), increased student learning (12) and teacher-to-teacher mentoring (13) are then possible.

Project Objectives

The Project Venture objectives are listed below in Table 3. The formative evaluation portion (Objective 3.1) is being reported in this document.

Table 3 Project Venture Objectives.

These objectives are reflected in the Logic Map above and in the published Plan of Operation (Figure 2) on the next few pages. This Plan of Operation, from the grant proposal, lists the activities that were planned to ensure project implementation.

Two additional columns have been added to the Plan of Operation as submitted in the grant proposal. These columns show progress to date and identify the evaluation instrument(s) that address the activity. The first column, Progress To Date, uses codes to show that the activity has been completed C), the activity has been implemented (I), the activity is ongoing (O), or that the activity has not yet been implemented (X).

The last column identifies the evaluation activity that provides information about the activity in this report. Activities that were completed in Year 1 of the grant are identified as well.

The Formative Evaluation Approach that follows the Plan of Operation is the plan that was used to connect the Logic Map and Project Venture Objectives and Activities with the evaluation instruments utilized for this report.

Plan of Operation

Objective 1.0 To increase the number of teachers trained to use technology for classroom instruction by 25% annually.

Objective 1.1 By the end of the project, thirty-eight percent (1,135) of the consortium’s teachers will be integrating technology into their curriculum at Levels III, IV and V.

Objective 1.2 By the end of the project, a minimum of 2,756 staff members will have achieved Level II skills and knowledge.

Objective 1.3 By the end of the project, all consortium staff members will have participated in Level I and II staff development activities.

Objective 1.4 By the end of the first project year, infrastructure, servers and networking software will be obtained and installed, with expansions and upgrades to occur annually.

Activities	Process/Technique	Person Responsible	Timeline	Outcome	Progress to Date	Evaluation Instrument
Hire Master Trainer and Technical Support Staff for Project	Develop qualifications list and job description	Creighton District	October 30, 1998	Contract signed	C	Year 1
Identify and place 14 TOTs (trainer of teachers) --5 @ Creighton; 7 @ Tri-District and 2 @ MCSSC	District to implement hiring procedures	Participating Districts	December 15, 1998	Contracts signed	C	Year 1
Identify certified classroom teacher replacements for TOTs	Districts to implement hiring procedures	Participating Districts TOTs	January 15, 1998	Positions filled	C	Year 1
Obtain and install wide area network infrastructure, servers and networking software	Compaq provides network servers Microsoft provides network operation software CISCO Systems provides web-based delivery framework	Technical Assistant Consultants Cisco Compaq Microsoft	Year One and ongoing	Technology infrastructure in place	I, O	Classroom Observations, Teacher Reflections, , TMT Time Reports, TMT Focus Group,

Activities	Process/Technique	Person Responsible	Timeline	Outcome	Progress to Date	Evaluation Instrument
Plan, develop and create Project Venture web pages	Web page development for: dissemination of training model instruction of Level I and II staff on-line self assessment administration and placement on-line integrated K-12 curriculum database	Technical Assistant Consultants	Year One and ongoing	Project Venture web page established and revised	I, O	Web Site Observation, Project Venture Database, Lessons and Units Published
Purchase required hardware/software for TOTs	Determine platform, hardware, presentation, and software needs	Project Master Trainer TOTs	January 30,1998	Training equipment in place for use	I, O	Year 1
Establish computer labs in replacement teacher classrooms	Obtain, set-up and maintain a five multi-media computers, presentation device, and laser printer	Purchased by Individual Districts Technical Assistant	First quarter year one	Labs in place for use in 14 classrooms; Teacher journal entries	C	Year 1
Initiate TOT activities	Facilitate 1 day per week for trainers to dialogue, reflect and receive direct instruction	Master Trainer	Ongoing	Minutes from TOT training sessions/meetings, TOT journals	I, O	Professional Development, TMT Time Report, TMT Focus Group
Purchase needed software for classroom use	Develop evaluation criteria; Conduct software needs assessment;.	TOTs with direction from Master Trainer Purchased by Individual Districts	Initiated first year and continued throughout project	Site software inventories, completed evaluation forms	I, O	Classroom Observations, Teacher Reflections, TMT Time Reports, TMT Focus Groups
Administer training level self-assessments to all consortium staff member	Develop assessment rubric and instrument, administer, and place staff in training program	Master Trainer TOTs	Year One; refine and re-administer annually	Training level self-assessment instrument; staff placement statistics	I	Technology Self-Evaluation

Activities	Process/Technique	Person Responsible	Timeline	Outcome	Progress to Date	Evaluation Instrument
Purchase, develop, field test and refine Level I and II training curriculum and materials using multiple means of delivery (accommodating all audiences) with common objectives.	Collaborative planning sessions; team teaching, revision of curriculum Stevens Institute to donate Internet training materials	TOTs in collaboration with Master Trainer Stevens Institute State Technology Resource Center	Initiated first year and continued across project years	Aligned training curriculum and materials posted for use and dissemination on Project Venture web page	I, O	Professional Development, Technology Training Courses, Technology Training Course Evaluations, TMT Time Reports, TMT Focus Groups
Conduct Technology Courses for Level I and Level II teachers	Conduct training levels self-assessments; coordinate outside training resources; coordinate registration procedures.	Master Trainer TOTS State Technology Resource Centers Stevens Institute	Years 1-5	Registration lists and sign-in sheets, course evaluations, trainer journal reflections	I, O	Technology Training Course Lists, Technology Training Course Evaluations
Initiate Level III in-class model/team teaching with classroom replacement teachers	TOTs team-teach with classroom teacher replacements focusing on technology integrated curriculum aligned with State Content Standards	TOTs with teacher replacements	Year One; four days at beginning of the year, less time as teacher replacements gain Level III-IV independence	TOT journals, replacement teacher journals, individual lesson plans, published curriculum	I, O	Classroom Observations, Teacher Reflections, TMT Time Reports, TMT Focus Groups
Determine criteria and select Level III classroom teachers (15 teachers per TOT)	Develop and administer application Initiate application process Select Level III teachers	Master Trainer; TOTs, Districts	Years 2-5	Application; selected Level III teachers	I, O	Project Venture Teacher Agreements

Activities	Process/Technique	Person Responsible	Timeline	Outcome	Progress to Date	Evaluation Instrument
Establish computer labs in Level III classroom teachers (210 per year)	Obtain, set-up and maintain a five multi-media computers, presentation device, and laser printer	Purchased by Individual Districts Technical Assistant	Years 2-5	Labs in place for use in 14 classrooms; Teacher journal entries	I, O	Classroom Observations, Budget Report filed with Year 2 Project Report, June 8, 2000
InitiateTOT mentoring activities with Level III teachers (210 project-wide)	TOTs and classroom teachers collaborate to develop integrated curriculum, team teach	TOTs Level III teachers Technical Assistant	Years 2-5	TOT journals, Level III teacher journals, integrated curriculum	I, O	Teacher Reflections, TMT Time Reports, TMT Focus Groups
Evaluate, refine and publish curriculum	Feedback from teachers, refine curriculum based on comments	Master Trainer, External Evaluator	Years 1-5	Revised curriculum model to be disseminated via project web pages	I, O	Technology Training Course Lists, Technology Training Course Evaluations, TMT Time Reports, TMT Focus Groups

Activities	Process/Technique	Person Responsible	Timeline	Outcome	Progress to Date	Evaluation Instrument
Determine degree of curriculum alignment with State Content Standards and amount of technology integration of individual districts	Local committees review current curriculum	Individual Districts in cooperation with TOTs and curriculum staff	Year One and ongoing	Report submitted to Project Venture	O	Classroom Observations, Project Venture Database
Complete curriculum alignment led by the Tri-District Consortium, including technology integration	Local content committees align curriculum to Standards	Tri-District Consortium Director	Year One and ongoing	Curriculum Published on Project Web Site	O	Project Venture Database
Completed curriculum compared to individual district’s curriculum to ensure continuity	District content committees to compare respective curriculum to Tri-District Model Curriculum	District Curriculum Directors	Year One and ongoing	Report submitted to Project Venture	O	Classroom Observations, Project Venture Database
Simultaneously develop an on-line curriculum that includes embedded technology	Level III and IV teachers adjust curriculum to include integration of technology	Master Teacher TOTs Level III and IV Teachers	Year One and ongoing	Publish Revised Curriculum on Project Web Site	O	Project Venture Database
Publish and disseminate aligned curriculum to multiple sites via project web pages	Use CISCO framework to provide searchable database of on-line curriculum	Technical Assistant	Year One and ongoing	Project Venture web pages	O	Project Venture Database
Field-test evaluate and refine technology curriculum	Project Venture Level III, IV teachers to pilot curriculum in their classrooms	Level III, IV Classroom Teachers	Year One and ongoing	Teacher Journals, and revisions and dissemination of published curriculum	I, O (District activity)	Project Venture Database

Objective 3.1 By the end of year two, a formative evaluation will occur to assist with ongoing project refinement and provide feedback for modifications in the training model.

Objective 3.2 By of the project, a summative evaluation will be submitted to the Governing Boards of participating Districts, the Arizona Department of Education and the United States Department of Education.

Activities	Process/Technique	Person Responsible	Timeline	Outcome	Progress To Date	Evaluation Instrument
Contract with Project Evaluator	RFP to be issued	Creighton District External Programs Coordinator	February, 1999	Contract Signed	C	Year 1
Select/develop evaluation instruments for identified data to be collected	Determine specific curriculum, training, attitudinal measures, and student achievement data to be analyzed; select/develop instruments	Master Trainer External Evaluator Participating districts	Year One	Evaluation instruments developed	O	Formative Evaluation Plan
Gather baseline data for evaluation	Administer evaluation procedures and analyze appropriate data	External Evaluator Master Trainer TOTs Participating Districts	Year One	Project statistics from various data sources gathered	C	Year 1
Conduct ongoing assessment and evaluation of staff training program	Data will be used to provide ongoing adjustments to training curriculum	External Evaluator Master Trainer TOTs	Ongoing	Ongoing statistical data, training participant evaluations	O	Evaluation Data Source Matrix, Professional Development, Technology Training Course Lists, Technology Training Course Evaluations

Activities	Process/Technique	Person Responsible	Timeline	Outcome	Progress to Date	Evaluation Instrument
Conduct annual assessment and evaluation of training programs	Data will be used to provide adjustments to training programs and will be disseminated via Project Venture’s web site to staff and community members of involved districts, State Department, Arizona districts, and federal government	External Evaluator	Years 1-5	Evaluation document	O	Evaluation Report
Conduct annual assessment and evaluation of teacher progress	Staff self-assessment will be analyzed to determine growth	External Evaluator Master Trainer TOTs	Years 1-5	Evaluation document	I (not ongoing, if Arizona assumes this activity)	Technology Self Evaluation
Conduct annual assessment and evaluation of teacher-generated curriculum	Assess published curriculum based on technology integration and alignment with State Content Standards	External Evaluator Master Trainer TOTs	Years 1-5	Evaluation document	I, O (by Districts only)	Project Venture Database
Conduct annual assessment and evaluation of student progress towards mastery of State Content Standards	Administer student performance assessments; Analyze performance data	External Evaluator Master Trainer TOTs	Years 1-5	Student performance analyzed	I (by Districts only)	Not evident in Year 2 Formative Report
Conduct annual assessment and evaluation of staff, student and community attitudes towards Project Venture implementation	Administer and analyze surveys	External Evaluator Master Trainer TOTs	Years 1-5	Evaluation document	X	Not evident in Year 2 Formative Report
Prepare and submit summative evaluation to USDOE, participating districts, Governing Boards	Data will be used evaluate overall success and impact towards systemic change Replication of the training model will be specifically examined	External Evaluator	Year 5	Evaluation document On-line publication of evaluation results	X	Year 5 Activity

The current external evaluators from Arizona State University are Professors Ann Igoe and James Klein of the Division of Psychology in Education and graduate students Patricia Fey and Sean Gilboy. During this second project year, the evaluation team created a Project Venture Logic Map and an Evaluation Data Source Matrix to support the map. This Formative Evaluation Approach plan evaluates the project goals and objectives and captures the strength of the Project Venture vision.

The evaluation team has developed an Evaluation Data Source Matrix (Table 4) that lists evaluation instruments used in Year 2 and suggests possible instruments for Years 3-5. The evaluation instruments are aligned to the Logic Map by using the numbers that have been assigned to the Intermediate and Terminal Goals on the logic map. Other columns of the matrix specify an identification number, define each instrument, its purpose, the provider of the data, proposed schedule for data collection, and the project year evaluation report that will include the measure. These instruments have been organized into seven categories: Administration (A), Evaluation (E), Lessons and Units (L), Technology Mentor Teacher (M), Project Venture Consortium Activities (P), Student (S), and Teacher (T). Each of the evaluation instruments has been identified and designed to meet the purpose of Objective 3.0. The evaluation instruments included in this formative report are described later in the report.

The Project Venture consortium members played a pivotal role in the creation of this evaluation report by offering extensive comments about the organization, format, and content of the report. The external evaluation team suggests that the instruments be reviewed and possibly revised on an annual basis to ensure useful data collection. This will allow the project to choose those instruments and measures that provide them with the most valuable information from which to make decisions.

Table 4: Evaluation Data Source Matrix

EVALUATION DATA SOURCE MATRIX

Category: Administration

ID #

Data Instrument

Purpose

Logic Map Goal(s)

Data Provider

Collection Schedule

Report Year 2 3 4 5

A1

Schools and District Baseline Information

Comparison of Project Venture activities and results to beginning of project for student and teacher demographics and classroom hardware and software in Project Venture schools

9, 10

Project Venture Administrators Grant Proposal

Beginning of grant period

A2

District and School Demographic Information

Comparison of Project Venture activities and results to prior periods

9, 10

Project Venture Administrators

Annually at end of school year

ü

ü

ü

ü

A3

Hardware and Infrastructure Installation Report

Assess Teacher and Student access to technology Identify issues faced by districts, schools, and teachers

9, 10

Project Venture Administrators

Annually at end of school year

ü

ü

ü

ü

A4

Administrator Interviews

General understanding and awareness of infrastructure issues, progress, and opportunities of technology integration for learning

All

Evaluation Team

Annually at end of school year

ü

ü

ü

Category: Technology Mentor Teacher

ID #

Data Instrument

Purpose

Logic Map Goal(s)

Data Provider

Collection Schedule

Report Year 2 3 4 5

M1

TMT Interviews

Alignment of TMT observation and evaluation instruments with Project observations and evaluation instruments to gain a general understanding and awareness of progress, opportunities, and issues of technology integration for learning

1-10

Evaluation Team

December and May

ü

ü

ü

M2

TMT Time Reports

Amount of time spent on types of task Comments provide information about issues, infrastructure, successes, skill level, etc.

1-10

TMTs on-line report

Weekly

ü

ü

ü

ü

M3

Technology Training Course Evaluations

Attitude of participants re: course content and quality

3, 4, 6

Training class participants on-line evaluation

Ongoing

ü

ü

ü

ü

M4

TMT Focus Group

General understanding of Project Venture from TMT perspective, what is good, what should be improved, etc.

1-10

TMTs, Evaluation Team

End of school year

ü

ü

ü

ü

Category: Project Venture Consortium Activities

ID #

Data Instrument

Purpose

Logic Map Goal(s)

Data Provider

Collection Schedule

Report Year 2 3 4 5

P1

Training Course/Class Registration Records

Number and type of courses presented , number of students taking courses

3-4

Project Venture and/or District Documentation

December and May

ü

ü

ü

ü

P2

Teacher Selection Criteria and Agreements

Selection differences, similarities, and constraints across districts, # teachers selected by district, agreement of criteria with Project Venture Levels and teacher skill

4-7

Project Venture and/or District Documentation

Annually, at end of school year

ü

ü

ü

ü

P3

On-line Technology Skill Self Evaluation

Self-report of technology skill set, helps to identify skill levels 1-2-3 related to project goals, provides individual comparison year-to-year

4-9

Self-evaluation on-line instrument

Annually

ü

ü

ü

ü

P4

TMT Selection Criteria

TMT selection processes for each district, helps to assess similarities and differences, expectations, skill sets, job duties

1

Project Venture and/or District Documentation,

Annually, at end of school year

ü

ü

ü

ü

P5

Annual TICG Database Project Report

Project Director’s Annual Report summarizes project goals and activities

All

Project Venture Documentation TICG Database

Annually at end of project year

ü

ü

ü

ü

Category: Student

ID #

Data Instrument

Purpose

Logic Map Goal(s)

Data Provider

Collection Schedule

Report Year 2 3 4 5

S1

Stanford 9 Scores

Measure of student achievement

121

Arizona Department of Education

Annually in August

ü

ü

ü

S2

Student Interviews or Focus Group

General understanding of student attitude regarding technology integration for learning

9, 10, 12

Evaluation Team

Focus Group and/or interviews annually Ongoing, during Classroom Observations

ü

ü

ü

This report represents results from the formative evaluation of Project Venture during the second funding year, completed June 8, 2000. Project Venture personnel successfully completed many foundational activities during this implementation year, which will allow them to continue to build the technology integration skills of their teachers. The findings and recommendations reported below can be used to inform stakeholders about project successes and opportunities for improvement. The recommendations are based on the results that are reported in following sections of this report, and upon the reflections of the external evaluators as they reviewed the project year from their perspective.

Finding 1

The Technology Mentor Teacher (TMT) role is an important aspect of project success and will have major impact on project replication and sustainability after the TICG grant funding ends. Technology training courses received positive evaluations both for instructor quality and content. Teachers felt that they learned well from both the training courses and mentoring. In addition they appreciated the personal approach and the opportunities for hands-on practice. Strategies for student learning and interaction in the classroom became more student-centered.

Finding 2

Evidence from Classroom Observations, Teacher Interviews, and Teacher Reflections from Level III and IV teachers suggest an increase in teacher skill and technology integration in the classroom as they worked with the TMTs throughout the project year.

Finding 3

The TMT Focus Group results validated the TMT job responsibilities as they had been defined in the TMT Time Report. The results from the focus group match the TMT Time Report results almost exactly. The four main task categories from the focus group were administration, instructional activities, professional development and technical support. Three of these categories also account for the majority of the time spent as reported on the TMT time report: Teacher Contact (which includes mentoring, training, and advisement) 45%, Administrative (which includes technical support) 28%, and Professional Development 12%. TMTs spent 45% of their time in direct teacher contact.

Finding 4

The diversity of the 17 districts that make up the Project Venture consortium results in very complex project activities and relationships. The Advisory Team directly supports the transition of district Project Venture activities into the supporting umbrella of the consortium.

Finding 5

The courses offered most often were in areas where the teachers already felt comfortable. The areas where teachers had the least skills (e.g., databases) also had the fewest opportunities for courses.

Finding 6

Project Venture Teacher Reflection documents were very helpful in defining a picture of classroom functions and successes.

Finding 7

The completion of equipment delivery and set up took most of the first semester in many classrooms, as mentioned in the Teacher Reflections, TMT Focus Group, and TMT Time Reports.

Finding 8

The TICG database entry process in May took significantly longer and was more intense than anticipated by the Project Advisory Team. The consortium members were not asked to provide their input to this required report until after the database was officially endorsed by the funding agency. In retrospect, they should have been advised in April that the database would probably be used for the Project Report in May so they could have begun gathering the information.

Finding 9

The Project Venture database of lessons and units is beginning to function as expected and provides world-wide access to Project Venture lessons and units.

Published Lessons and Units (L2)

Project Venture teachers in each district prepare lessons or units that are aligned with Arizona State Content and Technology Standards on an ongoing basis. At this time the units reside within each of the district schools and are being transferred to the Project Venture website as time permits. At the end of the second project year, 100 lessons or units had been created, and 75 were available for posting to the web site. Table 5 shows the number of units available in grade level groupings.

Table 5: Project Venture Published Curricular Units by Grade Level Groups

Each consortium member evaluates and submits lesson or unit materials that are available from their district and they are added to the web site. A consortium-wide rubric and/or checklist to evaluate these lessons or units is being developed.

TMT Time Report (M2)

An online reporting instrument is used to capture weekly time and activity reports from the Technology Mentor Teachers (TMTs) (See Appendix and http://research.ed.asu.edu/projectventure/tmttime/TMT.html). Project Venture Technology Mentor Teachers filled out the report on a weekly basis between September 1999 and June 2000. These reports document the amount of time spent in each of eleven reporting categories. Table 6 presents a complete description of each category. Responses to three open response items captured the following information: "successes this week," "frustrations this week," and "learned this week." Each of these areas captures an important aspect of the TMT role for Project Venture.

Table 6: TMT Weekly Report categories

Each week the TMTs connect to the evaluation website, enter the Time Report section and input the requested information. When the report is saved a copy should be automatically emailed to the TMT’s supervisor.

Consortium-wide

A selected sample of TMT Time Reports taken over the entire project year was analyzed provide a model for the average TMT task and time responsibilities.

Time Categories

The following graph (Figure 3) represents the average percentage of time spent in each of the eleven time reporting categories by the Project Venture TMTs. The Teacher Contact category combines Levels I, II, III, and IV Classroom Mentoring (20%), Technology Classes (13%), Individual training (5%), and Advisement (7%) to account for 45% of the TMT time being reported. The results in these categories showed a higher percentage of Level I and Level II activities taking place between September and December and a higher percentage of Level III and Level IV activities taking place during the last half of the school year. Administrative duties which included equipment setup and troubleshooting required 28% of TMT time. Some of this time should also be considered teacher contact time. This category was expected to be high because the project was just beginning its actual work of defining procedures and activities and putting equipment and software in place. TMT Professional Development accounted for 12% of the TMT time, Travel and Other account for 15%. Even though a selected sample of time reports from each district was used, all of the consortium districts reflect this general pattern of time in category. It is important to note that 45% of TMT time was devoted to teacher mentoring and training even though this was the first year of full project implementation.

The percentage of time spent in each category by school district is presented in the following table (Table 7) to point out the similarities and differences between school districts, and which will serve as focal points for discussion as Project Venture plans for project replication and sustainability.

Table 7: Average percentage of time spent in task categories during the 1999-2000 school year.

The following differences between districts can be noted from Table 7:

• Creighton and Kyrene School Districts have focused on higher level skill mentoring at Levels III and IV.
• Tempe Elementary District TMTs worked specifically with Level II and Level III classrooms and provided a very high percentage of technology courses and individual training.

• Maricopa Small School District Consortium does not have the same level of technology infrastructure as the other districts. It was important that the TMTs be able to assist the district schools with technology planning during the second half of the school year.

• Maricopa Small Schools devoted more time to Administrative duties in order to ensure the necessary infrastructure was in place for this diverse district.
• Tempe Union High School District assigned overall project administrative duties to one TMT when the district Project Venture administrator left the district and was not immediately replaced.

• Maricopa Small School District Consortium is geographically dispersed and requires the TMT to travel long distances.

• Creighton School District TMTs were often assigned to regular school duties, even though they were considered to be full-time TMTs.

Open Response Questions

The Project Year 1999-2000 constituted the first year of implementation of the mentoring model and the TMT position; therefore, special attention was given to discovering similarities of duties and implications during the analysis of this data in order to better define the role and discover aspects that would be replicable. The results are presented below in aggregate across districts.

Successes this week:

Five success categories emerged from data analysis: Teacher-TMT rapport, equipment installation and troubleshooting, administrative tasks, professional development, and support received are shown in Table 8.

Success Category	District
	Creighton	Kyrene	Maricopa	Tempe Elementary	Tempe Union	Response Totals
Teacher Rapport	47%	30%	25%	60%	33%	40%
Equipment/ Troubleshooting	15%	5%	44%	8%	5%	19%
Administrative Tasks	18%	20%	7%	12%	22%	15%
Professional Development	8%	20%	14%	16%	23%	13%
Support Received	12%	25%	9%	4%	17%	12%

Table 8. Success Category Response Percent by District.

The teacher rapport category comments focused most often on being able to "build a relationship with the Project Venture Teachers", "gain access to the classrooms", and "establish contact with the teachers" and were, for the most part, submitted in the first two months of the calendar year 2000. The second most common category referred to being able to successfully install and troubleshoot equipment and software, and was most often mentioned between September 1999 and January 2000, a period of initial equipment delivery and setup.

Frustrations:

Five categories for "frustrations this week" also emerged: Teacher-TMT rapport and participation, administrative tasks, equipment and troubleshooting, time management, and lack of support. These results are shown in Table 9.

Table 9. Frustration Category Response Percent by District.

Frustration Category	District
	Creighton	Kyrene	Maricopa	Tempe Elementary	Tempe Union	Response Totals
Time Management	31%	50%	17%	25%	44%	28%
Equipment/ Troubleshooting	23%	0%	21%	33%	28%	24%
Administrative Tasks	19%	50%	12%	17%	28%	19%
Lack of support	8%	0%	46%	0%	0%	16%
Teacher Rapport/ Participation	19%	0%	4%	25%	0%	14%

Time management frustrations were mentioned consistently throughout year and included lack of planning time with Project Venture Teachers, excessive amounts of time spent in meetings for Project Venture as well as school and district meetings, lack of time for mentoring due to troubleshooting and equipment related tasks. Being responsible for equipment and troubleshooting related tasks was a source of frustration, and were particularly common in the earlier part of the implementation process. Lack of support was mentioned in only two districts, but made up 16% of the comments. Within those districts this category included lack of follow through from the district on equipment acquisition and timely implementation.

Frustrations with teacher rapport and participation from the teachers were addressed in 14% of the comments. The most frequently occurring comments pertained to initial obstacles in building a mentoring relationship with teachers in the earlier part of school year, toward the end of the school year the comments referred to lack of participation from teachers in training courses or project planning.

Learned this week:

Categories developed for "learned this week" were: troubleshooting, mentoring skills, software applications and technical skills as shown in Table 10.

Table 10: Learned This Week Category Response Percent by District.

Learned This Week Category	District
	Creighton	Kyrene	Maricopa	Tempe El	Tempe Union	Response Totals
Troubleshooting	17%	33%	42%	100%	60%	38%
Mentoring skills	25%	17%	17%	0%	40%	22%
Software applications	33%	33%	17%	0%	0%	22%
Technical skills	25%	17%	25%	0%	0%	19%

Troubleshooting referred to both system setup and later time phases in the reporting interval. Both the acquisition of mentoring skills and knowledge in the use of software applications and programs were mentioned in 22% of the comments. Learning mentoring skills was consistently mentioned, especially the Facilitative Leadership Training that occurred in December 1999. TMTs specifically mentioned learning software applications such as Hyperstudio, Internet Explorer, PowerPoint and Excel and technical skills such as movie and video editing procedures, web page design and maintenance, and database use.

The responses to this portion of the TMT Time Report reflect both the role of the mentor and the nature of a program during initial implementation. Categories appearing as both successes and frustrations point to the beginning of the school year and project implementation and to the latter half of the school year when teachers and classrooms were functioning well.

Technology Training Course Evaluations (M3)

Project Venture consortium members offered Technology training courses to teachers, staff, administrators, and community members in their districts. After a course was finished each participant was asked to complete an online course evaluation. The evaluation instrument consisted of three sections. The first section collected demographic information, which included course location, content domain, level of difficulty, and instructor name(s).

The second section included 16 questions, in 3 subsections, regarding course content, course instruction and course impact. Course impact was divided into two measures entitled ‘Skill Gain’ and ‘Future Use’. ‘Skill Gain’ addressed the respondents’ perception of how their skills had improved and if they felt they had gained mastery of the skills. ‘Future use’ pertained to the respondents’ plan to use their new skills at work and outside of work. These questions were answered on a 4-point Likert scale ranging from ‘strongly disagree’ (1) to ‘strongly agree’ (4).

The third section included two open response questions: ‘The best aspect of the course was’ and ‘Suggestions for course improvement’. Responses to these questions were recorded as text responses with no length limit. The Evaluation Survey may be found in the Appendix and online at http://research.ed.asu.edu/projectventure/course_eval/ .

Training courses in the seven core areas of technology training (Desktop Basics and Troubleshooting, Word Processing, Internet/E-mail, Instructional / Reference Programs, Spreadsheets, Databases, and Multimedia) were offered during this project year. These course categories matched the content and training objectives the consortium created during the summer of 1999 (see the Program Description section of this report). A sample of the course evaluations submitted by participants in the Technology Training Courses was aggregated and analyzed.

Responses from the second section of the evaluation are reported as the mean of the responses to the questions in each category: the participants’ experience with content (4 questions), the instructor and instruction (7 questions), and course impact: skill gain (2 questions) and future use (3 questions).

As can be seen in Table 11, the means for each evaluation category were above 3, indicating that the respondents had a positive experience in terms of the course content, instructional approach, skill gain, and assumed future use.

Table 11: Mean Responses to Course Evaluation Categories

The third section of the evaluation asked the respondents to list the best aspect of the course and suggestions for course improvement. Content analysis on the responses to the question: "The best aspect of the course was’ resulted in 13 categories. The four categories with the most frequent responses are shown in Table 12.

Table 12: Open response categories for ‘the best aspect of the course was’

The most frequent comments referred to Instructor qualities and pertained to good teaching. Patience, knowledge, and organization were also noted numerous times. Other statements in this category included: ‘two instructors’, ‘flexibility’, ‘motivated’, ‘sensitive to individual needs’, ‘willing to answer questions’, ‘pacing’, and ‘compassionate teaching’.

For the content category, respondents liked specific programs that were taught and appreciated learning specific skills. Respondents also mentioned: ‘the sizable amount of information’, the ‘variety of information’, ‘written instructions’, ‘internet and email’, ‘increased awareness of technology in the classroom’, and ‘using peripherals’.

The second question, ‘Suggestions for improvement,’ resulted in 18 categories. Table 13 summarizes the results for the five highest-ranking categories.

Table 13: Open response categories for ‘Suggestions for improvement’

The ‘request for more time’ (24.3%) was primarily endorsed in a positive context; the teachers would have liked more time to explore the course content and associated technology. 18.6% of the respondents addressed course content. Several comments called into question why they were learning programs that were not made available at their school sites and having different version of programs (i.e. Word97 vs. Word2000) in the course and in the classroom. A few comments referred to requests for more classes in the same content area, a new content area or with a specific program. Finally, a few suggestions included a request for a review of program basics before delving into the course content.

Individual Course Evaluations are a valuable tool that informs TMTs regarding technology training course success and provides information about possible revisions and additions to the Technology Training Courses as necessary.

TMT Focus Group (M4)

The external evaluation team conducted two TMT focus group sessions at the end of the project year. TMTs were invited to attend one of two focus group meetings at their convenience. The meetings were scheduled for a four hour morning block. 100% of the TMT’s attended: Creighton (6), Kyrene (2), Maricopa Small Schools (2), Tempe Elementary (2), and Tempe Union High School (3). The first focus group had 12 participants representing 4 school districts and the second group had 5 participants with representatives from 3 districts. Two members of the administrator’s group, who had been performing TMT duties, attended the first focus group. In addition to the TMTs and administrators, there were two members of the ASU external evaluation team in attendance at each meeting.

There were two purposes of the focus group. The first was to discover the similarities in TMT job duties among districts. The evaluators hoped that a formal TMT job description for Project Venture could be defined that would allow the project to begin to document aspects of the training model that are replicable by others. The second purpose was to elicit information in a confidential manner that addressed two of the themes of the TMTs weekly time reports: the successes and frustrations of the job. The evaluators wanted to document issues that might not be broached with the project administration but which might impact successful implementation of the project.

During the first portion of the meeting, the participants were asked to write down every job duty they performed in their role as TMTs. These duties were listed on flipcharts as the TMTs reported them verbally. Discussion about many of the items allowed common definitions to emerge. Finally, the participants located their own responsibilities on the flipcharts and marked them with a checkmark. When this process was completed a list was formed that included only the TMT responsibilities that were selected by 100% of the TMTs as being an integral aspect of their job.

A similar process was followed during the discussion of successes and frustrations. However, the resulting lists of successes and frustrations included all of the comments, not just those that were selected by the entire group. The questions were asked in an open response manner and the TMTs were assured their answers would remain confidential and that all results would be reported in the aggregate. This assurance was mentioned often by the TMTs as an important aspect of their willingness to participate in the process. Therefore, no results will be reported by district.

Responsibilities:

The responsibilities reported by 100% of the TMTS were organized into four general categories: (1) Administration, (2) Instructional Activities, (3) Professional Development, and (4) Technical Support and are shown in Table 14. The categories are reported in alphabetical order and do not represent any ranking of importance.

Table 14: Universal TMT Job responsibilities reported by All TMT Focus Group Attendees

Within the Administration category (1) there were three sub-categories: (A) General/Communications, (B) Meetings, and (C) Project Venture. The first category, General/Communications (A), included all forms of communication, the collection and dissemination of information and all office tasks related to managing their position. The Meetings sub-category (B) contained meetings with over 6 different groups that were held on a regular basis. The Project Venture sub-category (C) outlined the tasks specific to project criteria, including filling out all information reporting instruments for self-monitoring and evaluation, creating technology integration standards and publishing materials and promoting the program within and between districts.

The Instructional Activities category (2) was divided into three sub-categories: (A) Direct Instruction/Contact, (B) Support Services, and (C) Delivery of Training. Direct Instruction (A) included all interaction directly impacting students and teachers in Project Venture classrooms. Support Services (B) was the broadest category including everything from coaching teachers in technology integration and developing lessons or units for the classroom to troubleshooting and finding resources for the classroom. The third sub-category, Delivery of Training (C) referred to providing Project Venture technology training courses. This included the development of training materials, teaching the courses and coordinating the implementation of the training program.

The third reported category was Professional Development (3). This entailed all formal and informal TMT training and was inclusive of not only educational technology skill building but also building skills of instructional material development and dissemination.

The last category, Technical Support (4), was divided into two sub-categories: (A) Hardware/Software Infrastructure and (B) Implementation. The former involved researching, evaluating and ordering the software and hardware as well as physically delivering and setting up much of it. The latter sub-category, (B), included technical troubleshooting, installation and hard and software, and playing a technology advisory role.

It is important to note that this is not an exhaustive list of duties carried out by each mentor; that list is much larger. Rather, these are the responsibilities agreed upon by 100% of the TMTs participating in the focus group.

Successes:

The reported successes were tabulated into six categories, three of which were directly experienced by the mentor (Professional Development, Job Gratification, and Support Received) and three of which were broader in scope (Teachers, Student Benefits, and Project Venture System Level). Table 15 lists these successes.

Table 15: Successes Reported by at least 80% of TMT Focus Group Attendees

Successes that were experienced directly by the TMTs included satisfaction with the various opportunities available for professional development, including on-the-job activities such as school visits with classroom observations and discussions with other TMTs. Cognitive Coaching training took place after the focus group meeting and is therefore not mentioned in this listing. Project Venture TMTs experienced a large measure of job gratification by being in a position to be an integral component of bringing integrated technology to the students and teachers. The support received category points out how important administrative, personal, and financial support is to the TMTs..

The latter three successes included seeing the Project Venture teachers successfully implement the technology integration training and grow in skill level. Student benefits were evident in the classroom as access to technology increased and student engagement and learning was observed. TMTs are generally satisfied with the overall concept and implementation of Project Venture and listed specifics as shown in the table above.

Five of the six successes are mirrored on the list of frustrations that follow in the next paragraph. These include Support Received (Lack of Support), Teacher Level (same), PV System Level (Administration), Professional Development (Lack of Support) and Job Gratification (intuited from the general level of frustration across categories). This indicates not only that they are important issues to the mentors but also key areas for the project to understand in order to minimize their presence on the frustration side of the equation.

Frustrations:

The frustrations have been categorized into seven categories: Administrative Burden, Lack of Support, Mission Conflict, Planning, Project Venture System Administration, Role Definition/Function, and Teacher Related. The categories are presented alphabetically in Table 16.

Table 16: Frustrations Reported by at least 80% of TMT Focus Group Attendees

The frustrations noted in the table above are directly related to many of the successes as well. This points out the necessity for a well-defined set of job responsibilities and administrative procedures across the project. Planning and administrative difficulties caused quite a few of the frustrations voiced.

The assurance of confidentiality allowed the TMTs to openly discuss their frustrations and to begin to put them in perspective when seen in conjunction with the successes and the job responsibilities. It should be remembered that this was the first year of TMT activities and they did not feel that they had well-defined consortium-wide roles and expectations for the project as a whole. This is an extremely dedicated group who took their responsibilities very seriously and who were excited to be performing in a valuable position.

Training Courses/Class-Registration Records (P1)

The training model adopted by Project Venture utilizes the technology skills and expertise of the Technology Mentor Teachers as developers and instructors of technology training courses. Each consortium member district scheduled a customized list of courses for the teachers of that district based on the teacher, staff, and administrative technology skill needs of the district. Each hands-on course was categorized as Level I — Level IV by the skill level of the technology being taught as required in the Training Objectives for the project that were developed by the consortium during the summer of 1999. The integration of technology with content and technology standards are presented in many Level I classes and all Level II and IV classes. A copy of the training objectives is located in the Appendix and online at http://www.creighton.k12.az.us/projectventure/pages/objectives.html.

Table 17 shows the number of courses taught by the 16 TMTs for the school year 1999-2000. Courses were often taught by teams of two TMTs and were considered to be on-the-job training opportunities that helped to build the training expertise of the TMTs.

Table 17: School Year 1999-2000 Level I - Level IV Technology Training Courses Taught and Number of Participants

LevelTempe Elem.	Tempe Union	Total
1	# courses / # participants	5 / 161	33 / 298	5 / 95		3 / 65	28 / 288	74 / 907
2	# courses / # participants	10 / 257	22 / 137	0 / 0		9 / 220	36 / 376	77 / 990
3	# courses / # participants	15 / 272	20 / 187	0 / 0		7 / 40	34 / 222	76 / 721
4	# courses / # participants	0 / 0	12 / 152	0 / 0		0 / 0	0 / 0	12 / 152
Total	# courses / # participants	30 / 690	87 / 774	5 / 95		19 / 325	98 / 886	239 / 2770

The average class size for all classes offered by Project Venture districts was 23 in the Creighton district, 8.9 in Kyrene Elementary, 19 in Maricopa Small Schools, 17.1 in Tempe Elementary, and 9.0 in Tempe Union High School.

On-Line Technology Skill Self Evaluation (P3)

The Project Venture consortium members prepared the Technology Self Evaluation instrument during the beginning months of the project’s second year. The evaluation could be taken by any teacher, staff, administrator, or parent at the consortium member schools in order to assess their technology skill level.

The intention of this effort was to match the consortium’s stated teacher technology skill and knowledge objectives to individual questions on the self-evaluation instrument and thereby place an individual’s skills in Level I-IV. This instrument focused primarily on the application of computer skills at Levels I and II in each of the core areas determined by the consortium (Desktop Basics and Troubleshooting, Word Processing and Desktop Publishing, Internet, Spreadsheets, Databases, Instructional Software, and Multimedia). Level III Integration and Level IV Mentoring skills were broadly defined in the Training Objectives document and therefore were not well represented in the self-evaluation instrument. The instrument easily identifies individuals at Level I skills and can provide information about Level II and III skills. The instrument cannot identify Level IV or V skills at this time.

The EdCare Lab hosts the instrument on their web site and agreed to conduct an analysis of the results. The original external evaluator planned to use a sophisticated statistical technique, Item Response Analysis, which would determine the specific questions that would most efficiently define the level of expertise a respondent had. This statistical approach would allow the instrument to be formulated very specifically to the Project Venture Training Objectives. A well-validated and reliable self-report instrument would inform the TMTs about the technology skill level of the teachers they are working with in the classroom. It is also important to know teacher skill level in order to track the success of the project goal to increase the training of teachers to integrate technology in their teaching.

The Project Advisory Team worked closely with the external evaluation team to develop the self-evaluation tool. It was revised many times due to changing consortium and evaluation personnel at various development and acceptance meetings. The final evaluation instrument was implemented in late fall of 1999. The TMTs and Project Venture Advisory Team comprised the pilot group for the self-evaluation instrument. The consortium made substantial changes after this pilot testing. The TMTs did not retake this revised version, therefore no comparisons are being made between the version of the self-evaluation instrument taken by the pilot group and the final version of the self-evaluation instrument.

Consortium members were asked to have their district’s teachers complete the self-evaluation process by the end of December 1999, but were given extensions for completion by the end of January 2000. A sample of 933 teachers completed a technology self-evaluation survey by the end of January 2000. This sample included 83 teachers who had been selected to participate in Project Venture and 850 teachers who were not identified as Project Venture teachers. Respondents used a three-point scale (1 = I have no experience, 2 = I have a little experience, 3 = I have more than a little experience) to rate their own experience using word processing programs, the Internet, spreadsheets, databases, multimedia programs, and instructional software. Answers of "I have a little experience" or "I have more than a little experience" moved the respondent into a more comprehensive layer of the survey that was intended to identify Level II-IV skills.

Consortium-wide.

Results are only reported for the first section of the survey that addressed a respondent’s initial reflection of their skill level. Results for 83 Project Venture teachers who completed the technology self-evaluation survey are presented in Table 18. A majority of Project Venture teachers responded that they had more than a little experience using word processing programs (91.6%), the Internet (90.4%), instructional software (66.3%), spreadsheets (55.4%), and multimedia programs (53%). A majority also indicated that they had a little experience using databases (57.8%).

Table 18: Frequency distribution (percent) of self-rated technology skills for Project Venture teachers-consortium wide (N=83)

Results for 850 non-Project Venture teachers who completed the self-evaluation survey are shown in Table 19. A majority of these teachers responded that they had more than a little experience using word processing programs (67.8%) and the Internet (56.8%). Most non-Project Venture teachers reported having a little experience using instructional software (51.4%) and spreadsheets (45%). However, a majority of these teachers indicated no experience using multimedia programs (51.4%). Non-Project Venture teachers were split between having a little experience (40.5%) and no experience (42%) using databases.

Table 19: Frequency distribution (percent) of self-rated technology skills for non-Project Venture teachers-consortium wide (N=850)

Individual results for each consortium member school district are located in the Appendix.

Teacher Journals and Notes (T1)

Teachers had been asked to keep Teacher Journals that were reviewed during Classroom Observations. Teacher reflection notes were piloted with the Level III Project Venture teachers from Creighton Elementary School District at the end of the project year. Based on the findings reported here, this measurement instrument holds promise as a valuable tool for Project Venture and procedures should be considered for gathering teacher reflection information on a regular basis.

Comments from the teacher reflection notes fell into three overall groupings: Positive, Negative, and Future-Oriented. Table 20 summarizes the categories that emerged following content analysis in the positive groupings. Tables 21 and 22 report the negative and future-oriented groupings respectively.

Table 20: Positive Categories Emerging from Project Venture Level III Teacher Notes

Table 21: Negative Categories Emerging from Project Venture Level III Teacher Notes

Table 22: Future Oriented Categories Emerging from Project Venture Level III Teacher Notes

Classroom Observations (T3)

The external evaluation team conducted classroom observations in April and May 2000. A total of 21 Level III and IV Project Venture teachers were observed as they taught a lesson to their students. These teachers were scheduled for observation by TMTs as classrooms were available. The classrooms that were observed ranged from grades 1-12 and covered a variety of subject and content areas. A structured classroom observation instrument was used to assess how technology was used and integrated and to determine the types of interactions and activities that occurred during these lessons (see Appendix A for sample form). Each observation lasted an average of 60 minutes. Observation data were analyzed using written transcripts of the structured classroom observation forms. Data analysis also included a review of the lessons observed and any supplemental lesson plan and instructional materials provided by the teachers or TMTs.

The following list provides the main findings from the classroom observations:

• Project Venture teachers exhibited knowledge of the technologies and computer programs they used. Teachers were able to answer students’ questions and provided guidance in skill acquisition.
• Teachers used technology in a variety of ways, ranging from information presentation during a lesson to using the Internet as a resource.

• Most instructional activities were student-centered and mainly focused on independent student learning or small group cooperative learning.
• Observed classrooms had the technological infrastructure in place to support lesson goals and objectives.
• The lesson plans provided by the teachers included detailed descriptions of instructional activities and provided evidence of alignment with state standards.

Teacher Interviews (T4)

The 21 Level III and IV Project Venture teachers who were observed were interviewed after the observation period. The questions for this interview were part of a structured interview form that constituted a subsection of the classroom observation form (see Appendix A). Data were analyzed using a content analysis of the responses from the structured interview.

Table 23 provides detailed results from the teacher interviews. The following list provides some of the main findings:

• Teachers who participated in the interview reported that they had increased their knowledge of technology use through their work with the TMTs and their participation in Project Venture. Every teacher said they would participate in Project Venture again.
• Most teachers indicated that TMTs add the most value to the project and that it is helpful having a TMT as an additional resource in the classroom when attempting complex activities.
• Most teachers reported that increased access to computers had impacted their teaching, classroom, and students.
• Several mentioned that student self-confidence had increased through the use of technology.
• Many teachers reported that their instructional style changed as a result of access to technology in the classroom. They mentioned that they saw their role as a facilitator of learning.

Table 23: Responses to Classroom Observation Interview Protocol.

Conclusion

As the consortium enters third year of Project Venture they can be confident that they have the technical and human resources necessary to assist them with the integration of technology for teaching and learning.

Teacher Technology Training Objectives

• Instructional Software page 2
• Desktop Basics & Troubleshooting page 3
• Word Processing & Desktop Publishing page 6
• Spreadsheets page 9
• Database page 12
• Internet page 14
• Multimedia page 19

Teachers take classes according to their training levels:

INSTRUCTIONAL SOFTWARE OBJECTIVES

Level I	Level II	Level III	Level IV
IS.I.1 Participant will use drill and practice software	IS.II.1 Participant will use drill and practice software that includes adjustments for student levels and for teacher reporting.
	IS.II.2 Participant will understand the three types of software available for classroom uses.	IS.III.1 Participants will evaluate software as it is related to curricular needs and teaching strategies	IS.IV.1 Participants will act as a resource and promote the use of software specific to curricular needs.
		IS.III.2 Participants will use a variety of software given specific curricular objectives.
	IS.II.3 Participants will use a variety of software titles that are available at the site level.	IS.III.3 Participants will research software titles based on curricular needs.	IS.IV.2 Participants will use external sources (i.e. Internet software review sites, journals, colleagues, etc.) to seek out quality software given specific needs.
		IS.III.4 Participants will apply strategies for efficiently and effectively learning the operation of a new program.
	IS.II.4 Participants will use a presentation device for large group instruction of software applications.	IS.III.5 Participants will use a variety of large group, small group, and individual implementation strategies having specific curricular objectives in mind.	IS.IV.3 Participants will design effective learning environments based upon student needs and curricular objectives.

DESKTOP BASICS & TROUBLESHOOTING OBJECTIVES

Level I	Level II	Level III	Level IV
DT.I.1 Participant will identify computer components and their functions. Define computer components Connect CPU, monitor, keyboard, mouse, peripherals, network, and power supply
DT.I.2 Participant will contact the appropriate person to obtain a password(s).	DT.II.1 Participant will periodically change passwords to maintain security.
DT.I.3 Participant will demonstrate proper computer start-up and shutdown. Start up computer system Shut down computer system Perform cold boot Perform warm boot	DT.II.2 Participant will install and upgrade applications from disk or CD-ROM.	DT.II.1 Participant will install and/or upgrade applications using downloads from e-mail or internet websites.
DT.I.4Participant will access programs/files with or without desktop security program.(At Ease, Foolproof, On Guard, etc.)	DT.II.3 Participants will bypass desktop security programs if allowed.
DT.I.5 Participant will demonstrate proper usage of computer menu items and other desktop elements. Locate and use menu bar Select appropriate menu item for a desired task Respond appropriately to dialog boxes. Apply appropriate mouse clicks to icons and buttons Distinguish between icons, buttons and menus Distinguish between applications, folders, documents, harddrive, servers and trash can/recycle bin Use vocabulary and terminology
DT.I.6 Participant will manage windows with a mouse. Use the mouse to click, double click and drag Open, close, move and resize a window Minimize/hide a window. Restore a minimized/hidden window Maximize a window Restore a maximized window. Tile and cascade displayed windows Hide all windows to display only the desktop	DT.II.4 Participant will increase computer functionality. Use appropriate keyboard shortcuts to execute commands Open and use more than one application at a time Put most used items in Start/Apple menu Use the find utility Use the control panel Lock file/folder Identify and manipulate preferences, defaults, and other selectable features of operating systems or applications Create an alias/shortcut when necessary/helpful. Use the alias/shortcut to locate target/original item	DT.II.2 Participant will customize their computer for advanced use. Make more memory available to one or more applications Configure network and internet settings Print to a network printer Add/setup printer software for local printer
DT.I.7 Participant will organize their files and folders so that they are easily accessible. Locate a file/folder Move a file/folder to a different location Copy a file/folder to a different location Delete/trash a file/folder Retrieve a deleted file/folder Empty recycle bin/trash Create a folder .	DT.II.5 Participant will connect to a server in order to access applications and access saved documents.	DT.III.3 Participant will share files between networked computers. Set up file sharing Log onto another computer Access and/or save to another computer
	DT.II.6 Participant will demonstrate essential computer maintenance. Make backup copies of files Identify system files Protect system files from deletion/modification Exchange files among different operating systems Install an application Upgrade an application	DT.III.4 Participant will perform advanced computer maintenance. Rebuild desktop(Mac only) Run appropriate disk repair utility Protect against computer viruses
	DT.II.7 Identify and describe the major operating systems associated with computer platforms found in the district (ISTE 4.4.1)	DT.III.5 Participant will use strategies to resolve compatibility issues. Save documents in cross-platform formats Format and/or use PC disks Use applications which are cross-platform (i.e., Word, html, etc,)
			DT.IV.1 Participant will teach troubleshooting techniques to another user.

Level I	Level II	Level III	Level IV
WP.I.1 Participant will navigate within an existing word processing document. Open an existing document by double-clicking on it Open a word processing application; open an existing document from within word processing application Change the insertion point location by using a variety of methods Use a variety of methods to move through a multiple-page document (i.e. scroll bar, arrows, page up, page down, etc.) Perform simple operations by using menu bar and toolbar Close a document and exit/quit the word processor application     WP.I.2 Participant will edit, save and print an existing word processing document. Use text wrap Start a new paragraph using the return or enter key Undo the last action Insert and delete text, spaces and returns Select text using a variety of methods Copy and paste text to a new location in the document Move text to a new location in the document using drag-and-drop Use a spell checker Name and save a document to a specific location or disk. Print a document. Utilize a template	WP.II.1 Participant will become proficient at independently navigating, editing and formatting with a word processing application. Use the help system as reference tool Use keyboard commands to navigate and edit the document Customize and use toolbars Use writing tools (i.e., thesaurus, grammar check, word count) Display and hide nonprinting characters
WP.I.3 Participant will create and format a new word processing document. Create new document and input text Change page orientation from portrait to landscape Understand default settings Change line spacing Change font, style and/or size Change justification Change margins of a document Use default tabs	WP.II.2 Participant will create, edit and format a new word processing document with graphics.   Find and replace text. Set tabs (i.e., left, center, right, decimal). Insert page breaks Save a document as a template. Insert headers and footers. Insert date, time and page number. Create labeled lists using application commands/buttons Change paragraph indentation settings using the ruler Insert a graphic from application's clip art collection. Manipulate a graphic (i.e., rotate, flip, resize, reshape) Wrap text around a graphic Create an outline using appropriate tools Insert and edit a table or chart	WP.III.1 Participant will create, edit and format a new document using desktop publishing techniques. Plan and design a visually appealing document (i.e., font, color, graphics, etc.) Create and use columns and columns breaks Copy and paste paragraph formatting/ruler Insert graphics from a variety of sources Use graphic tools to create special effects (i.e., shadows, transformation, etc.) Arrange multiple graphics (i.e., layer, align, group/ungroup, lock/unlock) WP.III.2 Participant will share/merge information from other applications with a word processing document. Mail merge with database or data documents Insert and link spreadsheet data Participant will use HTML features in a word processing application. Create a page with active internet links. Create a document and save as a HTML page.
		WP.III.3 Participant will develop and implement a whole group project which utilizes word processing/desktop publishing applications. (I.e., brochure, class newspaper, research, etc)	WP.IV.1 Participant will develop and implement a lesson or project where students utilize word processing/desktop publishing applications. (I.e., brochure, class newspaper, research, etc)

SPREADSHEETS OBJECTIVES

Level I	Level II	Level III	Level IV
SS.I.1 Participant will describe the various uses and educational applications of spreadsheets. I.e., develop various spreadsheets, analyze data, organize data
	SS.II.1 Participant will be able to input a given set of data into a predesigned spreadsheet. Enter data including text, numbers, and dates Save and close Use tool bars Open an existing document Print an existing document Use appropriate vocabulary regarding spreadsheets (cells, number cells, text cells, columns, rows, etc.)	SS.III.1 Participant will design a spreadsheet that presents in an organized fashion for a specific purpose. Column/row headings Row height, column width Entering data Formatting data for sorting
	SS.II.2 Participant will be able to edit and format a predesigned spreadsheet. Edit information within a cell (cut, copy, paste, clear) Edit information within a range of cells (cut, copy, paste, clear) Move data using "drag and drop" Delete data and undo action Select single cells, columns, rows, and whole spreadsheet Insert and delete rows and columns Adjust column widths and row heights Use numeric format Format data within a cell or groups of cells (fonts, size, color, alignment) Preview print jobs Use page setup options (print size, margins) Print spreadsheets Utilize fill commands	SS.III.2 After creating a spreadsheet, participant will be able to edit and format the document according to their needs. Print a document, cell ranges, etc. Sort in ascending vs. descending order Sort multiple rows and columns Protect and unprotect cells Shade ranges
	SS.II.3 Participant will be able to use basic formulas to obtain quantitative results given a set of data in a predesigned spreadsheet. Use a variety of cell references Use averaging tools Use auto sum	SS.III.3 Participant will be able to create formulas for numeric and text calculations. Format for calculations Use relative cell references Use absolute cell references Use relative and absolute references Use appropriate functions Use advanced formulas Use cell references (Absolute, Relative, and Mixed)
	SS.II.4 Participant will be able to create a graph or chart from a given set of data in a predesigned spreadsheet. Use wizard/menu bar/button bar to create a graph or chart Add footers and headers Use borders (outline, right, left, top, and bottom) Change display (Claris Works users) Use style sheets (create, modify, format cells)	SS.III.4 After creating a spreadsheet, participant will be able to analyze the data and determine appropriate charts and graphs to display the information. Identify purpose for each type of chart or graph Create appropriate chart and graph for intended purpose Format and modify charts and graphs to elaborate the purpose Modify charts and graphs (change types, reposition legends, resize/move)	SS.IV.1 After collecting relevant data, participant will be able to use spreadsheets to create charts and graphs which are then imported into other applications. Move and copy data between spreadsheets/worksheets Link data from one spreadsheet/worksheet to another Import/export spreadsheets into other applications Link spreadsheets to other applications Cut and paste spreadsheets into other applications
		SS.III.5 Participant will develop and implement a whole group lesson plan utilizing spreadsheets to collect, organize, and analyze information within a curricular topic. Determine appropriate spreadsheet uses Create a spreadsheet for data input Create appropriate graph or chart to analyze data; assist students in analysis/synthesis	SS.IV.2 Participants will develop and implement a lesson or project where students design/create spreadsheets for developmentally appropriate analytical / statistical purposes. Determine appropriate spreadsheet uses Create a spreadsheet for data input Create appropriate graph or chart to analyze data; assist students in analysis/synthesis

DATABASE OBJECTIVES

Level I	Level II	Level III	Level IV
DB.I.1 Participant will identify three database applications (e.g., search engines, library catalogs, etc.), what information they contain, and describe how they are used.
	DB.II.1 Participant will identify and define database components given an existing database. Use database vocabulary (i.e., tables, fields, etc.) Define database components
	DB.II.2 Participant will enter, edit, and print data, given an existing database Enter data Edit data Print database	DB.III.1 Participant will design a database for a specific purpose. Determine and develop categories of information Enter data Change, delete fields
	DB.II.3 Participant will organize data within a given database by sorting information using one criteria Sort alphabetically, numerically, chronologically, etc. Sort in ascending and descending order	DB.III.2 Participant will analyze and manipulate data in a database View, hide, sort, find and match records Share/merge database information with other applications Import data from other applications Add, change, and delete fields Design data tables (for Access users only) Define field types Copy, delete, and paste records
		DB.III.3 Participant will create, edit and print a report for a specific purpose. Create a n organized report/layout presenting selected data Create multiple reports within a given database for a variety of purposes. Edit a report/layout Print a report/layout
		DB.III.4 Participant will develop and implement a whole group lesson plan which utilizes a database to collect, organize, and analyze information. Determine appropriate database uses Create database for data input Create report to analyze data Assist students in analysis/synthesis	DB.IV.1 Participant will develop and implement a lesson or project where students create/manipulate databases for developmentally appropriate research purposes.   Determine appropriate database uses Create database for data input Create report to analyze data

INTERNET OBJECTIVES

Level I	Level II	Level III	Level IV
IN.I.1 Participant will have an understanding of Internet connectivity and structure. Understand hardware, browsers & networking to include LAN & WAN, & current ways of connecting to the Internet (school, home, business) Discuss historical & social implications Discuss Acceptable Use Policies for student and staff   IN.I.2 Participant will understand and describe the value of telecommunications to increase relevant and authentic learning opportunities. I.E., collaborative projects, real time data applications, access to experts
IN.I.3 Participant will locate web pages, given a list of web sites related to various curriculum topics. Start the navigation browser Input URLs to view web pages Identify domain types Use the navigation tool bar Navigate links	IN.II.1 Participant will locate and utilize web pages, given a list of web sites related to various curriculum topics. Print data Download graphics, video, sounds Save a web page Copy and paste web based text to another application Follow Fair Use Guidelines and Copyright procedures	IN.III.1 Participant will locate, utilize, and evaluate relevance and validity of web pages, given a list of web sites related to various curriculum topics. Determine validity of a web site to include: Author posted vs. juried (peer reviewed) website, broad over generalizations vs. specifics Find collaborating evidence Credentials of author listed Resources and references listed by author Biased or commercial interest Publication date of the information
IN.I.4 Participant will use simple procedures which support district Acceptable Use Policies Demonstrate personal security with respect to credit card numbers, names, phone numbers, addresses, email addresses, etc.. Use appropriate e-mail and "net etiquette"	IN.II.2 Participant will be able to develop classroom procedures as a way to provide safe internet use. Provide students with safe searching situations to include: Instructor generated bookmarks, arrange computers so that screens are visible to the instructor, use a teacher developed web page that links to predetermined sites, don’t allow free internet surfing, instructor screened email, web whacking
IN.I.5 Participant will research a relevant topic using a simple search technique. Perform category searches (i.e.. Yahoo links to categories) Perform simple keyword searches Perform searches via system software where provided (i.e. Sherlock)	IN.II.3 Participant will research a relevant topic using advanced search techniques. Perform advanced keyword searches using Boolean and logical operators (i.e., +, -, &, or, not, " " ) Use meta-search engines
		IN.III.2 Participant will utilize freeware/shareware programs available via the Internet. Follow copyright laws Understand the increased potential for conflicts when using non-commercial products Identify, locate freeware/shareware programs Scan for virus and install freeware/shareware programs	IN.IV.1 Participant will utilize plug-ins to enhance web browser capabilities. Identify, download and install required plug-in in appropriate location (i.e. Shockwave, RealAudio, etc.) Identify download and install players (i.e. Midi player, Movie player, MP3, etc.)
IN.I.6 Participant accesses and utilizes e-mail for professional/educational communications. Address and compose a message Send and receive e-mail messages Forward and reply to messages Save messages Delete unwanted messages Print messages	IN.II.4 Participant accesses and utilizes advanced features of an e-mail system for professional/educational communications. Create address books, Create distribution lists Attach and retrieve files Subscribe/ Unsubscribe to listservs Organize mail	IN.III.3 Participant will develop and implement a whole group lesson plan which utilizes telecommunication resources for research and collaboration with peers, experts, and other audiences to create an educational publication. Teacher will: Participate in video conferencing Cite resources in MLA or APA format Utilize E-mail / web mail Access web-based and/or collaborative projects Follow Fair Use Guidelines and Copyright procedures Create a product which integrates telecommunications into word processing, multi-media presentation, desktop publishing, and/or web page design	IN.IV.2 Participant will develop and implement a lesson or project in which students utilize telecommunication resources for research and collaboration with peers, experts, and other audiences to create an educational publication. Students will: Participate in video conferencing Cite resources in MLA or APA format Utilize E-mail / web mail Access web-based and/or collaborative projects Follow Fair Use Guidelines and Copyright procedures Create a product which integrates telecommunications into word processing, multi-media presentation, desktop publishing, and/or web page design
IN.I.7 Participant will access pre-existing bookmarks in a web browser.	IN.II.5 Participant will use a web browser to create and organize bookmarks/favorites of educational sites. Mark locations and access bookmarks/favorites Organize bookmarks/favorites by using separators Delete bookmarks	IN.III.4 Participant will utilize bookmarks for initial research and ongoing use during curriculum/lesson planning. Save bookmarks to disk, server, web, or email. Import and access bookmarks on another computer Save bookmarks as an HTML file, to be shared and accessed by other users Organize bookmarks in folders	IN.IV.3 Participant will develop and implement a lesson or a project where students utilize bookmarks for initial research and ongoing use during curriculum/lesson planning. Save bookmarks to disk, server, web, or email. Import and access bookmarks on another computer Save bookmarks as an HTML file, to be shared and accessed by other users Organize bookmarks in folders
		IN.III.5 Participant will develop and implement a whole group lesson plan in which they create an educational website (3 page minimum). Participant will: Create a storyboard for the web site, which includes elements of good screen design Use web publishing software to create a website, which includes: text, graphics, internal and external links, anchors, table and charts, bulleted or ordered lists, citations using MLA or APA format Follow Acceptable Use Policies, including copyright law	IN.IV.4 Participant will develop and implement a lesson or project where students create an educational website (3 page minimum). Student will: Create a storyboard for the web site, which includes elements of good screen design Use web publishing software to create a website, which includes: text, graphics, internal and external links, anchors, table and charts, bulleted or ordered lists, citations using MLA or APA format Follow Acceptable Use Policies, including copyright law

MULTIMEDIA OBJECTIVES

Level I	Level II	Level III	Level IV
MM.I.1 Participant will describe the value and purpose of using multimedia in the teaching/learning process
	MM.II.1 Participant will understand the planning procedures in multimedia production, given an existing presentation and its planning documents. Determine presentation objectives Analyze audience and setting Analyze flowchart Analyze storyboard Analyze screen design	MM.III.1 Participant will create a plan for a multimedia presentation on an educational topic (5 screens minimum). Determine objectives and audience Determine presentation medium & environment. Determine which multimedia software to use for the specific purpose Select linear or non-linear format Determine if use of multimedia elements are appropriate and meaningful Determine optimal design elements (font style, color, and size, background color or pattern and balanced layout, taking in account presentation format to be used, i.e. large screen projection, TV, computer monitor, etc.). Create flowchart Create storyboard
	MM.II.2 Participant will modify and enhance an existing multimedia presentation. Adhere to copyright laws Add text Add graphics Add buttons/hyperlinks Use tools to edit presentation	MM.III.2 Participant will produce a multimedia presentation on an educational topic (5 screens minimum). Create/delete slides/cards Rearranging slide/card order Copy/duplicate slides/cards Create transitions between slides/cards Create hyperlinks between slides/cards or other applications Add sound & video clips to add meaning if appropriate Create & print handouts from presentation/stack Import graphics from a variety of sources Import audio clips from a variety of sources Create bibliography, crediting sources of information & graphics Save presentation in self-contained format (file with player or "pack & go")     MM.III.3 Participant will evaluate effectiveness of a multimedia slideshow/stack when given a rubric.	MM.IV.1 Participant will produce a multimedia presentation on an educational topic using advanced development techniques. Use Master slide/background to customize entire presentation Create animations on slides/cards Create pop-up windows Embed objects into slide (i.e. tables, charts, spreadsheets, etc.) Save presentations/stacks to other media and platforms Connect and use presentation system to present Export presentation to other media (i.e. videotape, internet, CD-ROM etc.)               MM.IV.2 Participant will evaluate effectiveness of a multimedia presentation. Create an rubric to evaluate the product and presenter Present to peers Evaluate peer presentations
		MM.III.4 Participant will create digital media files and use in presentations. Scan photos and text Capture images with digital camera and download to computer Record audio clips Digitize video clips	MM.IV.3 Participant will manipulate digital media files and use in presentations. Manipulate digital images (i.e., crop, resize, filter, effects, color adjustments, etc.) Change digital images to other file formats (i.e. GIF, JPEG, PICT etc.) Use file compression utility Transfer digital sound clips from audio CD (copyright free only) Edit digital sound clips Edit video using computer video software Create own digital images using image manipulation software
		MM.III.5 Participant will develop and implement a multimedia presentation during the teaching and learning process.	MM.IV.4 Participant will develop and implement a lesson plan or project where students create a multimedia presentation on an educational topic.

Technology Self Evaluation

Project Venture is a federally funded Technology Innovations Challenge Grant awarded to Creighton, Kyrene, and Tempe Elementary School Districts, Tempe Union High School District, and the Maricopa County Small Schools Consortium.

The Project goals are:

• To increase the number of teachers trained to use technology for classroom instruction by 25% annually
• To develop and implement a curriculum that is in alignment with State Content Standards and includes the integration of technology
• To develop and implement an evaluation protocol that assists with project refinement and implementation, and that ensures sustainability and replication

The purposes of this self-assessment are to:

• Measure and report overall progress of Project Venture.
• Collect data to help individuals and schools plan staff development opportunities.

We appreciate your help collecting this data. The conditions of use of this data are:

• Data will not be used for the purposes of teacher or staff evaluation.
• Individual records will only be available to Project Venture personnel and will be kept confidential
• Only group summaries will be used for public reporting of progress.

• Creighton Elementary District

• Kyrene Elementary District

• Tempe Elementary District

• Tempe Union High School District

• Maricopa Small Schools District

• Visitor from outside project venture districts

If you have any questions or concerns, please contact your local project venture mentor or director.  If you are not sure who your mentor is, please refer to the list.

Thank you!

If you have questions or problems about using the self-assessment, email John Behrens at behrens@asu.edu