iSQI ctfl-pt practice test
ISTQB Certified Tester Foundation Level - Performance Testing Exam
Question 1
You have been asked to test the performance of a software and system update to a North American
mobile trivia quiz game application. The quiz app is available on major app stores serving English-
speaking users. The number of concurrent users planned for is 5 million Winning quiz participants
are promised a share of an allocated cash prize Once participants start playing, their quiz sessions
must not be lost especially if all previous questions were answered correctly. The application owner
wants to ensure that 99% of quiz sessions are maintained (up to 4 95 million sessions) until the last
answer has been successfully recorded
The performance test team is assembling criteria for tool selection. They have identified the
following criteria
Must be able to reliably generate 5 million concurrent virtual quiz participants from multiple data
centers with centralized control
Must be compatible with REST based web services and have tight integration with RESTful APIs'
OData and OpenAPI.
Must be able to support nested transaction timers, provide centralized performance data collection
and reporting, and integrate with platform monitoring agents used by the Operations team.
Which of the following criterion would best complete the provided set of criteria?
- A. Must protect undesirable performance results from being viewed from unauthorized users
- B. Must provide shell checking that checks spelling within test script comments
- C. Must be supported with documentation, training materials, books from known publishers, and an IDE that behaves similarly to popular in-house IDEs.
- D. Must have technical and software maintenance support for product versions two major revisions behind current revision
Answer:
C
Question 2
You have been asked to test the performance of a software and system update to a North American
mobile trivia quiz game application The quiz app is available on major app stores serving English-
speaking users. The number of concurrent users planned for is 5 million Winning quiz participants
are promised a share of an allocated cash prize Once participants start playing, their quiz sessions
must not be lost especially if all previous questions were answered correctly. The application owner
wants to ensure that 99% of quiz sessions are maintained (up to 4.95 million sessions) until the last
answer has been successfully recorded.
You performed 100 test runs that provided 5 million virtual concurrent users. You have
measurements of participant concurrency at the last question for each test run In your analysis, you
order all the concurrency test results by measured concurrency value from best to worst. You
determine 85%. 90%. 95% and 100% values by selecting the 85th. 90th. 95th and 100th (last and
worst) measured values by counting list positions starting from the best value going towards the
worst until you reach the desired list position (ex. 85). Keep in mind, the results in lower list positions
are as good or better than the results chosen for reporting.
After analyzing the results of the performance tests, you present the following results
85% 4.92 million valid concurrent users
90% 4.90 million valid concurrent users
95%: 4.0 million valid concurrent users
100%. 3.8 million valid concurrent users
Based on these results, what should you report to the stakeholders?
- A. The performance tests clearly have failed to produce a realistic and compatible load. The results should confirm the test oracle in that the 99% of five million participants concurrency objective has been achieved. If the performance test plan had worked correctly, the reported values should have been 85% 4.95, 90% 4.95, 95% 4 95 and 100% 4.95 million valid concurrent users.
- B. These results clearly show that the system update successfully supports 99% of five million participants
- C. Four samples from the first fifteen ordered test results are presented. The system update performed much worse than what is presented here. The system update has failed to meet the 99% of five million participants concurrency objective.
- D. If we pick 85 out of 100 test runs as the minimum level of reliability, the system update does not reliability meet the 99% of five million participants concurrency target, However not all results are as bad as these
Answer:
A
Question 3
You have been asked to test the performance of a software and system update to a North American
mobile trivia quiz game application. The quiz app is available on major app stores serving English-
speaking users. The number of concurrent users planned for is 5 million Winning quiz participants
are promised a share of an allocated cash prize Once participants start playing, their quiz sessions
must not be lost especially if all previous questions were answered correctly. The application owner
wants to ensure that 99% of quiz sessions are maintained (up to 4.95 million sessions) until the last
answer has been successfully recorded
In order to avoid cooperation among quiz participants, the answers to the multiple choice questions
are listed randomly Confidence in the game's fairness requires that a registered user originate from
only one quiz client device Every user must respond over the same short window of time in order to
avoid external assistance and learning the official answer before responding. Only participants who
respond correctly may answer the next question Any activity that appears to violate expected
behavior of a rule-following participant will cause the client session to be rejected.
You are developing test script code that handles quiz question responses. To do this you need to
ensure that the script emulates the production quiz app client's logic in terms of rule following and
client-backend interface specification.
Based on the question description provided, which is one test script behavior you would implement
in order to ensure the test script successfully delivers the operational and load profile of a quiz
participant?
- A. Recognize whether the previous question response was correct
- B. Set the device identifier as a constant to ensure all virtual user instances use the set value
- C. Randomly select the maximum think time limit used to delay question responses.
- D. Use the date-time clock in order to submit question responses on a synchronized schedule with each instance
Answer:
B
Question 4
You have been asked to test the performance of a software and system update to a North American
mobile trivia quiz game application. The quiz app is available on major app stores serving English-
speaking users. The number of concurrent users planned for is 5 million Winning quiz participants
are promised a share of an allocated cash prize Once participants start playing, their quiz sessions
must not be lost especially if all previous questions were answered correctly. The application owner
wants to ensure that 99% of quiz sessions are maintained (up to 4 95 million sessions) until the last
answer has been successfully recorded
In order to avoid cooperation among quiz participants, the answers to the multiple choice questions
are listed randomly Confidence in the game's fairness requires that a registered user originate from
only one quiz client device Every user must respond over the same short window of time in order to
avoid external assistance and learning the official answer before responding. Only participants who
respond correctly may answer the next question Any activity that appears to violate expected
behavior of a rule-following participant will cause the client session to be rejected.
You are currently preparing test scripts to function as a quiz participant.
What input data type is most critical for the test script to incorporate to ensure that the concurrency
objective will be properly tested?
- A. In order to property test the concurrency objective, the test script must answer each trivia question with the same answer as every other virtual user
- B. In order to property test the concurrency objective, the test script must provide a random password to the application's login system
- C. In order to properly test the concurrency objective, the test script must present a unique user name to the application's login system
- D. In order to properly test the concurrency objectives the test script must supply the system provided unique session ID with each answer provided
Answer:
B
Question 5
You have been asked to test the performance of a software and system update to a North American
mobile trivia quiz game application. The quiz app is available on major app stores serving English-
speaking users. The number of concurrent users planned for is 5 million. Winning quiz participants
are promised a share of an allocated cash prize Once participants start playing, their quiz sessions
must not be lost especially if all previous questions were answered correctly. The application owner
wants to ensure that 99% of quiz sessions are maintained (up to 4 95 million sessions) until the last
answer has been successfully recorded.
An essential operational profile you identified is the quiz participant. Your operational profile
discovery efforts have yielded a number of profile attributes. A quiz participant will answer twelve
questions if they manage to answer the first eleven correctly Every user is made aware of a new
question nearly simultaneously. The quiz user has limited time to respond using a mobile application.
They are unable to respond to subsequent questions after they incorrectly answer a question.
However, the quiz participant continues to see new questions even though they may have responded
incorrectly to a previous question If they manage to answer all twelve questions correctly, the quiz
application notifies them that they have tentatively won (subject to formal verification) an equal
share of a cash prize of an amount set for the particular quiz event In addition to this individualized
interaction, a broadcast video stream is presented in the quiz app providing entertaining
commentary quiz event coordination, event status and answer explanations.
The average quiz event duration when measured from when the first question is presented to the
last answer is reviewed is fifteen minutes. Over the fifteen minutes, twelve questions are presented
to quiz participants. After making the following assumptions:
1. Questions must be responded to within a four second window;
2. Question responses will be sent evenly across this window of time (resulting in an average
responder time of two seconds);
3. The designed maximum number of concurrent users will attempt the first question;
4. There will be no functional issues when handling the load.
5. The average time from response reception to database commitment is 1000 milliseconds under
maximum concurrent usage.
What is the approximate maximum average system throughput if that assumptions hold?
- A. 1.667 million transactions/second
- B. 1 million transactions/second
- C. 1 million transactions/millisecond
- D. 200.000 transactions/second
Answer:
B
Question 6
You have been asked to test the performance of a software and system update to a North American
mobile trivia quiz game application. The quiz app is available on major app stores serving English-
speaking users. The number of concurrent users planned for is 5 million Winning quiz participants
are promised a share of an allocated cash prize Once participants start playing, their quiz sessions
must not be lost especially if all previous questions were answered correctly. The application owner
wants to ensure that 99% of quiz sessions are maintained (up to 4 95 million sessions) until the last
answer has been successfully recorded
An essential operational profile you identified is the quiz participant Your operational profile
discovery efforts have yielded a number of profile attributes / quiz participant will answer twelve
questions if they manage to answer the first eleven correctly Every user is made aware of a new
question nearly simultaneously. The quiz user has limited time to respond using a mobile application.
They are unable to respond to subsequent questions after they incorrectly answer a question.
However, the quiz participant continues to see new questions even though they may have responded
incorrectly to a previous question, if they manage to answer all twelve questions correctly, the quiz
application notifies them that they have tentatively won (subject to formal verification) an equal
share of a cash prize of an amount set for the particular quiz event In addition to this individualized
interaction, a broadcast video stream is presented in the quiz app providing entertaining
commentary, quiz event coordination event status and answer explanations.
The average quiz event duration when measured from when the first question is presented to the
last answer is reviewed is fifteen minutes Over the fifteen minutes, twelve questions are presented
to quiz participants.
Which of the following Is a proper load profile for the quiz participants?
- A. 240.000,000 question responses per hour
- B. Steady ramp-up of quiz response transactions from one million until the design maximum is reached within the first three questions. The maximum is sustained until question twelve
- C. Stepped ramp-up of quiz response transactions from one million increasing by a one million after each question until the design maximum is reached. The minimum is sustained until question twelve.
- D. Maximum number of planned concurrent quiz responses will occur at the first question with a stepped ramp-down after each question. The size of each step and the final number of nearly simultaneous quiz responses are random values within historical ranges observed over the last three
Answer:
C
Question 7
You have been asked to test the performance of a software and system update to a North American
mobile trivia quiz game application. The quiz app is available on major app stores serving English-
speaking users. The number of concurrent users planned for is 5 million Winning quiz participants
are promised a share of an allocated cash prize Once participants start playing, their quiz sessions
must not be lost especially if all previous questions were answered correctly. The application owner
wants to ensure that 99% of quiz sessions are maintained (up to 4.95 million sessions) until the last
answer has been successfully recorded.
An essential operational profile you identified is the quiz participant Your operational profile
discovery efforts have yielded a number of profile attributes A quiz participant will answer twelve
questions if they manage to answer the first eleven correctly Every user is made aware of a new
question nearly simultaneously. The quiz user has limited time to respond using a mobile application.
They are unable to respond to subsequent questions after they incorrectly answer a question.
However, the quiz participant continues to see new questions even though they may have responded
incorrectly to a previous question. If they manage to answer all twelve questions correctly, the quiz
application notifies them that they have tentatively won (subject to formal verification) an equal
share of a cash prize of an amount set for the particular quiz event In addition to this individualized
interaction, a broadcast video stream is presented in the quiz app providing entertaining
commentary, quiz event coordination event status and answer explanations.
Given this information, what is missing to construct the operational profile for the quiz participant?
- A. The network bandwidth requirements of the video stream intermissions between questions is unknown. The network bandwidth requirements of the video stream intermissions between questions is unknown
- B. No survey results from past quiz winners that explore how they won No survey results from past quiz winners that explore how they won.
- C. The quiz response time limit is not specified. The quiz response time limit is not specified
- D. The design maximum number of users within quiz participant user class is unknown. The design maximum number of users within quiz participant user class is unknown
Answer:
A
Question 8
You have been asked to test the performance of a software and system update to a North American
mobile trivia quiz game application. The quiz app is available on major app stores serving English-
speaking users. The number of concurrent users planned for is 5 million. Winning quiz participants
are promised a share of an allocated cash prize Once participants start playing, their quiz sessions
must not be lost especially if all previous questions were answered correctly. The application owner
wants to ensure that 99% of quiz sessions are maintained (up to 4.95 million sessions) until the last
answer has been successfully recorded
Before your team begins testing, the team needs to present the performance test plan to technical
stakeholders
Which of the following sets of plan attributes would you recommend be included in this
presentation?
i. Sales has indicated that quiz events for Sponsor A offering $65,000. Sponsor B offering $45,500.
Sponsor C offering $25,000
will be taking place during the first week of performance testing.
ii. Simultaneous session tests are focused on the backend. and are not end-to-end. Variability in user
behavior, mobile devices and mobile communications introduces session drop opportunities outside
application control. Using load generators located
in 5 high availability data centers in North America will ensure session drops are most likely caused
by application issues, thus allowing us to isolate the root cause of any concurrency issues that arise
iii. Test scripts based on replaying recorded communication will take 30 hours each to write and test
due to the extensive parameterization required to
respond to quiz questions Development has agreed to share client code that can be scripted to
authenticate and respond to quiz questions
automatically. The fully programmed script will likely take 18 hours each to write
and test Fully programmed scripts will utilize the identical communications stack of the actual client
avoiding missed communication nuances.
iv The U S Federal Trade Commission has recently sued SmartyBogusFauxApp and OutSmartMe
example com over promises of prizes and unreliability
v The upgrade being tested was paid for by the latest round of venture funding Supporting 5 million
users is requirement that was a condition of their funding.
- A. i. ii include iii. iv, v exclude
- B. ii, iii include i, iv, v exclude
- C. ii, v include i, ii, iv exclude
- D. iii. iv, v include i, ii exclude
Answer:
B
Question 9
You have been asked to test the performance of a software and system update to a North American
mobile trivia quiz game application. The quiz app is available on major app stores serving English-
speaking users. The number of concurrent users planned for is 5 million Winning quiz participants
are promised a share of an allocated cash prize. Once participants start playing, their quiz sessions
must not be lost especially if all previous questions were answered correctly. The application owner
wants to ensure that 99% of quiz sessions are maintained (up to 4 95 million sessions) until the last
answer has been successfully recorded.
Before your team begins testing, the team needs to present the performance test plan to business
stakeholders
Which of the following sets of plan attributes would you recommend be part of this presentation?
- A. During the actual quiz, users logon gradually but play simultaneously. The load injection of quiz responses will not ramp up. therefore nearly 5 million quiz responses will need to be generated in a 4 second time frame with no opportunity to reach steady state. Test plans have been discussed with Operations Development and Data Center representatives and we believe the tests will tax the quiz application consistent with actual use. The JMeter monitoring system has been tuned however, with 5 million service requests over 4 seconds there is concern that the services queue measurements may be off by 25%
- B. There is a less then10% chance of missing a scheduled quiz session due to performance test activities, which will likely cost the company $50,000 in lost sponsor revenue and brand damage To avoid this risk, we are scheduling Operations and Development representatives to be available during test execution We will require that the RAID 5+1 LUNs associated with the database server cluster be broken at 22 30 and the RAID 5+1 LUNs restored, ensuring the offline set mirrors back to the RAID 5 set that was under test, at 03:00
- C. Since the client uses an asynchronous REST web services based sessions back to the QUZRAC services cluster, the load profile will consist of virtual users running programmed load scripts based in C# that will randomly select the correct answer following a normal distribution thus modeling the typical success distribution among users The strict 4 second cap on user responses will translate into think times that are randomly selected between 1/2 second and just under 4 seconds using a uniform distribution
- D. If the quiz application drops more than 1% of quiz sessions, users will likely write bad reviews on app stores and other gaming sites Performance tests will verify that system can support at least 4 95 million users Planned tests, which include the simultaneous sessions test, will cost $350,000 Since only two quizzes are scheduled a day, performance tests on the production system during the "down' times should be representative of what users experience. There is a 10% chance that the production system will not restored in time for a quiz.
Answer:
C
Question 10
You have been asked to test the performance of a software and system update to a North American
mobile trivia quiz game application. The quiz app is available on major app stores serving English-
speaking users. The number of concurrent users planned for is 5 million Winning quiz participants
are promised a share of an allocated cash prize Once participants start playing, their quiz sessions
must not be lost especially if all previous questions were answered correctly. The application owner
wants to ensure that 99% of quiz sessions are maintained (up to 4.95 million sessions) until the last
answer has been successfully recorded.
A coworker has located various test plan elements under the standard PTP sections in a draft You
have been asked to analyze the quality of the draft. The complete PTP draft cannot be provided here
A content sampling of the PTP is provided.
Based on the content sample below, what is the quality of the draft document when only considering
the alignment of content under standard PTP sections?
- A. 0 misalignments, good
- B. 1 misalignments, good
- C. 2 misalignments good
- D. 3 misalignments needs improvement
Answer:
B