14.1

It is difficult to see what the impact of the HB-HTA process is.

• Ask decision-makers directly about the use of the HB-HTA report in their decisions and/or the adoption of its recommendations.
• Include in your internal procedures the compilation of both the HB-HTA reports and the documents related to the final decision on the adoption of the technology. The agreement of the conclusions of both documents should be monitored, analysed and reported yearly.

14.2

HB-HTA reports are not used by hospital decision-makers. Therefore, the impact of HB-HTA results on decision-making cannot be measured and sustained.

• Meet decision-makers together with the clinicians you work with on HB-HTA. Jointly, ask for feedback on the recommendations that your HB-HTA report provided.
• Send decision-makers follow-up messages on the status of the HT.
• Constantly improve your visibility, disseminate the results of your work internally.
• Strive for excellence in making reports. They are a means of prompting decision-makers to use your HB-HTA reports’ recommendations to inform decision-making.
• Ask your in-hospital customers to share their experience with the HB-HTA unit with peers/colleagues.

14.3

The impact of the HB-HTA report is not measured (usually due to the small size of the unit and the high burden of work).

Suggest that the responsibility for this action should be on the managerial level.

14.4

The use given to the results of the HB-HTA report is not known by the clinical staff.

Evaluate and report on the clinical benefits stemming from your HB-HTA reports to healthcare professionals.

14.5

The role and benefits of the HB-HTA unit are questioned.

• Evaluate the economic benefits arising from your HB-HTA unit’s performance to show the financial impact on the hospital of the unit’s work.
• Evaluate the in-hospital customers’ satisfaction with the HB-HTA unit’s performance.

14.6

There are not enough resources to follow up the implementation of the assessment results.

• Ask the clinicians to follow up the performance of adopted HTs (usually clinicians do this in order to gather data for scientific purposes, publishing etc.).
• Look for funds by contacting the manufacturer of the HT. Industry may be keen to fund the follow-up as they are interested in collecting real-world evidence (RWE) on HT performance in an environment other than clinical studies.
• Look for specific funding schemes aimed at implementation/follow-up/monitoring activities.
• If you are unable to follow up the HT introduced systematically, perform an annual qualitative check-up aimed at describing what happened to the HT after a positive/negative recommendation. Interview clinicians about the status of the implementation of the recommendation and, if it was positive, the status of a given HT using a short questionnaire addressing issues such as what the current use of the HT is and how many patients have been scheduled to use the HT per year.

14.7

Once a new technology is adopted, it is difficult to obtain the collaboration of applicant clinicians in collecting follow-up data.

• In the decision-making workflow, limit the funding for the new technology to the first two years. Then, base the on-going funding on follow-up data.
• Consider the possibility of having an automatic system that impedes the purchase of HTs not supported by follow-up data.

14.8

Self-reported follow-up data on the performance of HTs may be biased.

When possible, give priority to follow-up data from the hospital information system over self-reported follow-up data.

14.9

The HB-HTA unit has no member with financial measurement skills.

Ask for support from the accounting department in your hospital.

14.10

There is no system in place to undertake the follow-up of the financial impact of the adopted/rejected HTs.

Modelise financial outcomes with the data used in the HB-HTA report.

14.11

There is no system in place for measuring the productivity of the HB-HTA unit’s staff.

Create a system (e.g. logbook) to record all the activities performed by staff members as well as the number of hours invested in them.

14.12

The level of satisfaction of the HB-HTA customers (clinicians, hospital manager) is not measured/checked.

Perform a simple satisfaction survey asking your customers’ perceptions on, for example:

• satisfaction with the assessment process,
• the value of your HB-HTA unit,
• willingness to use your HB-HTA’s unit services again,
• willingness to recommend your HB-HTA unit’s services,
• views on meeting customers‘ expectations,
• the usefulness of the information provided by your HB-HTA reports.

Repeat the survey on a permanent basis (e.g. every 5 years).

14.13

Dissatisfied customers do not turn to the HB-HTA unit for help again.

At the end of the HB-HTA process, collect experiences from applicants and ask them about their satisfaction with regard to the process and its outcome.

Assure customers that corrective/improvement measures will be adopted when necessary.

14.14

There is no expertise or system in place to measure the motivation of the staff of the HB-HTA unit.

Ask the HR department of the hospital to help you on this matter.

14.15

Timely delivery of HB-HTA reports is not measured/checked.

• Ask your HB-HTA network for the key steps they follow for the timely delivery of HB-HTA reports and which indicators they use to measure it.
• List the key steps for developing an HB-HTA report and the most serious bottlenecks (crucial steps that need to be completed in order to move forward with subsequent steps of an assessment).
• Measure how many person-months are invested by your unit in the production of an HB-HTA report.
• As your HB-HTA unit evolves, introduce more specific measurements (e.g. person-months for bottlenecks and for each step of the assessment process).

14.16

The HB-HTA report does not meet timeliness expectations of hospital decision-makers.

• Identify the relevant stakeholders to collaborate with on a particular HT assessment.
• Inform them about the key steps of the HB-HTA process and agree upon a detailed schedule for each task they are in charge of. Let them know about the potential delay if the time plan is not followed.
• If there is a delay in the HB-HTA process, keep the stakeholders and the hospital decision-makers informed of the status of the project.

14.17

The customers are often not satisfied with the time frame of the evaluations.

Inform customers of the prioritisation criteria and make them aware of the fact that that timeliness depends on the number of HTs to evaluate compared with the human resources available, as well as on the responsiveness of the clinicians and other professionals of the hospital involved in answering questions and providing data.

14.18

There is a trade-off between timeliness and robustness of an HB-HTA report.

Perform a time-consuming comprehensive analysis only when the HT has a high level of complexity or a high budget impact.

14.19

There is no systematic approach in place to measure external impact.

• Systematically review websites, papers, abstracts from meetings, scientific databases, where your work could be published or mentioned.
• Register on research databases that alert you when your work is used (e.g. ResearchGate).
• Periodically, ask colleagues in the HB-HTA or HTA fields whether they use your work (or if they know of someone who is using or has used it).
• Publish parts or the whole content of the HB-HTA reports in a database that is traceable (i.e. downloads can be registered).

15.1

Long-term results and impact are not achieved, measured and sustained by the HB-HTA unit.

To date, this key element is still unmet even by highly developed HB-HTA units with long-standing performance or by national or regional HTA organisations.

In your effort to achieve excellence in HB-HTA performance, focus on doable and realistic developments and, over time, start to look for more challenging good practices to follow by addressing long-terms results, such as the impact of your HB-HTA unit on the overall hospital performance in terms of achievements of outcomes for patients and impact on society.

Suggest that the responsibility of this action should be placed on the managerial level.

15.2

The HB-HTA reports are not perceived as very useful at a societal level.

• Create lay language communication materials on the clinical benefits (improved quality and safety) of targeted use of new technologies.
• Create a broad network of organisations and communicate your HTA report results to stakeholders outside the hospital in a proper fashion.