Versions Compared

Old Version 5

changes.mady.by.user Samantha Lieberman

Saved on

compared with

New Version Current

changes.mady.by.user annelouise

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

If the PV Watts API goes down, all jobs in organizations using PV Watts Solar Production Methodology will be blocked.

...

How can I tell if PV Watts is down?

  1. Check the Logs in Datadog, if the response is anything but a 200 response, then it is either responding in error or timing out.

    1. If you’re not seeing any logs, ping Tim Marchese and ask him to check loggly.

  2. Locate the sentry error

  3. Check the organization writing in

    1. Do they have PV Watts switched on in the Feature settings?

  4. Confirm with Engineering:

    1. Create a BT and include the Sentry, the org impacted, and the job

    2. Engineering will confirm if there is an issue with the PV Watts API. If so, please follow the steps below…

  5. Immediately slack the #softwaresupport channel notifying all parties that PV Watts is down so they can kickoff the mass-switch

What is the workaround when PV Watts is down?

Good news, the workaround for when PV Watts is down is simple, we disable PV Watts on the customer’s account while logged in as Sighten Admin under the feature settings. Here’s /wiki/spaces/CON/pages/45613314 more on how to do this.

On order to expedite this and unblock all partners, we will bulk switch all orgs using PV Watts on to the Sighten production methodology. Here are the steps that need to happen after the outage is confirmed:

  1. Samantha Lieberman will create a duplicate of the 4/14/21 email announcement and queue it up for sending to all org admins in orgs that use PV Watts.

  2. Zachary Tannenbaum will pull a list of all orgs that use PV Watts and queue up the datafix that will actually switch the orgs and link it to the BT.

  3. Kira Gaza will pull an email list of the organization admins in the affected orgs and link it to the BT.

  4. Samantha Lieberman will upload the email list to Hubspot and send it out. After sending Samantha Lieberman will let Zachary Tannenbaum know to initiate the switch.

  5. After the switch is complete Zachary Tannenbaum will reply to the #softwaresupport channel to close the loop.

What do I do when a customer writes in during a PV Watts Outage?

If a customer writes in during a PV Watts outage, they’re likely blocked on the project phase.

  1. Check to see if the org has PV Watts turned on. (If they’re writing in AFTER the switch described above, and PV Watts is still on in their account, flag this for Samantha Lieberman but continue with the steps outlined below. If they have already been switched OFF of PV Watts due to the switch outlined above and they’re still having issues, see “What if the customer is saying they’re still blocked?” below)

  2. If they have PV Watts on, you can send them the following chat or email:

    1. Chat (this can be accessed in chat using the PV Watts out shortcut): The PVWatts API is currently down, causing your issues. As a workaround, we can switch your account to use the Sighten production methodology until PVWatts service is fixed. Please note this will impact all users in your account.

    2. Email: Hi [Name],
      The PVWatts API is currently down, causing your issues saving projects. As a workaround, we can switch your account to use the Sighten custom production methodology until PVWatts service is fixed. Please note this will impact all users in your account.
      I am so sorry about the inconvenience. Please let us know if we can make this switch for you and otherwise we will let you know as soon as the issue is fixed.
      Regards,
      [your name]

    3. NOTE: Please always ask permission to turn off PV Watts for a customer. Do not do it for them unless they give you express consent to do soIf the customer wants to stay on PV Watts, prior to us running the switch let #softwaresupport know ASAP so they can remove the org from the list.

  3. If the customer says you should switch them, first make the switch and then let the customer know they should be unblocked.

    1. Email:Hi [Name],
      This is to confirm that we have temporarily switched your organization off of PVWatts API. We will follow up here to let you know when it is back up and running.
      If you’re still having issues, please try creating a new project on a job. Existing projects will attempt to use the production methodology that was active when it was created. We apologize for the inconvenience this outage may have caused.
      Regards,
      [your name]

  4. Then, log what you did in this spreadsheet: https://docs.google.com/spreadsheets/d/1_BKiz9XWi8i1Oi5n8Uaf3ogaatVL0l2QaSZD5JD7xbs/edit#gid=545030185

  5. Next link the SS ticket to the BT ticket created above.

    1. if it was a chat, please send the transcript to Jira, create the SS ticket and then link it to the BT ticket.

  6. Follow up with the customer and let them know that we will notify them when they have been switched back. You can use the following email template:

    1. Hi [Name],

      PV Watts has been temporarily disabled for your organization. We will let you know when it is back up and running and you are switched back. Please let us know if you’re still experiencing issues. We truly apologize for the inconvenience.

      Best,
      [Your Name]

  7. When PV Watts is working again:

    1. Switch customers back to PV Watts

      1. Follow up with customers that were switched off with the following email:

        1. Hi [Name],
          Good morning. I wanted to let you know that the PV Watts issue has been resolved, and we have switched your organization back to using this methodology. Please let us know if you have any questions or need any additional assistance. We apologize for the inconvenience this may have caused.

          Have a great day!

          Best,

          [Your Name]

    2. For customers that were impacted but did not switch, follow up with the following email:

      1. Hi [Name],
        Good morning. I wanted to let you know that the PV Watts API is back up and running. Please let us know if you have any questions or need any additional assistance. We apologize for the inconvenience this may have caused.

        Have a great day!

        Best,

        [Your Name]

    3. Mark that you have followed up in this spreadsheet: https://docs.google.com/spreadsheets/d/1_BKiz9XWi8i1Oi5n8Uaf3ogaatVL0l2QaSZD5JD7xbs/edit#gid=0

...

What if they want more information about Sighten’s production methodology?

You can send them: How is solar production calculated in Sighten?

What if the customer is saying they’re still blocked?

If a customer has already been turned off of PV Watts, but they’re still having issues there are 2 possibilities:

  1. It might be an issue with invalidation, this should be your first guess if they say the phases are spinning How do I make the quote and usage phases stop spinning?

  2. Or if they are trying to recalculate on an existing project, then it is going to keep trying with the same calc method. So please instruct them that the best way to get unstuck is to start a new project from scratch.