how can I input a value that would evaluate to FALSE that is not a number?

Use ISNUMBER.

Syntax
ISNUMBER(value)

value - The value to be verified as a number.

*ISNUMBER returns TRUE if this is a number or a reference to a cell containing a numeric value and FALSE otherwise.

Short answer

1. FALSE and TRUE are Google Sheets keywords, they represent the respective boolean values. To use them as text, use ' as prefix, In the regular expression change False to FALSE.
2. The "Apply to range" start cell should be L2 instead of L1.

Explanation

Apply to range

The relative references in custom formatting formulas takes the start cell as the pivot. As the values FALSE/TRUE/Unsure start on row 2, on Apply to range instead of using L1 as the start cell, use L2.

Custom formula

The formula

=(sum(arrayformula(n(regexmatch($L2:$P2, "False")))) = 5)

returns #VALUE! and the following error description:

Error
Function REGEXMATCH parameter 1 expects text values. But 'FALSE' is a boolean and cannot be coerced to a text.

Note: To see the above error message in a Google Sheets spreadsheet, add the above formula to any cell not in the columns L-P

As REGEX is case sensitive the formula to use is:

 =(sum(arrayformula(n(regexmatch($L2:$P2, "FALSE")))) = 5)

An alternative formula to avoid the use of prefix is the following

 =ARRAYFORMULA(SUM(IFERROR(IF($L2:$P2,1,0),1))=0)