Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions PanoramicData.Blazor.Demo/Pages/PDChatDocumentation.razor
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
<tr><td><code>Messages</code></td><td><code>List&lt;ChatMessage&gt;</code></td><td>Messages to display</td></tr>
<tr><td><code>OnSend</code></td><td><code>EventCallback&lt;string&gt;</code></td><td>Fired when user sends message</td></tr>
<tr><td><code>Icons</code></td><td><code>PDChatIcons</code></td><td>The CSS classes of the header and toolbar icons. Font Awesome by default. See <a href="#icons-and-theming">Icons and theming</a>.</td></tr>
<tr><td><code>VoiceModeShortcutKey</code></td><td><code>string?</code></td><td>A key code such as <code>"KeyM"</code>: Alt with that key, while focus is in the chat, cycles Voice Mode through Mic off and its modes. <code>null</code> (the default) offers no shortcut. See <a href="#input-toolbar">Input toolbar, voice and wake phrase</a>.</td></tr>
</tbody>
</table>
</section>
Expand Down Expand Up @@ -159,6 +160,12 @@
The microphone is one split control: the button starts and stops listening, and the select beside it picks
the mode. The button shows the mode as its icon (the HAL eye, a microphone, or an ear) and the state through
its colour and motion: listening, editing, thinking, speaking, dormant or paused. Its title says both in words.
While the microphone is off, a small badge (<code>Icons.MicOff</code>, a microphone with a slash) marks the
button; set it to an empty string to show none. The select shows Mic off while the microphone is off and the
running mode otherwise.
Set the <code>VoiceModeShortcutKey</code> parameter to a key code such as <code>"KeyM"</code> to let Alt with
that key, while focus is in the chat, cycle the select through Mic off and the modes; the select's tooltip
then names it. Without it there is no shortcut.
In <code>Hal</code> and <code>Wake</code>, clicking in the text box switches to manual: dictation keeps
streaming into the box, Send submits, and the next pause sends again. A message's
<code>SpokenMessage</code>, when set, is read aloud in place of the message, which is displayed unchanged; an
Expand Down
1 change: 1 addition & 0 deletions PanoramicData.Blazor.Demo/Shared/MainLayout.razor
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@
UserIconSelector="GetUserIcon"
PriorityIconSelector="GetPriorityIcon"
SoundSelector="GetSoundUrl"
VoiceModeShortcutKey="KeyM"
User="User" />
</ChatContent>
<ChildContent>
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,210 @@
using AwesomeAssertions;
using Bunit;
using Microsoft.AspNetCore.Components;
using Microsoft.AspNetCore.Components.Web;
using PanoramicData.Blazor.Interfaces;
using PanoramicData.Blazor.Models;

namespace PanoramicData.Blazor.Test;

/// <summary>
/// The voice controls' state: the picker shows what the microphone is doing, the voice button marks Mic off, and a host's
/// shortcut (here Alt+M) cycles the choices.
/// </summary>
public partial class PDChatTests
{
private static readonly KeyboardEventArgs _altM = new() { Code = "KeyM", Key = "m", AltKey = true };

/// <summary>Verifies that the picker reads Mic off after a load, whatever mode was last chosen, because nothing listens yet.</summary>
[Theory]
[InlineData(PDChatVoiceMode.Hal)]
[InlineData(PDChatVoiceMode.Dictate)]
[InlineData(PDChatVoiceMode.Wake)]
public void The_picker_reads_Mic_off_after_a_load(PDChatVoiceMode lastMode)
{
var service = WakeService();
((IChatService)service).VoiceMode = lastMode;

PickerValue(RenderChat(service)).Should().Be("MicOff");
}

/// <summary>Verifies that once the ear button resumes listening after Mic off, the picker reads the mode, not Mic off.</summary>
[Theory]
[InlineData(PDChatVoiceMode.Hal)]
[InlineData(PDChatVoiceMode.Dictate)]
[InlineData(PDChatVoiceMode.Wake)]
public async Task The_picker_reads_the_mode_when_the_ear_button_resumes_listening(PDChatVoiceMode mode)
{
SetUpVoiceModule();
var component = RenderChat(WakeService());
await component.Find(".pdchat-voice-mode-picker select").ChangeAsync(new ChangeEventArgs { Value = mode.ToString() });
await component.Find(".pdchat-voice-mode-picker select").ChangeAsync(new ChangeEventArgs { Value = "MicOff" });
PickerValue(component).Should().Be("MicOff");

await component.Find(".pdchat-voice-toggle").ClickAsync(new());

PickerValue(component).Should().Be(mode.ToString());
}

/// <summary>Verifies that the voice button shows a Mic off badge while the microphone is off, in every mode.</summary>
[Theory]
[InlineData(PDChatVoiceMode.Hal)]
[InlineData(PDChatVoiceMode.Dictate)]
[InlineData(PDChatVoiceMode.Wake)]
public void The_voice_button_shows_a_Mic_off_badge_while_the_microphone_is_off(PDChatVoiceMode mode)
{
var service = WakeService();
((IChatService)service).VoiceMode = mode;

var toggle = RenderChat(service).Find(".pdchat-voice-toggle");

IconClass(toggle.QuerySelector(".pdchat-voice-off-badge")!).Should().Be("fas fa-microphone-slash");
}

/// <summary>Verifies that the voice button carries no Mic off badge, and keeps its mode's glyph, while listening.</summary>
[Theory]
[InlineData(PDChatVoiceMode.Hal, ".pdchat-hal-eye")]
[InlineData(PDChatVoiceMode.Dictate, "i.fa-microphone")]
[InlineData(PDChatVoiceMode.Wake, "i.fa-assistive-listening-systems")]
public async Task The_voice_button_has_no_Mic_off_badge_while_listening(PDChatVoiceMode mode, string glyph)
{
SetUpVoiceModule();
var component = await RenderInModeAsync(WakeService(), mode);

var toggle = component.Find(".pdchat-voice-toggle");

toggle.QuerySelector(".pdchat-voice-off-badge").Should().BeNull();
toggle.QuerySelector(glyph).Should().NotBeNull();
}

/// <summary>Verifies that a host's Mic off icon replaces the default badge icon.</summary>
[Fact]
public void A_host_s_Mic_off_icon_replaces_the_default()
{
var component = RenderChat(VoiceService(), p => p.Add(x => x.Icons, new PDChatIcons { MicOff = "bi bi-mic-mute" }));

IconClass(component.Find(".pdchat-voice-off-badge")).Should().Be("bi bi-mic-mute");
}

/// <summary>Verifies that a host hides the Mic off badge by giving it no icon.</summary>
[Fact]
public void An_empty_Mic_off_icon_shows_no_badge()
=> RenderChat(VoiceService(), p => p.Add(x => x.Icons, new PDChatIcons { MicOff = string.Empty }))
.FindAll(".pdchat-voice-off-badge").Should().BeEmpty();

/// <summary>Verifies that Alt+M in the text box cycles Mic off, HAL, Dictate and back to Mic off.</summary>
[Fact]
public async Task Alt_M_cycles_Mic_off_HAL_and_Dictate()
{
var module = SetUpVoiceModule();
var component = RenderWithShortcut(VoiceService());

var seen = new List<string?>();
for (var press = 0; press < 3; press++)
{
await component.Find("textarea").KeyDownAsync(_altM);
seen.Add(PickerValue(component));
}

seen.Should().Equal("Hal", "Dictate", "MicOff");
component.Instance.VoiceState.Should().Be(PDChatVoiceState.Off);
module.Invocations["stop"].Should().ContainSingle();
}

/// <summary>Verifies that Alt+M includes the wake phrase mode when the host offers it.</summary>
[Fact]
public async Task Alt_M_includes_the_wake_phrase_mode_when_offered()
{
SetUpVoiceModule();
var component = RenderWithShortcut(WakeService());

var seen = new List<string?>();
for (var press = 0; press < 4; press++)
{
await component.Find("textarea").KeyDownAsync(_altM);
seen.Add(PickerValue(component));
}

seen.Should().Equal("Hal", "Dictate", "Wake", "MicOff");
}

/// <summary>Verifies that only Alt with the host's key cycles, so AltGr (Ctrl and Alt) and other keys type as usual.</summary>
[Theory]
[InlineData("KeyM", false, false, false, false)]
[InlineData("KeyM", true, true, false, false)]
[InlineData("KeyM", true, false, true, false)]
[InlineData("KeyM", true, false, false, true)]
[InlineData("KeyN", true, false, false, false)]
public async Task Other_keys_do_not_cycle(string code, bool alt, bool ctrl, bool shift, bool meta)
{
var module = SetUpVoiceModule();
var component = RenderWithShortcut(VoiceService());

await component.Find("textarea").KeyDownAsync(new KeyboardEventArgs { Code = code, AltKey = alt, CtrlKey = ctrl, ShiftKey = shift, MetaKey = meta });

component.Instance.VoiceState.Should().Be(PDChatVoiceState.Off);
module.Invocations.Should().BeEmpty();
}

/// <summary>Verifies that the shortcut is off unless the host names its key, so a host's own Alt shortcuts are left alone.</summary>
[Fact]
public async Task Alt_M_does_nothing_unless_the_host_names_the_key()
{
var module = SetUpVoiceModule();
var component = RenderChat(VoiceService());

await component.Find("textarea").KeyDownAsync(_altM);

component.Instance.VoiceState.Should().Be(PDChatVoiceState.Off);
module.Invocations.Should().BeEmpty();
}

/// <summary>Verifies that cycling leaves read aloud as the user set it.</summary>
[Theory]
[InlineData(true)]
[InlineData(false)]
public async Task Alt_M_leaves_read_aloud_as_set(bool readAloud)
{
SetUpVoiceModule();
var service = VoiceService(readAloud: readAloud);
var component = RenderWithShortcut(service);

for (var press = 0; press < 3; press++)
{
await component.Find("textarea").KeyDownAsync(_altM);
}

((IChatService)service).IsReadAloudEnabled.Should().Be(readAloud);
}

/// <summary>Verifies that Alt+M does nothing while the chat is minimised, where the voice controls are not shown.</summary>
[Fact]
public async Task Alt_M_does_nothing_while_the_chat_is_minimised()
{
var module = SetUpVoiceModule();
var service = VoiceService();
service.DockMode = PDChatDockMode.Minimized;
var component = RenderWithShortcut(service);

await component.Find(".pdchat-toggle-collapsed").KeyDownAsync(_altM);

component.Instance.VoiceState.Should().Be(PDChatVoiceState.Off);
module.Invocations.Should().BeEmpty();
}

/// <summary>Verifies that the picker's tooltip names the shortcut when the host has one, and is unchanged when not.</summary>
[Fact]
public void The_picker_s_tooltip_names_the_shortcut_when_there_is_one()
{
RenderWithShortcut(VoiceService()).Find(".pdchat-voice-mode-picker select").GetAttribute("title")
.Should().Be("How Voice Mode listens. Alt+M cycles the choices.");
RenderChat(VoiceService()).Find(".pdchat-voice-mode-picker select").GetAttribute("title")
.Should().Be("How Voice Mode listens");
}

private IRenderedComponent<PDChat> RenderWithShortcut(FakeChatService service)
=> RenderChat(service, p => p.Add(x => x.VoiceModeShortcutKey, "KeyM"));

private static string? PickerValue(IRenderedComponent<PDChat> component)
=> component.Find(".pdchat-voice-mode-picker select").GetAttribute("value");
}
3 changes: 3 additions & 0 deletions PanoramicData.Blazor/Models/PDChatIcons.cs
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,9 @@ public sealed record PDChatIcons
/// <summary>Gets the badge on the voice button while dictation is paused.</summary>
public string VoicePaused { get; init; } = "fas fa-pause";

/// <summary>Gets the badge on the voice button while the microphone is off. Empty shows no badge.</summary>
public string MicOff { get; init; } = "fas fa-microphone-slash";

/// <summary>Gets the attach images button's icon.</summary>
public string Attach { get; init; } = "fas fa-paperclip";

Expand Down
39 changes: 39 additions & 0 deletions PanoramicData.Blazor/PDChat.Voice.cs
Original file line number Diff line number Diff line change
Expand Up @@ -185,6 +185,45 @@ private async Task StartVoiceModeAsync()

private const string MicOffValue = "MicOff";

// Set as the select's value: per-option selected attributes stop moving the selection once the user has chosen.
private string VoicePickerValue => IsVoiceModeOn ? VoiceMode.ToString() : MicOffValue;

private string VoicePickerTitle => VoiceModeShortcutKey is { Length: > 0 } key
? $"How Voice Mode listens. Alt+{ShortcutKeyName(key)} cycles the choices."
: "How Voice Mode listens";

// KeyboardEvent.code names the physical key, so "KeyM" is shown as M and "Digit5" as 5.
private static string ShortcutKeyName(string code)
=> code.StartsWith("Key", StringComparison.Ordinal) ? code[3..]
: code.StartsWith("Digit", StringComparison.Ordinal) ? code[5..]
: code;

private async Task OnChatKeyDownAsync(KeyboardEventArgs args)
{
if (!IsVoiceModeShortcut(args) || !IsVoiceModeOffered || ChatService.DockMode == PDChatDockMode.Minimized)
{
return;
}

await OnVoiceModeSelectedAsync(new ChangeEventArgs { Value = NextVoiceChoice() });
}

// Alt alone: Ctrl with Alt is AltGr on many layouts, which types characters.
private bool IsVoiceModeShortcut(KeyboardEventArgs args)
=> VoiceModeShortcutKey is { Length: > 0 } key && args.Code == key
&& args.AltKey && !args.CtrlKey && !args.ShiftKey && !args.MetaKey;

private string NextVoiceChoice()
{
List<string> choices = [MicOffValue, nameof(PDChatVoiceMode.Hal), nameof(PDChatVoiceMode.Dictate)];
if (IsWakeModeOffered)
{
choices.Add(nameof(PDChatVoiceMode.Wake));
}

return choices[(choices.IndexOf(VoicePickerValue) + 1) % choices.Count];
}

private async Task OnVoiceModeSelectedAsync(ChangeEventArgs args)
{
if (args.Value?.ToString() == MicOffValue)
Expand Down
17 changes: 11 additions & 6 deletions PanoramicData.Blazor/PDChat.razor
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@
@* Issue #106: one cascade for the whole component, rather than threading a callback through
PDMessages three times - once per dock mode - and PDMessage besides. *@
<CascadingValue Value="FormContext" IsFixed="true">
<div class="pdchat-container @GetDockModeClasses() @( ChatService.DockMode != PDChatDockMode.Minimized ? "open" : "" )">
<div class="pdchat-container @GetDockModeClasses() @( ChatService.DockMode != PDChatDockMode.Minimized ? "open" : "" )"
@onkeydown="OnChatKeyDownAsync">
@switch (ChatService.DockMode)
{
case PDChatDockMode.Minimized:
Expand Down Expand Up @@ -390,13 +391,13 @@
aria-label="@VoiceButtonTitle"
aria-pressed="@(IsVoiceModeOn ? "true" : "false")">@VoiceButtonGlyph</button>
<label class="pdchat-toolbar-picker pdchat-voice-mode-picker">
<select title="How Voice Mode listens" aria-label="Voice mode" @onchange="OnVoiceModeSelectedAsync">
<option value="@MicOffValue" title="The microphone is off" selected="@(!IsVoiceModeOn)">Mic off</option>
<option value="@PDChatVoiceMode.Hal" title="Listens at once and sends when you pause" selected="@(IsVoiceModeOn && VoiceMode == PDChatVoiceMode.Hal)">HAL</option>
<option value="@PDChatVoiceMode.Dictate" title="Listens while the text box has focus; you press Send" selected="@(IsVoiceModeOn && VoiceMode == PDChatVoiceMode.Dictate)">Dictate</option>
<select title="@VoicePickerTitle" aria-label="Voice mode" value="@VoicePickerValue" @onchange="OnVoiceModeSelectedAsync">
<option value="@MicOffValue" title="The microphone is off">Mic off</option>
<option value="@PDChatVoiceMode.Hal" title="Listens at once and sends when you pause">HAL</option>
<option value="@PDChatVoiceMode.Dictate" title="Listens while the text box has focus; you press Send">Dictate</option>
@if (IsWakeModeOffered)
{
<option value="@PDChatVoiceMode.Wake" title="Waits for the wake phrase, then sends when you pause" selected="@(IsVoiceModeOn && VoiceMode == PDChatVoiceMode.Wake)">@FirstWakePhrase</option>
<option value="@PDChatVoiceMode.Wake" title="Waits for the wake phrase, then sends when you pause">@FirstWakePhrase</option>
}
</select>
</label>
Expand Down Expand Up @@ -425,5 +426,9 @@
<span class="pdchat-voice-paused-badge" aria-hidden="true"><i class="@Icons.VoicePaused"></i></span>
}
}
@if (VoiceState == PDChatVoiceState.Off && !string.IsNullOrEmpty(Icons.MicOff))
{
<span class="pdchat-voice-off-badge" aria-hidden="true"><i class="@Icons.MicOff"></i></span>
}
</text>;
}
8 changes: 8 additions & 0 deletions PanoramicData.Blazor/PDChat.razor.cs
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,14 @@ public partial class PDChat : JSModuleComponentBase
[Parameter]
public PDChatIcons Icons { get; set; } = new();

/// <summary>
/// Gets or sets the key that, pressed with Alt while focus is in the chat, cycles Voice Mode through Mic off and its
/// modes. A KeyboardEvent.code such as "KeyM", so it is the same physical key on every layout. Null, the default,
/// offers no shortcut.
/// </summary>
[Parameter]
public string? VoiceModeShortcutKey { get; set; }

/// <summary>
/// A function to select a user icon for a given message.
/// </summary>
Expand Down
3 changes: 2 additions & 1 deletion PanoramicData.Blazor/PDChat.razor.css
Original file line number Diff line number Diff line change
Expand Up @@ -770,7 +770,8 @@ body:has(.pdchat-container.dock-fullscreen) {
}
}

.pdchat-voice-paused-badge {
.pdchat-voice-paused-badge,
.pdchat-voice-off-badge {
position: absolute;
right: 3px;
bottom: 3px;
Expand Down
Loading
Loading