o 0iN @sdZddlmZddlZddlZddlZddlmZddlm Z ddl Z ddl Z ddlmZmZmZeeZejdejd Zid d d d d d ddddddddddddddddddddddddddddddiZed Zed!Zed"d#fed$d%fed&d'fed(d)fed*d+fed,d-fed.dfgZed/Zed0Zed1ZdEdFd7d8ZdGd=d>Z Gd?d@d@Z!GdAdBdBZ"GdCdDdDZ#dS)HzCore TTS engine and text processing components. This module contains the main Supertonic TTS engine, text processor, and supporting utilities for audio synthesis. ) annotationsN)Optional) normalize)AVAILABLE_LANGUAGES MAX_SPEED MIN_SPEEDuk[😀-🙏🌀-🗿🚀-🛿🜀-🝿🞀-🟿🠀-🣿🤀-🧿🨀-🩯🩰-🫿☀-⛿✀-➿🇦-🇿]+)flagsu–-u‑u—¯ _u“"u”u‘'u’´`[]|/#u→u←zt[\u0302\u0303\u0304\u0305\u0306\u0307\u0308\u030A\u030B\u030C\u0327\u0328\u0329\u032A\u032B\u032C\u032D\u032E\u032F]u[♥☆♡©\\]z ,,z \..z !!z \??z ;;z ::z 'z (["\'\`])\1+z\s+u+[.!?;:,'\"')\]}…。」』】〉》›»]$lengths np.ndarraymax_len Optional[int]returncCs@|p|}td|}|tj|ddktj}|dd|S)z Convert lengths to binary mask. Args: lengths: (B,) max_len: int Returns: mask: (B, 1, max_len) rr)Zaxis)maxnpZarangeZ expand_dimsastypefloat32Zreshape)rrZidsmaskr(I/home/kim/smarthome/.venv/lib/python3.10/site-packages/supertonic/core.pylength_to_maskUs r* wav_lengthsbase_chunk_sizeintchunk_compress_factorcCs$||}||d|}t|}|S)z)Generate mask for latent representations.rr*)r+r,r.Z latent_sizeZlatent_lengths latent_maskr(r(r)get_latent_maskfsr1c@seZdZdZd7ddZd8dd Zd9d d Zed9d dZd:ddZ d:ddZ d:ddZ d:ddZ d:ddZ d:ddZd:ddZd:ddZd;d!d"Zdd)d*Zd?d+d,Zd@d.d/ZdAd2d3Z #dtext zInvalid language: 'z'. Supported languages: z, <>z&z.Zdtype)r$arrayZuint16)r:rVZunicode_valuesr(r(r)_text_to_unicode_values%sz(UnicodeProcessor._text_to_unicode_valuestuple[bool, list[str]]cCs^t|}t}|D]}t||}||j}t|dkr"||q t|dktt|fS)aXValidate if text can be processed by the model. Args: text: Text to validate Returns: Tuple of (is_valid, unsupported_chars): - is_valid: True if text can be processed - unsupported_chars: List of unsupported characters (empty if valid) Example: ```python processor = UnicodeProcessor("unicode_indexer.json") is_valid, unsupported = processor.validate_text("Hello world") if not is_valid: print(f"Cannot process: {unsupported}") ``` r)rNr~rUrEupdatesortedr<)r:rVZ input_charsZunsupported_charsZ input_charZp_charsZus_charsr(r(r) validate_text)s   zUnicodeProcessor.validate_text text_list list[str]cCsd|}||S)zValidate a list of texts.rX)ryr)r:rZtext_catr(r(r)validate_text_listEs  z#UnicodeProcessor.validate_text_listtuple[np.ndarray, np.ndarray]c sfdd|D}tjdd|Dtjd}tjt||ftjd}t|D] \}}|}tjfdd|Dtjd||dt|f<q)|} || fS)aProcess a list of texts into model inputs. Args: text_list: List of text strings to process lang: Language code for multilingual support (en, ko, es, pt, fr). If None, no language tokens are added (v1 compatibility). Returns: Tuple of (text_ids, text_mask): - text_ids: Array of shape (batch_size, max_length) with unicode indices - text_mask: Array of shape (batch_size, 1, max_length) with attention mask csg|]}|qSr()r~)rtrvr:r(r)rYsz-UnicodeProcessor.__call__..cSrr()rE)rrVr(r(r)rZrrcsg|]}j|qSr()r7)rvalrTr(r)r_sN) r$rint64ZzerosrEr#rOrr) r:rrvZpreprocessed_textsrtext_idsirVZ unicode_valsrr(rr)__call__Js  zUnicodeProcessor.__call__)r3r4)r3r4r!r<)r!rM)rVr4r!r4)rVr4rvr4r!r4r5)rVr4rvr{r!r4)rrr!r)rVr4r!r)rVr4r!r)rrr!r)rrrvr{r!r)rD __module__ __qualname____doc__r;r6r8propertyrUr\rbrerirmrorrrurzr~rrrrrr(r(r(r)r2ps.             ,   r2c@seZdZdZdddZdS) StyleaVoice style representation for TTS synthesis. This class encapsulates the style vectors used to control the voice characteristics during speech synthesis. Args: style_ttl_onnx (numpy.ndarray): Style vector for the text-to-latent model style_dp_onnx (numpy.ndarray): Style vector for the duration predictor Attributes: ttl (numpy.ndarray): Text-to-latent style vector dp (numpy.ndarray): Duration predictor style vector style_ttl_onnxr style_dp_onnxcCsPt|tjstdt|jt|tjs tdt|j||_||_dS)Nz#style_ttl must be numpy array, got z"style_dp must be numpy array, got )rAr$Zndarray TypeErrorrCrDttldp)r:rrr(r(r)r;ts   zStyle.__init__N)rrrr)rDrrrr;r(r(r(r)resrc@s6eZdZdZd"d d Zd#ddZ   d$d%d d!ZdS)& SupertonicaCore TTS engine for Supertonic speech synthesis. This class orchestrates the entire text-to-speech pipeline, from text encoding through duration prediction and waveform generation. Args: cfgs: Model configuration dictionary text_processor: Unicode text processor instance dp_ort: Duration predictor ONNX session text_enc_ort: Text encoder ONNX session vector_est_ort: Vector estimator ONNX session vocoder_ort: Vocoder ONNX session Attributes: sample_rate (int): Audio sample rate in Hz base_chunk_size (int): Base chunk size for latent representation chunk_compress_factor (int): Compression factor for chunks ldim (int): Latent dimension size cfgsdicttext_processorr2dp_ortort.InferenceSession text_enc_ortvector_est_ort vocoder_ortc Cs$t|tstdt|jd|fd|fd|fd|ffD]\}}t|tjs3t|dt|jq||_||_||_ ||_ ||_ ||_ z|dd|_ |dd |_|d d |_|d d |_Wnty} ztd | td| d| d} ~ wwtd|j d|jddS)Nz-text_processor must be UnicodeProcessor, got rrrrz must be InferenceSession, got Zae sample_rater,rr. latent_dimzMissing required config key: z0Model configuration is incomplete. Missing key: z<. Please ensure you have downloaded the correct model files.z+Initialized Supertonic engine (sample_rate=zHz, latent_dim=))rAr2rrCrDortZInferenceSessionrrrrrrrr,r.ldimKeyErrorrFrIrBrG) r:rrrrrrnamesessionrLr(r(r)r;sJ    zSupertonic.__init__durationrr!rc Cst|}||j}||jtj}|j|j}||d|tj}|j |j}tj |||tj }t ||j|j} || }|| fS)Nr)rEr#rr%r$rr,r.Zint32rrandomZrandnr&r1) r:rbszZ wav_len_maxr+ chunk_sizeZ latent_lenr noisy_latentr0r(r(r)sample_noisy_latents  zSupertonic.sample_noisy_latent?Nrrstyler total_stepr-speedfloatrvr{c CsFt||jjdkrtdt|d|jjdd|tks#|tkr2tdtdtd|dd t|}|||\}}|jd ||j |d ^} } | |} |j d ||j|d ^} } | | \} } t j |g|t jd }t|D]}t j |g|t jd }|jd | | |j|| ||d^} } qt|jd d| i^}} || fS)aaSynthesize speech from text using the specified style. Args: text_list: List of text strings to synthesize style: Voice style object containing style vectors total_step: Number of diffusion steps (higher = better quality, slower) speed: Speech speed multiplier (0.7 = slower, 2.0 = faster) lang: Language code for multilingual support (en, ko, es, pt, fr). Required for supertonic-2 model. Supported languages: - "en": English - "ko": Korean - "es": Spanish - "pt": Portuguese - "fr": French Returns: Tuple of (waveform, duration): - waveform: Audio array of shape (batch_size, num_samples) - duration: Duration in seconds for each sample rzNumber of texts (z&) must match number of style vectors (z%). Please provide one style per text.zSpeed must be between z and z, got z.6fz4. Use values closer to 1.05 for more natural speech.N)rZstyle_dpr)r style_ttlrr)rZtext_embrrr0 current_steprZlatent)rErshaperBrrrrrunrrrr$rr&rangerr)r:rrrrrvrrrZdur_onnxr Z text_emb_onnxZxtr0Z total_step_npsteprZwavr(r(r)rsJ      zSupertonic.__call__) rrrr2rrrrrrrr)rrr!r)rrN) rrrrrr-rrrvr{r!r)rDrrrr;rrr(r(r(r)rs  0rr5)rrrr r!r)r+rr,r-r.r-r!r)$r __future__rr?loggingretypingr unicodedatarnumpyr$Z onnxruntimerconfigrrr getLoggerrDrFcompileUNICODErYr]rcrdrjrnrprsr*r1r2rrr(r(r(r)s                       v