o 0i9&@sdZddlZddlmZmZmZddlZddlmZm Z ddl m Z ddl m Z mZmZmZmZmZmZedd Zed d Zeeejd d fZeeeejd d fZeeeejd d fZd eddfddZejdddgddejddfdede dededdf ddZejddgddddddddddd ed!eeejd"d fd#eeejd$d fd%eeejd&d fd'eeejd(d fd)eeejd*d fd+eeejd,d fd-eeejd.d fded/eeeejd0d fded1ee ejd2d fd3ee ejd4d fd5eee ejd6d fd7eee ejd8d fd9eee!ejd:d fddf"d;dZ"ejddd?Z# dddeddfd@dAZ$ejddBgde$ejdCdDdEe$ej%edFdGejdHgd  ded edededdfdIdJZ&ejdKgd              dfd eded!eeeejd"d fd%eeeejd&d fd'eeeejd(d fd)eeeejd*d fd#eeeejdLd fdMeeeejdNd fd/eeeejd0d fd1eee ejd2d fd3eee ejd4d fd5eee ejd6d fd7eee ejd8d fd9eee!ejd:d fdeddf dOdPZ'ejdQgd   dgd ededRee(ejdSdTd fdeddf dUdVZ)ejdWgd  ded edededdfdXdYZ*ejdZgd   dgd eded[ee(ejd\d]d fdeddf d^d_Z+ejd`gd  ded edededdfdadbZ,dS)hz2CLI commands for Hugging Face Inference Endpoints.N) AnnotatedAnyOptional)InferenceEndpointInferenceEndpointScalingMetric)HfHubHTTPError) FormatOpt OutputFormatQuietOptTokenOpt get_hf_apiprint_list_output typer_factoryz(Manage Hugging Face Inference Endpoints.)helpz.Interact with the Inference Endpoints catalog.zEndpoint name.z_The namespace associated with the Inference Endpoint. Defaults to the current user's namespace.endpointreturncCsttj|jddddS)NTindent sort_keys)typerechojsondumpsraw)rra/home/kim/smarthome/.venv/lib/python3.10/site-packages/huggingface_hub/cli/inference_endpoints.py_print_endpoint,srz list | lszhf endpoints lsz"hf endpoints ls --namespace my-org)examplesF namespaceformatquiettokenc Cst|d}z |j||d}Wnty+}ztd|tj|jjd|d}~wwdd|D}dtt t fd t t fd d }t |||d gd |ddS)z6Lists all Inference Endpoints for the given namespace.r#)r r#zListing failed: codeNcSsg|]}|jqSr)r).0rrrr ?szls..itemrc Ss |di}|di}|di}|di}t|ddtt|tr+|ddndtt|tr9|ddndtt|trG|d dndtt|trU|d dndtt|trc|d dndtt|trq|d dndtt|tr|d dgSdgS)Nstatusmodelcomputeprovidername repositorystatetask frameworkZ instanceTypevendorregion)getstr isinstancedict)r)r*r+r,r-rrrrow_fnAs    zls..row_fnr.)NAMEZMODELZSTATUSZTASKZ FRAMEWORKZINSTANCEZVENDORZREGION)itemsr!r"Zid_keyheadersr:) r Zlist_inference_endpointsrrrExitresponse status_coder9r7rlistr) r r!r"r#apiZ endpointserrorresultsr:rrrls0s$  rEdeployzChf endpoints deploy my-endpoint --repo gpt2 --framework pytorch ...)r.r)r r2r# min_replica max_replicascale_to_zero_timeoutscaling_metricscaling_thresholdr.repozeThe name of the model repository associated with the Inference Endpoint (e.g. 'openai/gpt-oss-120b').r3z@The machine learning framework used for the model (e.g. 'vllm'). acceleratorz?The hardware accelerator to be used for inference (e.g. 'cpu'). instance_sizezNThe size or type of the instance to be used for hosting the model (e.g. 'x4'). instance_typezYThe cloud instance type where the Inference Endpoint will be deployed (e.g. 'intel-icl').r5zTThe cloud region in which the Inference Endpoint will be created (e.g. 'us-east-1').r4zVThe cloud provider or vendor where the Inference Endpoint will be hosted (e.g. 'aws').r2zCThe task on which to deploy the model (e.g. 'text-classification').rGzVThe minimum number of replicas (instances) to keep running for the Inference Endpoint.rHzRThe maximum number of replicas (instances) to scale to for the Inference Endpoint.rIzFThe duration in minutes before an inactive endpoint is scaled to zero.rJz!The metric reference for scaling.rKzeThe scaling metric threshold used to trigger a scale up. Ignored when scaling metric is not provided.cCst| d}|jdid|d|d|d|d|d|d|d |d |d | d | d | d| d|d|d| }t|dS)z3Deploy an Inference Endpoint from a Hub repository.r$r.r0r3rMrNrOr5r4r r2r#rGrHrJrKrINr)r Zcreate_inference_endpointr)r.rLr3rMrNrOr5r4r r2r#rGrHrIrJrKrBrrrrrF[sF V       zChf endpoints catalog deploy --repo meta-llama/Llama-3.2-1B-InstructzPThe hardware accelerator to be used for inference (e.g. 'cpu', 'gpu', 'neuron').c Csjt|d}z |j|||||d}Wnty.}ztd|tj|jjd|d}~wwt|dS)z4Deploy an Inference Endpoint from the Model Catalog.r$)Zrepo_idr.rMr r#zDeployment failed: r%N) r Z&create_inference_endpoint_from_catalogrrrr>r?r@r)rLr.rMr r#rBrrCrrrdeploy_from_catalogs   rPc Cstt|d}z|j|d}Wnty*}ztd|tj|jjd|d}~wwttj d|iddddS) zList available Catalog models.r$zCatalog fetch failed: r%NmodelsrTr) r Zlist_inference_catalogrrrr>r?r@rr)r#rBrQrCrrr list_catalogs rRzhf endpoints catalog lsz list-catalogT)r.Zhiddencatalog)r.z!hf endpoints describe my-endpointc Cft|d}z |j|||d}Wnty,}ztd|tj|jjd|d}~wwt|dS)z+Get information about an existing endpoint.r$r.r r#zFetch failed: r%N) r Zget_inference_endpointrrrr>r?r@rr.r r#rBrrCrrrdescribe  rWz/hf endpoints update my-endpoint --min-replica 2zBThe machine learning framework used for the model (e.g. 'custom').revisionzrThe specific model revision to deploy on the Inference Endpoint (e.g. '6c0e6080953db56375760c0471a8c5f2929baf11').cCs~t|d}z|j|||||||||| | | | | |d}Wnty8}ztd|tj|jjd|d}~wwt|dS)zUpdate an existing endpoint.r$)r.r r0r3rYr2rMrNrOrGrHrIrJrKr#zUpdate failed: r%N) r Zupdate_inference_endpointrrrr>r?r@r)r.r rLrMrNrOr3rYr2rGrHrIrJrKr#rBrrCrrrupdates2 O  rZzhf endpoints delete my-endpointyesz--yeszSkip confirmation prompts.c Cs|std|d}||krtdtjddt|d}z |j|||dWntyF}ztd|tj|jjd|d }~wwtd |d d S) z)Delete an Inference Endpoint permanently.zDelete endpoint 'z'? Type the name to confirm.zAborted.rr%r$rUzDelete failed: Nz Deleted 'z'.) rpromptrr>r Zdelete_inference_endpointrr?r@)r.r r[r#Z confirmationrBrCrrrdeleteys    r]zhf endpoints pause my-endpointc CrT)zPause an Inference Endpoint.r$rUzPause failed: r%N) r Zpause_inference_endpointrrrr>r?r@rrVrrrpauserXr^zhf endpoints resume my-endpointfail_if_already_runningz--fail-if-already-runningzWIf `True`, the method will raise an error if the Inference Endpoint is already running.c Csjt|d}z |j|||| d}Wnty.}ztd|tj|jjd|d}~wwt|dS)zResume an Inference Endpoint.r$)r.r r#Z running_okzResume failed: r%N) r Zresume_inference_endpointrrrr>r?r@r)r.r r_r#rBrrCrrrresumes   r`z&hf endpoints scale-to-zero my-endpointc CrT)z$Scale an Inference Endpoint to zero.r$rUzScale To Zero failed: r%N) r Z scale_to_zero_inference_endpointrrrr>r?r@rrVrrr scale_to_zerorXra)NNNN)N)NN)NNNNNNNNNNNNNN)NFN)-__doc__rtypingrrrrZ$huggingface_hub._inference_endpointsrrZhuggingface_hub.errorsrZ _cli_utilsr r r r r rrZie_cliZ catalog_appr7ZArgumentZNameArgOptionZNameOptZ NamespaceOptrcommandtablerEintfloatrFrPrRZ add_typerrWrZboolr]r^r`rarrrrs" $    *. &-.45;AGMS l #    !'-39?EKL g