Jul 01 20:44:21.870250 ip-10-0-140-104 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 01 20:44:21.870262 ip-10-0-140-104 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 01 20:44:21.870268 ip-10-0-140-104 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 01 20:44:21.870545 ip-10-0-140-104 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 01 20:44:32.064656 ip-10-0-140-104 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 01 20:44:32.064677 ip-10-0-140-104 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 501d87bfa0ee40e8b56f2addc91792bd --
Jul 01 20:46:21.394912 ip-10-0-140-104 systemd[1]: Starting Kubernetes Kubelet...
Jul 01 20:46:21.695921 ip-10-0-140-104 kubenswrapper[2571]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 01 20:46:21.695921 ip-10-0-140-104 kubenswrapper[2571]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 01 20:46:21.695921 ip-10-0-140-104 kubenswrapper[2571]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 01 20:46:21.695921 ip-10-0-140-104 kubenswrapper[2571]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 01 20:46:21.695921 ip-10-0-140-104 kubenswrapper[2571]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 01 20:46:21.696686 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.696601    2571 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 01 20:46:21.699832 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699819    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 20:46:21.699832 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699833    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 20:46:21.699896 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699836    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 20:46:21.699896 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699840    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 20:46:21.699896 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699843    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 20:46:21.699896 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699847    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 20:46:21.699896 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699849    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 20:46:21.699896 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699852    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 20:46:21.699896 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699855    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 20:46:21.699896 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699858    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 20:46:21.699896 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699861    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 20:46:21.699896 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699864    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 20:46:21.699896 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699866    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 20:46:21.699896 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699869    2571 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 20:46:21.699896 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699872    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 20:46:21.699896 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699874    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 20:46:21.699896 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699877    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 20:46:21.699896 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699880    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 20:46:21.699896 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699882    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 20:46:21.699896 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699885    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 20:46:21.699896 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699888    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 20:46:21.699896 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699891    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 20:46:21.699896 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699893    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 20:46:21.700387 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699896    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 20:46:21.700387 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699898    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 20:46:21.700387 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699901    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 20:46:21.700387 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699904    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 20:46:21.700387 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699907    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 20:46:21.700387 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699910    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 20:46:21.700387 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699912    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 20:46:21.700387 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699914    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 20:46:21.700387 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699918    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 20:46:21.700387 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699922    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 20:46:21.700387 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699925    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 20:46:21.700387 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699927    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 20:46:21.700387 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699930    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 20:46:21.700387 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699932    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 20:46:21.700387 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699935    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 20:46:21.700387 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699940    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 20:46:21.700387 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699943    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 20:46:21.700387 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699946    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 20:46:21.700387 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699948    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 20:46:21.700870 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699951    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 20:46:21.700870 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699953    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 20:46:21.700870 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699956    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 20:46:21.700870 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699958    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 20:46:21.700870 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699960    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 20:46:21.700870 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699963    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 20:46:21.700870 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699965    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 20:46:21.700870 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699968    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 20:46:21.700870 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699970    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 20:46:21.700870 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699972    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 20:46:21.700870 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699975    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 20:46:21.700870 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699977    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 20:46:21.700870 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699980    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 20:46:21.700870 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699982    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 20:46:21.700870 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699985    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 20:46:21.700870 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699988    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 20:46:21.700870 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699990    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 20:46:21.700870 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699993    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 20:46:21.700870 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699995    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 20:46:21.700870 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.699998    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 20:46:21.701348 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700000    2571 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 20:46:21.701348 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700002    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 20:46:21.701348 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700005    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 20:46:21.701348 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700007    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 20:46:21.701348 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700009    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 20:46:21.701348 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700012    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 20:46:21.701348 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700015    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 20:46:21.701348 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700017    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 20:46:21.701348 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700019    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 20:46:21.701348 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700022    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 20:46:21.701348 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700025    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 20:46:21.701348 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700027    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 20:46:21.701348 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700030    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 20:46:21.701348 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700032    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 20:46:21.701348 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700034    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 20:46:21.701348 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700037    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 20:46:21.701348 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700039    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 20:46:21.701348 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700041    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 20:46:21.701348 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700044    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 20:46:21.701348 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700046    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 20:46:21.701840 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700049    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 20:46:21.701840 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700052    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 20:46:21.701840 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700054    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 20:46:21.701840 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700057    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 20:46:21.701840 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700061    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 20:46:21.701840 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700064    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 20:46:21.701840 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700068    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 20:46:21.701840 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700071    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 20:46:21.701840 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700074    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 20:46:21.701840 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700077    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 20:46:21.701840 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700079    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 20:46:21.701840 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700082    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 20:46:21.701840 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700084    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 20:46:21.701840 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700086    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 20:46:21.701840 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700089    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 20:46:21.701840 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700091    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 20:46:21.701840 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700094    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 20:46:21.701840 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700768    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 20:46:21.701840 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700774    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 20:46:21.702310 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700777    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 20:46:21.702310 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700779    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 20:46:21.702310 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700782    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 20:46:21.702310 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700785    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 20:46:21.702310 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700788    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 20:46:21.702310 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700790    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 20:46:21.702310 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700793    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 20:46:21.702310 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700795    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 20:46:21.702310 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700819    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 20:46:21.702310 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700822    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 20:46:21.702310 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700825    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 20:46:21.702310 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700827    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 20:46:21.702310 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700830    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 20:46:21.702310 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700833    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 20:46:21.702310 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700835    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 20:46:21.702310 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700838    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 20:46:21.702310 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700840    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 20:46:21.702310 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700843    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 20:46:21.702310 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700845    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 20:46:21.702310 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700849    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 20:46:21.702821 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700852    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 20:46:21.702821 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700854    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 20:46:21.702821 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700857    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 20:46:21.702821 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700859    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 20:46:21.702821 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700862    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 20:46:21.702821 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700865    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 20:46:21.702821 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700867    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 20:46:21.702821 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700869    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 20:46:21.702821 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700872    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 20:46:21.702821 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700875    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 20:46:21.702821 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700877    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 20:46:21.702821 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700880    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 20:46:21.702821 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700882    2571 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 20:46:21.702821 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700885    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 20:46:21.702821 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700887    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 20:46:21.702821 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700890    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 20:46:21.702821 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700893    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 20:46:21.702821 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700895    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 20:46:21.702821 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700898    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 20:46:21.702821 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700900    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 20:46:21.703306 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700904    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 20:46:21.703306 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700907    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 20:46:21.703306 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700910    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 20:46:21.703306 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700913    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 20:46:21.703306 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700915    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 20:46:21.703306 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700918    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 20:46:21.703306 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700921    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 20:46:21.703306 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700923    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 20:46:21.703306 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700925    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 20:46:21.703306 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700928    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 20:46:21.703306 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700930    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 20:46:21.703306 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700935    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 20:46:21.703306 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700938    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 20:46:21.703306 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700941    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 20:46:21.703306 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700944    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 20:46:21.703306 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700947    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 20:46:21.703306 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700950    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 20:46:21.703306 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700952    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 20:46:21.703306 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700954    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 20:46:21.703787 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700957    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 20:46:21.703787 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700959    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 20:46:21.703787 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700962    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 20:46:21.703787 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700964    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 20:46:21.703787 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700967    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 20:46:21.703787 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700969    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 20:46:21.703787 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700971    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 20:46:21.703787 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700974    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 20:46:21.703787 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700977    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 20:46:21.703787 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700979    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 20:46:21.703787 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700982    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 20:46:21.703787 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700984    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 20:46:21.703787 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700988    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 20:46:21.703787 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700990    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 20:46:21.703787 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700993    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 20:46:21.703787 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700995    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 20:46:21.703787 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.700997    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 20:46:21.703787 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701000    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 20:46:21.703787 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701002    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 20:46:21.703787 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701005    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 20:46:21.704261 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701007    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 20:46:21.704261 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701009    2571 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 20:46:21.704261 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701012    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 20:46:21.704261 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701014    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 20:46:21.704261 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701017    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 20:46:21.704261 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701020    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 20:46:21.704261 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701022    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 20:46:21.704261 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701025    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 20:46:21.704261 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701028    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 20:46:21.704261 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701030    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 20:46:21.704261 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701033    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 20:46:21.704261 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701037    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 20:46:21.704261 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701040    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 20:46:21.704261 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701043    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 20:46:21.704261 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701045    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 20:46:21.704261 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701048    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 20:46:21.704261 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701050    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 20:46:21.704261 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701053    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 20:46:21.704261 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701385    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 20:46:21.704763 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701390    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 20:46:21.704763 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701393    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 20:46:21.704763 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701396    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 20:46:21.704763 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701399    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 20:46:21.704763 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701402    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 20:46:21.704763 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701405    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 20:46:21.704763 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701407    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 20:46:21.704763 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701410    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 20:46:21.704763 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701412    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 20:46:21.704763 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701415    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 20:46:21.704763 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701417    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 20:46:21.704763 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701420    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 20:46:21.704763 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701422    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 20:46:21.704763 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701425    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 20:46:21.704763 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701427    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 20:46:21.704763 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701430    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 20:46:21.704763 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701432    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 20:46:21.704763 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701436    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 20:46:21.704763 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701440    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 20:46:21.705226 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701444    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 20:46:21.705226 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701447    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 20:46:21.705226 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701450    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 20:46:21.705226 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701453    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 20:46:21.705226 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701456    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 20:46:21.705226 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701460    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 20:46:21.705226 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701463    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 20:46:21.705226 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701465    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 20:46:21.705226 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701468    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 20:46:21.705226 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701470    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 20:46:21.705226 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701473    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 20:46:21.705226 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701475    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 20:46:21.705226 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701478    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 20:46:21.705226 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701480    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 20:46:21.705226 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701484    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 20:46:21.705226 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701487    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 20:46:21.705226 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701490    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 20:46:21.705226 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701493    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 20:46:21.705226 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701496    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 20:46:21.705717 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701498    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 20:46:21.705717 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701501    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 20:46:21.705717 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701504    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 20:46:21.705717 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701506    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 20:46:21.705717 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701508    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 20:46:21.705717 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701511    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 20:46:21.705717 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701514    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 20:46:21.705717 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701516    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 20:46:21.705717 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701520    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 20:46:21.705717 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701523    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 20:46:21.705717 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701526    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 20:46:21.705717 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701528    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 20:46:21.705717 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701531    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 20:46:21.705717 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701533    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 20:46:21.705717 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701536    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 20:46:21.705717 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701538    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 20:46:21.705717 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701540    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 20:46:21.705717 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701543    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 20:46:21.705717 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701545    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 20:46:21.706233 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701548    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 20:46:21.706233 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701550    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 20:46:21.706233 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701553    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 20:46:21.706233 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701555    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 20:46:21.706233 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701557    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 20:46:21.706233 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701560    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 20:46:21.706233 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701562    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 20:46:21.706233 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701564    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 20:46:21.706233 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701567    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 20:46:21.706233 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701569    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 20:46:21.706233 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701571    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 20:46:21.706233 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701573    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 20:46:21.706233 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701576    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 20:46:21.706233 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701579    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 20:46:21.706233 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701581    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 20:46:21.706233 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701586    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 20:46:21.706233 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701588    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 20:46:21.706233 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701590    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 20:46:21.706233 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701593    2571 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 20:46:21.706233 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701596    2571 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 20:46:21.706736 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701598    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 20:46:21.706736 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701601    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 20:46:21.706736 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701603    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 20:46:21.706736 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701606    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 20:46:21.706736 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701608    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 20:46:21.706736 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701610    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 20:46:21.706736 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701613    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 20:46:21.706736 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701615    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 20:46:21.706736 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701618    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 20:46:21.706736 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701620    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 20:46:21.706736 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701640    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 20:46:21.706736 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701643    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 20:46:21.706736 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701646    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 20:46:21.706736 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701648    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 20:46:21.706736 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701650    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 20:46:21.706736 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701653    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 20:46:21.706736 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701655    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 20:46:21.706736 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701658    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 20:46:21.706736 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701660    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 20:46:21.706736 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701663    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 20:46:21.707216 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.701665    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 20:46:21.707216 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701729    2571 flags.go:64] FLAG: --address="0.0.0.0"
Jul 01 20:46:21.707216 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701735    2571 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 01 20:46:21.707216 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701741    2571 flags.go:64] FLAG: --anonymous-auth="true"
Jul 01 20:46:21.707216 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701745    2571 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 01 20:46:21.707216 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701750    2571 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 01 20:46:21.707216 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701753    2571 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 01 20:46:21.707216 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701759    2571 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 01 20:46:21.707216 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701763    2571 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 01 20:46:21.707216 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701766    2571 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 01 20:46:21.707216 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701769    2571 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 01 20:46:21.707216 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701772    2571 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 01 20:46:21.707216 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701775    2571 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 01 20:46:21.707216 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701778    2571 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 01 20:46:21.707216 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701781    2571 flags.go:64] FLAG: --cgroup-root=""
Jul 01 20:46:21.707216 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701783    2571 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 01 20:46:21.707216 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701786    2571 flags.go:64] FLAG: --client-ca-file=""
Jul 01 20:46:21.707216 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701789    2571 flags.go:64] FLAG: --cloud-provider="external"
Jul 01 20:46:21.707216 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701791    2571 flags.go:64] FLAG: --cluster-dns="[]"
Jul 01 20:46:21.707216 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701795    2571 flags.go:64] FLAG: --cluster-domain=""
Jul 01 20:46:21.707216 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701797    2571 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 01 20:46:21.707216 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701800    2571 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 01 20:46:21.707216 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701803    2571 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 01 20:46:21.707216 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701806    2571 flags.go:64] FLAG: --container-log-max-files="5"
Jul 01 20:46:21.707837 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701809    2571 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 01 20:46:21.707837 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701812    2571 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 01 20:46:21.707837 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701815    2571 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 01 20:46:21.707837 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701818    2571 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 01 20:46:21.707837 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701821    2571 flags.go:64] FLAG: --contention-profiling="false"
Jul 01 20:46:21.707837 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701826    2571 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 01 20:46:21.707837 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701828    2571 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 01 20:46:21.707837 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701831    2571 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 01 20:46:21.707837 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701834    2571 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 01 20:46:21.707837 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701838    2571 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 01 20:46:21.707837 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701841    2571 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 01 20:46:21.707837 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701844    2571 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 01 20:46:21.707837 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701846    2571 flags.go:64] FLAG: --enable-load-reader="false"
Jul 01 20:46:21.707837 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701849    2571 flags.go:64] FLAG: --enable-server="true"
Jul 01 20:46:21.707837 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701853    2571 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 01 20:46:21.707837 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701858    2571 flags.go:64] FLAG: --event-burst="100"
Jul 01 20:46:21.707837 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701861    2571 flags.go:64] FLAG: --event-qps="50"
Jul 01 20:46:21.707837 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701864    2571 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 01 20:46:21.707837 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701867    2571 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 01 20:46:21.707837 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701869    2571 flags.go:64] FLAG: --eviction-hard=""
Jul 01 20:46:21.707837 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701873    2571 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 01 20:46:21.707837 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701875    2571 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 01 20:46:21.707837 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701878    2571 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 01 20:46:21.707837 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701881    2571 flags.go:64] FLAG: --eviction-soft=""
Jul 01 20:46:21.707837 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701884    2571 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 01 20:46:21.708424 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701886    2571 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 01 20:46:21.708424 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701889    2571 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 01 20:46:21.708424 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701892    2571 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 01 20:46:21.708424 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701894    2571 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 01 20:46:21.708424 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701897    2571 flags.go:64] FLAG: --fail-swap-on="true"
Jul 01 20:46:21.708424 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701900    2571 flags.go:64] FLAG: --feature-gates=""
Jul 01 20:46:21.708424 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701903    2571 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 01 20:46:21.708424 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701906    2571 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 01 20:46:21.708424 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701909    2571 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 01 20:46:21.708424 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701911    2571 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 01 20:46:21.708424 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701914    2571 flags.go:64] FLAG: --healthz-port="10248"
Jul 01 20:46:21.708424 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701917    2571 flags.go:64] FLAG: --help="false"
Jul 01 20:46:21.708424 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701921    2571 flags.go:64] FLAG: --hostname-override="ip-10-0-140-104.ec2.internal"
Jul 01 20:46:21.708424 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701923    2571 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 01 20:46:21.708424 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701926    2571 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 01 20:46:21.708424 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701929    2571 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 01 20:46:21.708424 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701932    2571 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 01 20:46:21.708424 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701935    2571 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 01 20:46:21.708424 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701938    2571 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 01 20:46:21.708424 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701940    2571 flags.go:64] FLAG: --image-service-endpoint=""
Jul 01 20:46:21.708424 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701943    2571 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 01 20:46:21.708424 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701946    2571 flags.go:64] FLAG: --kube-api-burst="100"
Jul 01 20:46:21.708424 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701950    2571 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 01 20:46:21.709126 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701953    2571 flags.go:64] FLAG: --kube-api-qps="50"
Jul 01 20:46:21.709126 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701956    2571 flags.go:64] FLAG: --kube-reserved=""
Jul 01 20:46:21.709126 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701959    2571 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 01 20:46:21.709126 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701961    2571 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 01 20:46:21.709126 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701964    2571 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 01 20:46:21.709126 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701967    2571 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 01 20:46:21.709126 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701969    2571 flags.go:64] FLAG: --lock-file=""
Jul 01 20:46:21.709126 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701972    2571 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 01 20:46:21.709126 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701974    2571 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 01 20:46:21.709126 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701977    2571 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 01 20:46:21.709126 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701982    2571 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 01 20:46:21.709126 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701985    2571 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 01 20:46:21.709126 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701987    2571 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 01 20:46:21.709126 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701990    2571 flags.go:64] FLAG: --logging-format="text"
Jul 01 20:46:21.709126 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701992    2571 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 01 20:46:21.709126 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701995    2571 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 01 20:46:21.709126 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.701998    2571 flags.go:64] FLAG: --manifest-url=""
Jul 01 20:46:21.709126 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702000    2571 flags.go:64] FLAG: --manifest-url-header=""
Jul 01 20:46:21.709126 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702004    2571 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 01 20:46:21.709126 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702007    2571 flags.go:64] FLAG: --max-open-files="1000000"
Jul 01 20:46:21.709126 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702022    2571 flags.go:64] FLAG: --max-pods="110"
Jul 01 20:46:21.709126 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702027    2571 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 01 20:46:21.709126 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702030    2571 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 01 20:46:21.709126 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702033    2571 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 01 20:46:21.709126 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702036    2571 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 01 20:46:21.709794 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702038    2571 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 01 20:46:21.709794 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702041    2571 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 01 20:46:21.709794 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702044    2571 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 01 20:46:21.709794 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702050    2571 flags.go:64] FLAG: --node-status-max-images="50"
Jul 01 20:46:21.709794 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702053    2571 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 01 20:46:21.709794 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702056    2571 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 01 20:46:21.709794 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702060    2571 flags.go:64] FLAG: --pod-cidr=""
Jul 01 20:46:21.709794 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702064    2571 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8a2632cd31d78133be7b5fb6999469b2acf9b98d077c7f6ae5d0d94083d76b04"
Jul 01 20:46:21.709794 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702070    2571 flags.go:64] FLAG: --pod-manifest-path=""
Jul 01 20:46:21.709794 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702072    2571 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 01 20:46:21.709794 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702075    2571 flags.go:64] FLAG: --pods-per-core="0"
Jul 01 20:46:21.709794 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702078    2571 flags.go:64] FLAG: --port="10250"
Jul 01 20:46:21.709794 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702081    2571 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 01 20:46:21.709794 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702084    2571 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d101a82eca7580fd"
Jul 01 20:46:21.709794 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702087    2571 flags.go:64] FLAG: --qos-reserved=""
Jul 01 20:46:21.709794 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702089    2571 flags.go:64] FLAG: --read-only-port="10255"
Jul 01 20:46:21.709794 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702092    2571 flags.go:64] FLAG: --register-node="true"
Jul 01 20:46:21.709794 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702095    2571 flags.go:64] FLAG: --register-with-taints=""
Jul 01 20:46:21.709794 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702098    2571 flags.go:64] FLAG: --registry-burst="10"
Jul 01 20:46:21.709794 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702101    2571 flags.go:64] FLAG: --registry-qps="5"
Jul 01 20:46:21.709794 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702103    2571 flags.go:64] FLAG: --reserved-cpus=""
Jul 01 20:46:21.709794 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702106    2571 flags.go:64] FLAG: --reserved-memory=""
Jul 01 20:46:21.709794 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702109    2571 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 01 20:46:21.709794 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702112    2571 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 01 20:46:21.709794 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702114    2571 flags.go:64] FLAG: --rotate-certificates="false"
Jul 01 20:46:21.710432 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702117    2571 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 01 20:46:21.710432 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702120    2571 flags.go:64] FLAG: --runonce="false"
Jul 01 20:46:21.710432 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702122    2571 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 01 20:46:21.710432 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702125    2571 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 01 20:46:21.710432 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702129    2571 flags.go:64] FLAG: --seccomp-default="false"
Jul 01 20:46:21.710432 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702132    2571 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 01 20:46:21.710432 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702134    2571 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 01 20:46:21.710432 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702137    2571 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 01 20:46:21.710432 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702140    2571 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 01 20:46:21.710432 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702143    2571 flags.go:64] FLAG: --storage-driver-password="root"
Jul 01 20:46:21.710432 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702145    2571 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 01 20:46:21.710432 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702148    2571 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 01 20:46:21.710432 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702151    2571 flags.go:64] FLAG: --storage-driver-user="root"
Jul 01 20:46:21.710432 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702154    2571 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 01 20:46:21.710432 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702157    2571 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 01 20:46:21.710432 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702161    2571 flags.go:64] FLAG: --system-cgroups=""
Jul 01 20:46:21.710432 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702163    2571 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 01 20:46:21.710432 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702168    2571 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 01 20:46:21.710432 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702171    2571 flags.go:64] FLAG: --tls-cert-file=""
Jul 01 20:46:21.710432 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702173    2571 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 01 20:46:21.710432 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702177    2571 flags.go:64] FLAG: --tls-min-version=""
Jul 01 20:46:21.710432 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702180    2571 flags.go:64] FLAG: --tls-private-key-file=""
Jul 01 20:46:21.710432 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702182    2571 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 01 20:46:21.710432 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702185    2571 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 01 20:46:21.710432 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702187    2571 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 01 20:46:21.711045 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702190    2571 flags.go:64] FLAG: --v="2"
Jul 01 20:46:21.711045 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702194    2571 flags.go:64] FLAG: --version="false"
Jul 01 20:46:21.711045 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702198    2571 flags.go:64] FLAG: --vmodule=""
Jul 01 20:46:21.711045 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702201    2571 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 01 20:46:21.711045 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.702204    2571 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 01 20:46:21.711045 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702311    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 20:46:21.711045 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702315    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 20:46:21.711045 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702318    2571 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 20:46:21.711045 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702320    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 20:46:21.711045 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702323    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 20:46:21.711045 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702326    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 20:46:21.711045 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702329    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 20:46:21.711045 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702331    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 20:46:21.711045 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702334    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 20:46:21.711045 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702336    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 20:46:21.711045 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702339    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 20:46:21.711045 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702341    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 20:46:21.711045 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702344    2571 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 20:46:21.711045 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702346    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 20:46:21.711045 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702348    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 20:46:21.711045 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702356    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 20:46:21.711045 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702358    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 20:46:21.711594 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702361    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 20:46:21.711594 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702364    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 20:46:21.711594 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702367    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 20:46:21.711594 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702371    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 20:46:21.711594 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702374    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 20:46:21.711594 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702377    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 20:46:21.711594 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702380    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 20:46:21.711594 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702383    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 20:46:21.711594 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702386    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 20:46:21.711594 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702388    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 20:46:21.711594 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702390    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 20:46:21.711594 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702393    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 20:46:21.711594 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702395    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 20:46:21.711594 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702398    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 20:46:21.711594 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702400    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 20:46:21.711594 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702402    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 20:46:21.711594 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702405    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 20:46:21.711594 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702407    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 20:46:21.711594 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702409    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 20:46:21.712074 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702412    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 20:46:21.712074 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702414    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 20:46:21.712074 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702417    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 20:46:21.712074 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702420    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 20:46:21.712074 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702422    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 20:46:21.712074 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702424    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 20:46:21.712074 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702427    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 20:46:21.712074 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702429    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 20:46:21.712074 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702432    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 20:46:21.712074 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702434    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 20:46:21.712074 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702437    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 20:46:21.712074 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702441    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 20:46:21.712074 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702443    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 20:46:21.712074 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702446    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 20:46:21.712074 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702450    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 20:46:21.712074 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702453    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 20:46:21.712074 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702456    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 20:46:21.712074 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702459    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 20:46:21.712074 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702461    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 20:46:21.712538 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702464    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 20:46:21.712538 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702466    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 20:46:21.712538 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702468    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 20:46:21.712538 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702471    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 20:46:21.712538 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702474    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 20:46:21.712538 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702476    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 20:46:21.712538 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702478    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 20:46:21.712538 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702481    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 20:46:21.712538 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702483    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 20:46:21.712538 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702485    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 20:46:21.712538 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702488    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 20:46:21.712538 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702490    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 20:46:21.712538 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702493    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 20:46:21.712538 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702495    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 20:46:21.712538 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702498    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 20:46:21.712538 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702500    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 20:46:21.712538 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702503    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 20:46:21.712538 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702505    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 20:46:21.712538 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702508    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 20:46:21.712538 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702510    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 20:46:21.713059 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702512    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 20:46:21.713059 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702515    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 20:46:21.713059 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702517    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 20:46:21.713059 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702520    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 20:46:21.713059 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702524    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 20:46:21.713059 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702526    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 20:46:21.713059 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702528    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 20:46:21.713059 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702531    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 20:46:21.713059 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702533    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 20:46:21.713059 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702536    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 20:46:21.713059 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702538    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 20:46:21.713059 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702541    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 20:46:21.713059 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702543    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 20:46:21.713059 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702545    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 20:46:21.713059 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702548    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 20:46:21.713059 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702550    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 20:46:21.713059 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702552    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 20:46:21.713059 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702555    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 20:46:21.713059 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702557    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 20:46:21.713059 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702559    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 20:46:21.713553 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702562    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 20:46:21.713553 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702564    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 20:46:21.713553 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702568    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 20:46:21.713553 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.702571    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 20:46:21.713553 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.703138    2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 01 20:46:21.713553 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.710515    2571 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 01 20:46:21.713553 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.710529    2571 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 01 20:46:21.713553 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.710543    2571 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 01 20:46:21.713553 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.710548    2571 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 01 20:46:21.713553 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710601    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 20:46:21.713553 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710606    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 20:46:21.713553 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710610    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 20:46:21.713553 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710612    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 20:46:21.713553 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710615    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 20:46:21.713553 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710618    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 20:46:21.714008 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710621    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 20:46:21.714008 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710641    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 20:46:21.714008 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710644    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 20:46:21.714008 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710647    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 20:46:21.714008 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710650    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 20:46:21.714008 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710652    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 20:46:21.714008 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710656    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 20:46:21.714008 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710659    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 20:46:21.714008 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710662    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 20:46:21.714008 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710664    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 20:46:21.714008 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710667    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 20:46:21.714008 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710669    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 20:46:21.714008 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710672    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 20:46:21.714008 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710674    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 20:46:21.714008 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710676    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 20:46:21.714008 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710680    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 20:46:21.714008 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710682    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 20:46:21.714008 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710685    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 20:46:21.714008 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710687    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 20:46:21.714008 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710690    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 20:46:21.714498 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710692    2571 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 20:46:21.714498 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710694    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 20:46:21.714498 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710697    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 20:46:21.714498 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710700    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 20:46:21.714498 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710703    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 20:46:21.714498 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710705    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 20:46:21.714498 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710709    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 20:46:21.714498 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710713    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 20:46:21.714498 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710716    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 20:46:21.714498 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710718    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 20:46:21.714498 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710720    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 20:46:21.714498 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710723    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 20:46:21.714498 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710725    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 20:46:21.714498 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710728    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 20:46:21.714498 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710730    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 20:46:21.714498 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710733    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 20:46:21.714498 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710735    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 20:46:21.714498 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710738    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 20:46:21.714498 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710740    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 20:46:21.714498 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710743    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 20:46:21.715026 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710746    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 20:46:21.715026 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710748    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 20:46:21.715026 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710750    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 20:46:21.715026 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710753    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 20:46:21.715026 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710755    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 20:46:21.715026 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710757    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 20:46:21.715026 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710760    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 20:46:21.715026 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710762    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 20:46:21.715026 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710765    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 20:46:21.715026 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710767    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 20:46:21.715026 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710770    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 20:46:21.715026 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710772    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 20:46:21.715026 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710775    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 20:46:21.715026 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710777    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 20:46:21.715026 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710780    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 20:46:21.715026 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710782    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 20:46:21.715026 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710785    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 20:46:21.715026 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710788    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 20:46:21.715026 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710790    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 20:46:21.715026 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710793    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 20:46:21.715503 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710795    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 20:46:21.715503 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710799    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 20:46:21.715503 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710802    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 20:46:21.715503 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710805    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 20:46:21.715503 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710808    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 20:46:21.715503 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710812    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 20:46:21.715503 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710814    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 20:46:21.715503 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710817    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 20:46:21.715503 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710820    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 20:46:21.715503 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710822    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 20:46:21.715503 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710825    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 20:46:21.715503 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710828    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 20:46:21.715503 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710831    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 20:46:21.715503 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710834    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 20:46:21.715503 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710837    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 20:46:21.715503 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710839    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 20:46:21.715503 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710842    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 20:46:21.715503 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710844    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 20:46:21.715957 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710846    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 20:46:21.715957 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710849    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 20:46:21.715957 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710851    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 20:46:21.715957 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710853    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 20:46:21.715957 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710856    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 20:46:21.715957 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710858    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 20:46:21.715957 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710861    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 20:46:21.715957 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710863    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 20:46:21.715957 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710866    2571 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 20:46:21.715957 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710868    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 20:46:21.715957 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710871    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 20:46:21.715957 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710874    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 20:46:21.715957 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710876    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 20:46:21.715957 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710879    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 20:46:21.715957 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.710881    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 20:46:21.716309 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.710886    2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 01 20:46:21.716309 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711000    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 20:46:21.716309 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711004    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 20:46:21.716309 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711007    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 20:46:21.716309 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711010    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 20:46:21.716309 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711013    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 20:46:21.716309 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711016    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 20:46:21.716309 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711019    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 20:46:21.716309 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711021    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 20:46:21.716309 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711024    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 20:46:21.716309 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711027    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 20:46:21.716309 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711029    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 20:46:21.716309 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711033    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 20:46:21.716309 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711035    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 20:46:21.716309 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711038    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 20:46:21.716700 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711040    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 20:46:21.716700 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711043    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 20:46:21.716700 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711045    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 20:46:21.716700 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711047    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 20:46:21.716700 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711050    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 20:46:21.716700 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711053    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 20:46:21.716700 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711055    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 20:46:21.716700 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711058    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 20:46:21.716700 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711060    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 20:46:21.716700 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711063    2571 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 20:46:21.716700 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711065    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 20:46:21.716700 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711068    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 20:46:21.716700 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711071    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 20:46:21.716700 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711074    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 20:46:21.716700 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711076    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 20:46:21.716700 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711078    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 20:46:21.716700 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711081    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 20:46:21.716700 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711083    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 20:46:21.716700 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711085    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 20:46:21.716700 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711089    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 20:46:21.717179 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711092    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 20:46:21.717179 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711094    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 20:46:21.717179 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711097    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 20:46:21.717179 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711099    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 20:46:21.717179 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711103    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 20:46:21.717179 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711106    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 20:46:21.717179 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711109    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 20:46:21.717179 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711111    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 20:46:21.717179 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711114    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 20:46:21.717179 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711116    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 20:46:21.717179 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711119    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 20:46:21.717179 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711121    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 20:46:21.717179 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711124    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 20:46:21.717179 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711126    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 20:46:21.717179 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711129    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 20:46:21.717179 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711131    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 20:46:21.717179 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711134    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 20:46:21.717179 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711136    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 20:46:21.717179 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711139    2571 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 20:46:21.717179 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711141    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 20:46:21.717704 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711144    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 20:46:21.717704 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711146    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 20:46:21.717704 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711149    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 20:46:21.717704 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711151    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 20:46:21.717704 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711153    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 20:46:21.717704 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711156    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 20:46:21.717704 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711158    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 20:46:21.717704 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711160    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 20:46:21.717704 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711163    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 20:46:21.717704 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711165    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 20:46:21.717704 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711169    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 20:46:21.717704 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711172    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 20:46:21.717704 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711175    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 20:46:21.717704 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711177    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 20:46:21.717704 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711180    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 20:46:21.717704 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711183    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 20:46:21.717704 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711185    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 20:46:21.717704 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711188    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 20:46:21.717704 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711191    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 20:46:21.718163 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711193    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 20:46:21.718163 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711196    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 20:46:21.718163 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711199    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 20:46:21.718163 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711201    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 20:46:21.718163 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711204    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 20:46:21.718163 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711206    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 20:46:21.718163 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711208    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 20:46:21.718163 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711211    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 20:46:21.718163 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711213    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 20:46:21.718163 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711216    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 20:46:21.718163 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711219    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 20:46:21.718163 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711221    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 20:46:21.718163 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711224    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 20:46:21.718163 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711226    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 20:46:21.718163 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711228    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 20:46:21.718163 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711231    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 20:46:21.718163 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711233    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 20:46:21.718163 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711235    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 20:46:21.718163 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711238    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 20:46:21.718163 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711240    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 20:46:21.718858 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711242    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 20:46:21.718858 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711244    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 20:46:21.718858 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711247    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 20:46:21.718858 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711249    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 20:46:21.718858 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711251    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 20:46:21.718858 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:21.711253    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 20:46:21.718858 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.711258    2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 01 20:46:21.718858 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.711353    2571 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 01 20:46:21.718858 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.713826    2571 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 01 20:46:21.718858 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.714913    2571 server.go:1019] "Starting client certificate rotation"
Jul 01 20:46:21.718858 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.715012    2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 01 20:46:21.718858 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.715622    2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 01 20:46:21.718858 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.716963    2571 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 01 20:46:21.722062 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.722043    2571 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 01 20:46:21.729983 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.729967    2571 log.go:25] "Validated CRI v1 runtime API"
Jul 01 20:46:21.732042 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.732028    2571 log.go:25] "Validated CRI v1 image API"
Jul 01 20:46:21.732893 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.732875    2571 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 01 20:46:21.735898 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.735880    2571 fs.go:135] Filesystem UUIDs: map[41b7616a-3d71-4863-9f35-22e2f93ebb52:/dev/nvme0n1p3 5ccc8131-bace-4288-b753-baa820475125:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 01 20:46:21.735971 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.735897    2571 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 01 20:46:21.740561 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.740466    2571 manager.go:216] Machine: {Timestamp:2026-07-01 20:46:21.739395054 +0000 UTC m=+0.251870235 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3112848 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d75686a574dcdd7a307cd59e0e53c SystemUUID:ec2d7568-6a57-4dcd-d7a3-07cd59e0e53c BootID:501d87bf-a0ee-40e8-b56f-2addc91792bd Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:9a:e6:5b:9c:91 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:9a:e6:5b:9c:91 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ba:a3:01:60:8c:95 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 01 20:46:21.740561 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.740556    2571 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 01 20:46:21.740682 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.740619    2571 manager.go:232] Version: {KernelVersion:5.14.0-570.124.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260623-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 01 20:46:21.742981 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.742954    2571 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 01 20:46:21.743109 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.742983    2571 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-104.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 01 20:46:21.743640 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.743617    2571 topology_manager.go:138] "Creating topology manager with none policy"
Jul 01 20:46:21.743673 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.743651    2571 container_manager_linux.go:309] "Creating device plugin manager"
Jul 01 20:46:21.743673 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.743659    2571 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 01 20:46:21.744205 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.744194    2571 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 01 20:46:21.744946 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.744928    2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 01 20:46:21.745095 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.745084    2571 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 01 20:46:21.748919 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.748908    2571 state_mem.go:36] "Initialized new in-memory state store"
Jul 01 20:46:21.749155 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.749146    2571 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 01 20:46:21.749188 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.749174    2571 kubelet.go:486] "Attempting to sync node with API server"
Jul 01 20:46:21.749188 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.749185    2571 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 01 20:46:21.749244 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.749200    2571 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 01 20:46:21.749244 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.749210    2571 kubelet.go:398] "Adding apiserver pod source"
Jul 01 20:46:21.749244 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.749218    2571 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 01 20:46:21.749551 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.749540    2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 01 20:46:21.749616 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.749557    2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 01 20:46:21.751678 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.751661    2571 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 01 20:46:21.754252 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.754181    2571 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 01 20:46:21.755604 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.755585    2571 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 01 20:46:21.756284 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.756270    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 01 20:46:21.756342 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.756288    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 01 20:46:21.756342 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.756294    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 01 20:46:21.756342 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.756300    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 01 20:46:21.756342 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.756309    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 01 20:46:21.756342 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.756317    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 01 20:46:21.756342 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.756327    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 01 20:46:21.756342 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.756333    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 01 20:46:21.756342 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.756339    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 01 20:46:21.756342 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.756345    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 01 20:46:21.756639 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.756357    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 01 20:46:21.756639 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.756362    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 01 20:46:21.756953 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.756943    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 01 20:46:21.756993 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.756955    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 01 20:46:21.760202 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.760188    2571 server.go:1268] "Started kubelet"
Jul 01 20:46:21.760329 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.760279    2571 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 01 20:46:21.760389 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.760350    2571 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 01 20:46:21.760481 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.760299    2571 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 01 20:46:21.760988 ip-10-0-140-104 systemd[1]: Started Kubernetes Kubelet.
Jul 01 20:46:21.761559 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.761477    2571 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 01 20:46:21.761933 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:21.761901    2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 01 20:46:21.762109 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.762018    2571 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-104.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 01 20:46:21.762152 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:21.762125    2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-104.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 01 20:46:21.762316 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.762298    2571 server.go:310] "Adding debug handlers to kubelet server"
Jul 01 20:46:21.766030 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:21.766000    2571 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 01 20:46:21.767541 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.767527    2571 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 01 20:46:21.767602 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.767549    2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 01 20:46:21.767683 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:21.767598    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-104.ec2.internal\" not found"
Jul 01 20:46:21.768119 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.768065    2571 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 01 20:46:21.768119 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.768083    2571 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 01 20:46:21.768119 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.768105    2571 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 01 20:46:21.768326 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:21.767148    2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-104.ec2.internal.18be45602714f96b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-104.ec2.internal,UID:ip-10-0-140-104.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-104.ec2.internal,},FirstTimestamp:2026-07-01 20:46:21.760158059 +0000 UTC m=+0.272633239,LastTimestamp:2026-07-01 20:46:21.760158059 +0000 UTC m=+0.272633239,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-104.ec2.internal,}"
Jul 01 20:46:21.768398 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.768367    2571 reconstruct.go:97] "Volume reconstruction finished"
Jul 01 20:46:21.768398 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.768378    2571 reconciler.go:29] "Reconciler: start to sync state"
Jul 01 20:46:21.768520 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.768504    2571 factory.go:55] Registering systemd factory
Jul 01 20:46:21.768520 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.768522    2571 factory.go:223] Registration of the systemd container factory successfully
Jul 01 20:46:21.768725 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.768713    2571 factory.go:153] Registering CRI-O factory
Jul 01 20:46:21.768831 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.768729    2571 factory.go:223] Registration of the crio container factory successfully
Jul 01 20:46:21.768831 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.768778    2571 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 01 20:46:21.768831 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.768800    2571 factory.go:103] Registering Raw factory
Jul 01 20:46:21.768831 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.768815    2571 manager.go:1196] Started watching for new ooms in manager
Jul 01 20:46:21.769146 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.769133    2571 manager.go:349] Starting recovery of all containers
Jul 01 20:46:21.771867 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:21.771842    2571 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 01 20:46:21.772023 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:21.772000    2571 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-104.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 01 20:46:21.772161 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.772143    2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-98rxl"
Jul 01 20:46:21.779760 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.779601    2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-98rxl"
Jul 01 20:46:21.780972 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.780954    2571 manager.go:354] Recovery completed
Jul 01 20:46:21.785841 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.785829    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 20:46:21.787557 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.787540    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-104.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 20:46:21.787651 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.787567    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-104.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 20:46:21.787651 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.787578    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-104.ec2.internal" event="NodeHasSufficientPID"
Jul 01 20:46:21.788088 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.788075    2571 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 01 20:46:21.788129 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.788085    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 20:46:21.788129 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.788087    2571 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 01 20:46:21.788129 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.788124    2571 state_mem.go:36] "Initialized new in-memory state store"
Jul 01 20:46:21.788764 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.788747    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-104.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 20:46:21.788842 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.788773    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-104.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 20:46:21.788842 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.788782    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-104.ec2.internal" event="NodeHasSufficientPID"
Jul 01 20:46:21.789403 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:21.789345    2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-104.ec2.internal.18be456028b702cf  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-104.ec2.internal,UID:ip-10-0-140-104.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-104.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-104.ec2.internal,},FirstTimestamp:2026-07-01 20:46:21.787554511 +0000 UTC m=+0.300029695,LastTimestamp:2026-07-01 20:46:21.787554511 +0000 UTC m=+0.300029695,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-104.ec2.internal,}"
Jul 01 20:46:21.790114 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.790101    2571 policy_none.go:49] "None policy: Start"
Jul 01 20:46:21.790181 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.790121    2571 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 01 20:46:21.790181 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.790134    2571 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 01 20:46:21.792234 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.792221    2571 policy_none.go:47] "Start"
Jul 01 20:46:21.825167 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.825155    2571 manager.go:337] "Starting Device Plugin manager"
Jul 01 20:46:21.845528 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:21.825188    2571 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 01 20:46:21.845528 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.825197    2571 server.go:87] "Starting device plugin registration server"
Jul 01 20:46:21.845528 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.825387    2571 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 01 20:46:21.845528 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.825399    2571 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 01 20:46:21.845528 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.825464    2571 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 01 20:46:21.845528 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.825533    2571 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 01 20:46:21.845528 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.825541    2571 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 01 20:46:21.845528 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:21.826050    2571 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 01 20:46:21.845528 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:21.826088    2571 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-104.ec2.internal\" not found"
Jul 01 20:46:21.845528 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.831505    2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 01 20:46:21.845528 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.832754    2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 01 20:46:21.845528 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.832781    2571 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 01 20:46:21.845528 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.832797    2571 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 01 20:46:21.845528 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:21.832824    2571 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 01 20:46:21.845528 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.834815    2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 20:46:21.926299 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.926282    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 20:46:21.927148 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.927132    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-104.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 20:46:21.927214 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.927161    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-104.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 20:46:21.927214 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.927171    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-104.ec2.internal" event="NodeHasSufficientPID"
Jul 01 20:46:21.927214 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.927191    2571 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-104.ec2.internal"
Jul 01 20:46:21.933710 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.933689    2571 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-140-104.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-104.ec2.internal"]
Jul 01 20:46:21.933782 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.933752    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 20:46:21.934424 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.934409    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-104.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 20:46:21.934482 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.934431    2571 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-104.ec2.internal"
Jul 01 20:46:21.934482 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.934435    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-104.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 20:46:21.934482 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.934445    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-104.ec2.internal" event="NodeHasSufficientPID"
Jul 01 20:46:21.934482 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:21.934446    2571 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-104.ec2.internal\": node \"ip-10-0-140-104.ec2.internal\" not found"
Jul 01 20:46:21.936756 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.936744    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 20:46:21.936857 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.936843    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-104.ec2.internal"
Jul 01 20:46:21.936889 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.936870    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 20:46:21.937407 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.937394    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-104.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 20:46:21.937474 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.937418    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-104.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 20:46:21.937474 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.937422    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-104.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 20:46:21.937474 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.937430    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-104.ec2.internal" event="NodeHasSufficientPID"
Jul 01 20:46:21.937474 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.937442    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-104.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 20:46:21.937474 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.937452    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-104.ec2.internal" event="NodeHasSufficientPID"
Jul 01 20:46:21.939743 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.939724    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-104.ec2.internal"
Jul 01 20:46:21.939827 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.939748    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 20:46:21.940666 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.940652    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-104.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 20:46:21.940747 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.940670    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-104.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 20:46:21.940747 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.940679    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-104.ec2.internal" event="NodeHasSufficientPID"
Jul 01 20:46:21.952582 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:21.952543    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-104.ec2.internal\" not found"
Jul 01 20:46:21.958216 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:21.958199    2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-104.ec2.internal\" not found" node="ip-10-0-140-104.ec2.internal"
Jul 01 20:46:21.962450 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:21.962430    2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-104.ec2.internal\" not found" node="ip-10-0-140-104.ec2.internal"
Jul 01 20:46:21.969602 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.969588    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/23a6143e19ae3dec90500595f50e9ee3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-104.ec2.internal\" (UID: \"23a6143e19ae3dec90500595f50e9ee3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-104.ec2.internal"
Jul 01 20:46:21.969669 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.969609    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/23a6143e19ae3dec90500595f50e9ee3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-104.ec2.internal\" (UID: \"23a6143e19ae3dec90500595f50e9ee3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-104.ec2.internal"
Jul 01 20:46:21.969738 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:21.969640    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/289aec27ab6274a41b4614fd9786336a-config\") pod \"kube-apiserver-proxy-ip-10-0-140-104.ec2.internal\" (UID: \"289aec27ab6274a41b4614fd9786336a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-104.ec2.internal"
Jul 01 20:46:22.053086 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:22.053065    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-104.ec2.internal\" not found"
Jul 01 20:46:22.070529 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:22.070509    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/289aec27ab6274a41b4614fd9786336a-config\") pod \"kube-apiserver-proxy-ip-10-0-140-104.ec2.internal\" (UID: \"289aec27ab6274a41b4614fd9786336a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-104.ec2.internal"
Jul 01 20:46:22.070572 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:22.070534    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/23a6143e19ae3dec90500595f50e9ee3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-104.ec2.internal\" (UID: \"23a6143e19ae3dec90500595f50e9ee3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-104.ec2.internal"
Jul 01 20:46:22.070572 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:22.070553    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/23a6143e19ae3dec90500595f50e9ee3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-104.ec2.internal\" (UID: \"23a6143e19ae3dec90500595f50e9ee3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-104.ec2.internal"
Jul 01 20:46:22.070663 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:22.070586    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/23a6143e19ae3dec90500595f50e9ee3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-104.ec2.internal\" (UID: \"23a6143e19ae3dec90500595f50e9ee3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-104.ec2.internal"
Jul 01 20:46:22.070663 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:22.070592    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/289aec27ab6274a41b4614fd9786336a-config\") pod \"kube-apiserver-proxy-ip-10-0-140-104.ec2.internal\" (UID: \"289aec27ab6274a41b4614fd9786336a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-104.ec2.internal"
Jul 01 20:46:22.070663 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:22.070622    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/23a6143e19ae3dec90500595f50e9ee3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-104.ec2.internal\" (UID: \"23a6143e19ae3dec90500595f50e9ee3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-104.ec2.internal"
Jul 01 20:46:22.154132 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:22.154109    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-104.ec2.internal\" not found"
Jul 01 20:46:22.254846 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:22.254801    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-104.ec2.internal\" not found"
Jul 01 20:46:22.261975 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:22.261957    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-104.ec2.internal"
Jul 01 20:46:22.266192 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:22.266035    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-104.ec2.internal"
Jul 01 20:46:22.355523 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:22.355497    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-104.ec2.internal\" not found"
Jul 01 20:46:22.455996 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:22.455964    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-104.ec2.internal\" not found"
Jul 01 20:46:22.556562 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:22.556517    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-104.ec2.internal\" not found"
Jul 01 20:46:22.657177 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:22.657156    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-104.ec2.internal\" not found"
Jul 01 20:46:22.693462 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:22.693441    2571 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 20:46:22.714291 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:22.714265    2571 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 01 20:46:22.714869 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:22.714382    2571 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 01 20:46:22.714869 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:22.714453    2571 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 01 20:46:22.757813 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:22.757789    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-104.ec2.internal\" not found"
Jul 01 20:46:22.784639 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:22.784600    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-30 20:41:22 +0000 UTC" deadline="2028-01-28 23:12:39.188027498 +0000 UTC"
Jul 01 20:46:22.784711 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:22.784640    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13826h26m16.403405128s"
Jul 01 20:46:22.858865 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:22.858842    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-104.ec2.internal\" not found"
Jul 01 20:46:22.959214 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:22.959185    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-104.ec2.internal\" not found"
Jul 01 20:46:23.004928 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.004904    2571 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 20:46:23.068597 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.068572    2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-104.ec2.internal"
Jul 01 20:46:23.080537 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.080518    2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 01 20:46:23.081336 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.081322    2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-104.ec2.internal"
Jul 01 20:46:23.091163 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.091143    2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 01 20:46:23.143270 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:23.143245    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23a6143e19ae3dec90500595f50e9ee3.slice/crio-ace8b1f59f79c1ca26da86ab7f659181fe94273839ee2e21373112523f04b8ca WatchSource:0}: Error finding container ace8b1f59f79c1ca26da86ab7f659181fe94273839ee2e21373112523f04b8ca: Status 404 returned error can't find the container with id ace8b1f59f79c1ca26da86ab7f659181fe94273839ee2e21373112523f04b8ca
Jul 01 20:46:23.143836 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:23.143817    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod289aec27ab6274a41b4614fd9786336a.slice/crio-9362819da4ca14995a8c25d8fee99eac3483acf5a63a0f15712ee753b705aa7d WatchSource:0}: Error finding container 9362819da4ca14995a8c25d8fee99eac3483acf5a63a0f15712ee753b705aa7d: Status 404 returned error can't find the container with id 9362819da4ca14995a8c25d8fee99eac3483acf5a63a0f15712ee753b705aa7d
Jul 01 20:46:23.150181 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.150167    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 01 20:46:23.346999 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.346981    2571 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 20:46:23.750074 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.750031    2571 apiserver.go:52] "Watching apiserver"
Jul 01 20:46:23.760645 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.760600    2571 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 01 20:46:23.761445 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.761096    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-140-104.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5hwh","openshift-cluster-node-tuning-operator/tuned-4x2pr","openshift-dns/node-resolver-ncfhm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-104.ec2.internal","openshift-multus/multus-m8mqn","openshift-network-operator/iptables-alerter-lhmmx","kube-system/global-pull-secret-syncer-t6lw6","openshift-image-registry/node-ca-swv7c","openshift-multus/multus-additional-cni-plugins-qbnbc","openshift-multus/network-metrics-daemon-cthwx","openshift-network-diagnostics/network-check-target-fz5ld","openshift-ovn-kubernetes/ovnkube-node-sw9rx","kube-system/konnectivity-agent-njzrp"]
Jul 01 20:46:23.763878 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.763854    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lhmmx"
Jul 01 20:46:23.767044 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.766237    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:46:23.767044 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:23.766315    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cthwx" podUID="3779f941-e08c-4e16-a1ab-be86f76ee83d"
Jul 01 20:46:23.767044 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.766636    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2stvp\""
Jul 01 20:46:23.767044 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.766885    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 01 20:46:23.767044 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.766917    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 01 20:46:23.767414 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.767145    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 01 20:46:23.771223 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.768701    2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 01 20:46:23.771223 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.769686    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.772581 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.772559    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ncfhm"
Jul 01 20:46:23.773043 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.772794    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4nc2v\""
Jul 01 20:46:23.773152 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.773103    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 01 20:46:23.773152 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.773104    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 01 20:46:23.775491 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.775473    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 01 20:46:23.775646 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.775630    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-c8hxm\""
Jul 01 20:46:23.775734 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.775635    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 01 20:46:23.776209 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.776190    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-njzrp"
Jul 01 20:46:23.779657 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.778549    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.779657 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.779084    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 01 20:46:23.779657 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.779116    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zb6pl\""
Jul 01 20:46:23.779657 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.779358    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 01 20:46:23.779657 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.779394    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/95b88ca8-8085-428f-a64d-b141a08f26ea-etc-modprobe-d\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.779657 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.779413    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/95b88ca8-8085-428f-a64d-b141a08f26ea-host\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.779657 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.779441    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/95b88ca8-8085-428f-a64d-b141a08f26ea-etc-sysconfig\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.779657 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.779460    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95b88ca8-8085-428f-a64d-b141a08f26ea-etc-kubernetes\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.779657 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.779475    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95b88ca8-8085-428f-a64d-b141a08f26ea-tmp\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.779657 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.779490    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsjjg\" (UniqueName: \"kubernetes.io/projected/95b88ca8-8085-428f-a64d-b141a08f26ea-kube-api-access-lsjjg\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.779657 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.779527    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/66cd16df-9b52-4df7-ac4c-ac525b188137-host-slash\") pod \"iptables-alerter-lhmmx\" (UID: \"66cd16df-9b52-4df7-ac4c-ac525b188137\") " pod="openshift-network-operator/iptables-alerter-lhmmx"
Jul 01 20:46:23.779657 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.779550    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3779f941-e08c-4e16-a1ab-be86f76ee83d-metrics-certs\") pod \"network-metrics-daemon-cthwx\" (UID: \"3779f941-e08c-4e16-a1ab-be86f76ee83d\") " pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:46:23.779657 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.779583    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/95b88ca8-8085-428f-a64d-b141a08f26ea-etc-sysctl-conf\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.779657 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.779600    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/95b88ca8-8085-428f-a64d-b141a08f26ea-etc-systemd\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.779657 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.779621    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/95b88ca8-8085-428f-a64d-b141a08f26ea-lib-modules\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.779657 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.779639    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95b88ca8-8085-428f-a64d-b141a08f26ea-var-lib-kubelet\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.780506 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.779677    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/95b88ca8-8085-428f-a64d-b141a08f26ea-etc-tuned\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.780506 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.779760    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/66cd16df-9b52-4df7-ac4c-ac525b188137-iptables-alerter-script\") pod \"iptables-alerter-lhmmx\" (UID: \"66cd16df-9b52-4df7-ac4c-ac525b188137\") " pod="openshift-network-operator/iptables-alerter-lhmmx"
Jul 01 20:46:23.780506 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.779799    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mddc\" (UniqueName: \"kubernetes.io/projected/66cd16df-9b52-4df7-ac4c-ac525b188137-kube-api-access-8mddc\") pod \"iptables-alerter-lhmmx\" (UID: \"66cd16df-9b52-4df7-ac4c-ac525b188137\") " pod="openshift-network-operator/iptables-alerter-lhmmx"
Jul 01 20:46:23.780506 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.779815    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/95b88ca8-8085-428f-a64d-b141a08f26ea-etc-sysctl-d\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.780506 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.779833    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/95b88ca8-8085-428f-a64d-b141a08f26ea-run\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.780506 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.779856    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/95b88ca8-8085-428f-a64d-b141a08f26ea-sys\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.780506 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.779884    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mjzp\" (UniqueName: \"kubernetes.io/projected/3779f941-e08c-4e16-a1ab-be86f76ee83d-kube-api-access-8mjzp\") pod \"network-metrics-daemon-cthwx\" (UID: \"3779f941-e08c-4e16-a1ab-be86f76ee83d\") " pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:46:23.781060 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.781040    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-swv7c"
Jul 01 20:46:23.781266 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.781248    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 01 20:46:23.784345 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.781675    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 01 20:46:23.784345 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.781687    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 01 20:46:23.784345 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.781697    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-lnwpp\""
Jul 01 20:46:23.784345 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.781918    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 01 20:46:23.784345 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.783659    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 01 20:46:23.784345 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.784297    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.784712 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.784365    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 01 20:46:23.784712 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.784586    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-bmr8d\""
Jul 01 20:46:23.784922 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.784900    2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 01 20:46:23.785158 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.785133    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 01 20:46:23.786419 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.786399    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5hwh"
Jul 01 20:46:23.787644 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.787625    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 01 20:46:23.787752 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.787717    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 01 20:46:23.787818 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.787627    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-x6mpj\""
Jul 01 20:46:23.790167 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.790145    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 01 20:46:23.790453 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.790437    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 01 20:46:23.790786 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.790769    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 01 20:46:23.791083 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.791061    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 01 20:46:23.791355 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.791336    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 01 20:46:23.792257 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.792235    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 01 20:46:23.792569 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.792551    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-p5m62\""
Jul 01 20:46:23.792862 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.792844    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 01 20:46:23.795914 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.795897    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qbnbc"
Jul 01 20:46:23.799459 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.799443    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 01 20:46:23.799546 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.799506    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 01 20:46:23.799687 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.799653    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-f5r6k\""
Jul 01 20:46:23.799955 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.799941    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:46:23.800038 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:23.800017    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lw6" podUID="1dfa26e7-23bf-4e56-808d-e59cee52aee7"
Jul 01 20:46:23.803905 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.803886    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:46:23.804014 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:23.803951    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fz5ld" podUID="a02110cb-ee26-4f3e-a854-ececca41783e"
Jul 01 20:46:23.808534 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.808512    2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-pwvbk"
Jul 01 20:46:23.816589 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.816570    2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pwvbk"
Jul 01 20:46:23.842288 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.842247    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-104.ec2.internal" event={"ID":"289aec27ab6274a41b4614fd9786336a","Type":"ContainerStarted","Data":"9362819da4ca14995a8c25d8fee99eac3483acf5a63a0f15712ee753b705aa7d"}
Jul 01 20:46:23.846033 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.846003    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-104.ec2.internal" event={"ID":"23a6143e19ae3dec90500595f50e9ee3","Type":"ContainerStarted","Data":"ace8b1f59f79c1ca26da86ab7f659181fe94273839ee2e21373112523f04b8ca"}
Jul 01 20:46:23.869600 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.869582    2571 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 01 20:46:23.880318 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.880287    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mjzp\" (UniqueName: \"kubernetes.io/projected/3779f941-e08c-4e16-a1ab-be86f76ee83d-kube-api-access-8mjzp\") pod \"network-metrics-daemon-cthwx\" (UID: \"3779f941-e08c-4e16-a1ab-be86f76ee83d\") " pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:46:23.880416 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.880324    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-run-openvswitch\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.880416 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.880356    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9c3897a7-ac96-41d8-95fe-8101c9933ddf-device-dir\") pod \"aws-ebs-csi-driver-node-n5hwh\" (UID: \"9c3897a7-ac96-41d8-95fe-8101c9933ddf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5hwh"
Jul 01 20:46:23.880416 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.880381    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-multus-cni-dir\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.880416 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.880406    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-host-run-k8s-cni-cncf-io\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.880595 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.880432    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9a45d9e0-af21-4075-9578-09d1913a1082-os-release\") pod \"multus-additional-cni-plugins-qbnbc\" (UID: \"9a45d9e0-af21-4075-9578-09d1913a1082\") " pod="openshift-multus/multus-additional-cni-plugins-qbnbc"
Jul 01 20:46:23.880595 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.880458    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-ovnkube-script-lib\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.880595 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.880481    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9c3897a7-ac96-41d8-95fe-8101c9933ddf-kubelet-dir\") pod \"aws-ebs-csi-driver-node-n5hwh\" (UID: \"9c3897a7-ac96-41d8-95fe-8101c9933ddf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5hwh"
Jul 01 20:46:23.880595 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.880504    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1dfa26e7-23bf-4e56-808d-e59cee52aee7-dbus\") pod \"global-pull-secret-syncer-t6lw6\" (UID: \"1dfa26e7-23bf-4e56-808d-e59cee52aee7\") " pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:46:23.880595 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.880530    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6cb9481e-9331-4e53-abc4-3336c04034c2-agent-certs\") pod \"konnectivity-agent-njzrp\" (UID: \"6cb9481e-9331-4e53-abc4-3336c04034c2\") " pod="kube-system/konnectivity-agent-njzrp"
Jul 01 20:46:23.880595 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.880566    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-os-release\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.880912 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.880590    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6zb8\" (UniqueName: \"kubernetes.io/projected/bc52f915-112e-4617-b476-01b645a52eb3-kube-api-access-m6zb8\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.880912 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.880624    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a45d9e0-af21-4075-9578-09d1913a1082-cnibin\") pod \"multus-additional-cni-plugins-qbnbc\" (UID: \"9a45d9e0-af21-4075-9578-09d1913a1082\") " pod="openshift-multus/multus-additional-cni-plugins-qbnbc"
Jul 01 20:46:23.880912 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.880639    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eaa37651-7fb2-4625-a15d-e7427a744196-tmp-dir\") pod \"node-resolver-ncfhm\" (UID: \"eaa37651-7fb2-4625-a15d-e7427a744196\") " pod="openshift-dns/node-resolver-ncfhm"
Jul 01 20:46:23.880912 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.880665    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/95b88ca8-8085-428f-a64d-b141a08f26ea-etc-sysctl-conf\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.880912 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.880701    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/95b88ca8-8085-428f-a64d-b141a08f26ea-lib-modules\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.880912 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.880746    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-etc-openvswitch\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.880912 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.880770    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-env-overrides\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.880912 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.880798    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5czp7\" (UniqueName: \"kubernetes.io/projected/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-kube-api-access-5czp7\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.880912 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.880825    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ws6hp\" (UniqueName: \"kubernetes.io/projected/111e269e-a761-4eee-9a78-3fd554431c39-kube-api-access-ws6hp\") pod \"node-ca-swv7c\" (UID: \"111e269e-a761-4eee-9a78-3fd554431c39\") " pod="openshift-image-registry/node-ca-swv7c"
Jul 01 20:46:23.880912 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.880861    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9a45d9e0-af21-4075-9578-09d1913a1082-cni-binary-copy\") pod \"multus-additional-cni-plugins-qbnbc\" (UID: \"9a45d9e0-af21-4075-9578-09d1913a1082\") " pod="openshift-multus/multus-additional-cni-plugins-qbnbc"
Jul 01 20:46:23.880912 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.880889    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mddc\" (UniqueName: \"kubernetes.io/projected/66cd16df-9b52-4df7-ac4c-ac525b188137-kube-api-access-8mddc\") pod \"iptables-alerter-lhmmx\" (UID: \"66cd16df-9b52-4df7-ac4c-ac525b188137\") " pod="openshift-network-operator/iptables-alerter-lhmmx"
Jul 01 20:46:23.881349 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.880929    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/95b88ca8-8085-428f-a64d-b141a08f26ea-etc-sysctl-d\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.881349 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.880966    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-run-systemd\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.881349 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.880990    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-host-cni-netd\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.881349 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881014    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9c3897a7-ac96-41d8-95fe-8101c9933ddf-socket-dir\") pod \"aws-ebs-csi-driver-node-n5hwh\" (UID: \"9c3897a7-ac96-41d8-95fe-8101c9933ddf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5hwh"
Jul 01 20:46:23.881349 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881042    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/eaa37651-7fb2-4625-a15d-e7427a744196-hosts-file\") pod \"node-resolver-ncfhm\" (UID: \"eaa37651-7fb2-4625-a15d-e7427a744196\") " pod="openshift-dns/node-resolver-ncfhm"
Jul 01 20:46:23.881349 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881066    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-host-var-lib-kubelet\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.881349 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881092    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbmgs\" (UniqueName: \"kubernetes.io/projected/eaa37651-7fb2-4625-a15d-e7427a744196-kube-api-access-qbmgs\") pod \"node-resolver-ncfhm\" (UID: \"eaa37651-7fb2-4625-a15d-e7427a744196\") " pod="openshift-dns/node-resolver-ncfhm"
Jul 01 20:46:23.881349 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881117    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/95b88ca8-8085-428f-a64d-b141a08f26ea-host\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.881349 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881142    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-run-ovn\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.881349 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881168    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-log-socket\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.881349 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881201    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-host-run-ovn-kubernetes\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.881349 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881336    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-ovnkube-config\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.881968 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881366    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-multus-socket-dir-parent\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.881968 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881391    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bc52f915-112e-4617-b476-01b645a52eb3-multus-daemon-config\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.881968 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881419    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-host-kubelet\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.881968 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881443    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-systemd-units\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.881968 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881459    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-host-slash\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.881968 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881475    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-system-cni-dir\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.881968 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881494    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-host-run-netns\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.881968 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881511    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/66cd16df-9b52-4df7-ac4c-ac525b188137-host-slash\") pod \"iptables-alerter-lhmmx\" (UID: \"66cd16df-9b52-4df7-ac4c-ac525b188137\") " pod="openshift-network-operator/iptables-alerter-lhmmx"
Jul 01 20:46:23.881968 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881527    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cj4fn\" (UniqueName: \"kubernetes.io/projected/a02110cb-ee26-4f3e-a854-ececca41783e-kube-api-access-cj4fn\") pod \"network-check-target-fz5ld\" (UID: \"a02110cb-ee26-4f3e-a854-ececca41783e\") " pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:46:23.881968 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881558    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-cnibin\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.881968 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881614    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/111e269e-a761-4eee-9a78-3fd554431c39-serviceca\") pod \"node-ca-swv7c\" (UID: \"111e269e-a761-4eee-9a78-3fd554431c39\") " pod="openshift-image-registry/node-ca-swv7c"
Jul 01 20:46:23.881968 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881614    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/95b88ca8-8085-428f-a64d-b141a08f26ea-etc-sysctl-d\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.881968 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881642    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-var-lib-openvswitch\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.881968 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881672    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9jkh\" (UniqueName: \"kubernetes.io/projected/9c3897a7-ac96-41d8-95fe-8101c9933ddf-kube-api-access-b9jkh\") pod \"aws-ebs-csi-driver-node-n5hwh\" (UID: \"9c3897a7-ac96-41d8-95fe-8101c9933ddf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5hwh"
Jul 01 20:46:23.881968 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881717    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-host-var-lib-cni-bin\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.881968 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881745    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a45d9e0-af21-4075-9578-09d1913a1082-system-cni-dir\") pod \"multus-additional-cni-plugins-qbnbc\" (UID: \"9a45d9e0-af21-4075-9578-09d1913a1082\") " pod="openshift-multus/multus-additional-cni-plugins-qbnbc"
Jul 01 20:46:23.881968 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881769    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/95b88ca8-8085-428f-a64d-b141a08f26ea-etc-sysctl-conf\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.882751 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881775    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9a45d9e0-af21-4075-9578-09d1913a1082-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qbnbc\" (UID: \"9a45d9e0-af21-4075-9578-09d1913a1082\") " pod="openshift-multus/multus-additional-cni-plugins-qbnbc"
Jul 01 20:46:23.882751 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881806    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/95b88ca8-8085-428f-a64d-b141a08f26ea-host\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.882751 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881804    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9jj8\" (UniqueName: \"kubernetes.io/projected/9a45d9e0-af21-4075-9578-09d1913a1082-kube-api-access-r9jj8\") pod \"multus-additional-cni-plugins-qbnbc\" (UID: \"9a45d9e0-af21-4075-9578-09d1913a1082\") " pod="openshift-multus/multus-additional-cni-plugins-qbnbc"
Jul 01 20:46:23.882751 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881842    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-node-log\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.882751 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881861    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-ovn-node-metrics-cert\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.882751 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881871    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/66cd16df-9b52-4df7-ac4c-ac525b188137-host-slash\") pod \"iptables-alerter-lhmmx\" (UID: \"66cd16df-9b52-4df7-ac4c-ac525b188137\") " pod="openshift-network-operator/iptables-alerter-lhmmx"
Jul 01 20:46:23.882751 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881880    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-hostroot\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.882751 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881903    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/95b88ca8-8085-428f-a64d-b141a08f26ea-etc-sysconfig\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.882751 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881921    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95b88ca8-8085-428f-a64d-b141a08f26ea-etc-kubernetes\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.882751 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881938    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95b88ca8-8085-428f-a64d-b141a08f26ea-tmp\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.882751 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.881968    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsjjg\" (UniqueName: \"kubernetes.io/projected/95b88ca8-8085-428f-a64d-b141a08f26ea-kube-api-access-lsjjg\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.882751 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.882045    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/95b88ca8-8085-428f-a64d-b141a08f26ea-etc-sysconfig\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.882751 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.882078    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-host-cni-bin\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.882751 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.882105    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1dfa26e7-23bf-4e56-808d-e59cee52aee7-original-pull-secret\") pod \"global-pull-secret-syncer-t6lw6\" (UID: \"1dfa26e7-23bf-4e56-808d-e59cee52aee7\") " pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:46:23.882751 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.882137    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/95b88ca8-8085-428f-a64d-b141a08f26ea-etc-systemd\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.882751 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.882170    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/95b88ca8-8085-428f-a64d-b141a08f26ea-etc-tuned\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.882751 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.882196    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9c3897a7-ac96-41d8-95fe-8101c9933ddf-etc-selinux\") pod \"aws-ebs-csi-driver-node-n5hwh\" (UID: \"9c3897a7-ac96-41d8-95fe-8101c9933ddf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5hwh"
Jul 01 20:46:23.883335 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.882222    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bc52f915-112e-4617-b476-01b645a52eb3-cni-binary-copy\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.883335 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.882209    2571 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 01 20:46:23.883335 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.882247    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-multus-conf-dir\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.883335 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.882273    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-host-run-multus-certs\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.883335 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.882298    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/111e269e-a761-4eee-9a78-3fd554431c39-host\") pod \"node-ca-swv7c\" (UID: \"111e269e-a761-4eee-9a78-3fd554431c39\") " pod="openshift-image-registry/node-ca-swv7c"
Jul 01 20:46:23.883335 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.882403    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a45d9e0-af21-4075-9578-09d1913a1082-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qbnbc\" (UID: \"9a45d9e0-af21-4075-9578-09d1913a1082\") " pod="openshift-multus/multus-additional-cni-plugins-qbnbc"
Jul 01 20:46:23.883335 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.882469    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/95b88ca8-8085-428f-a64d-b141a08f26ea-run\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.883335 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.882497    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9c3897a7-ac96-41d8-95fe-8101c9933ddf-registration-dir\") pod \"aws-ebs-csi-driver-node-n5hwh\" (UID: \"9c3897a7-ac96-41d8-95fe-8101c9933ddf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5hwh"
Jul 01 20:46:23.883335 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.882524    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-etc-kubernetes\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.883335 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.882555    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9a45d9e0-af21-4075-9578-09d1913a1082-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qbnbc\" (UID: \"9a45d9e0-af21-4075-9578-09d1913a1082\") " pod="openshift-multus/multus-additional-cni-plugins-qbnbc"
Jul 01 20:46:23.883335 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.882590    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/95b88ca8-8085-428f-a64d-b141a08f26ea-etc-modprobe-d\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.883335 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.882622    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.883335 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.882653    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3779f941-e08c-4e16-a1ab-be86f76ee83d-metrics-certs\") pod \"network-metrics-daemon-cthwx\" (UID: \"3779f941-e08c-4e16-a1ab-be86f76ee83d\") " pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:46:23.883335 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.882679    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95b88ca8-8085-428f-a64d-b141a08f26ea-var-lib-kubelet\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.883335 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.882722    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1dfa26e7-23bf-4e56-808d-e59cee52aee7-kubelet-config\") pod \"global-pull-secret-syncer-t6lw6\" (UID: \"1dfa26e7-23bf-4e56-808d-e59cee52aee7\") " pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:46:23.883335 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.882748    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-host-var-lib-cni-multus\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.883335 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.882777    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/66cd16df-9b52-4df7-ac4c-ac525b188137-iptables-alerter-script\") pod \"iptables-alerter-lhmmx\" (UID: \"66cd16df-9b52-4df7-ac4c-ac525b188137\") " pod="openshift-network-operator/iptables-alerter-lhmmx"
Jul 01 20:46:23.884123 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.882784    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95b88ca8-8085-428f-a64d-b141a08f26ea-etc-kubernetes\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.884123 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.882804    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/95b88ca8-8085-428f-a64d-b141a08f26ea-sys\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.884123 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.882885    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95b88ca8-8085-428f-a64d-b141a08f26ea-var-lib-kubelet\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.884123 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.882952    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/95b88ca8-8085-428f-a64d-b141a08f26ea-lib-modules\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.884123 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.882983    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-host-run-netns\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.884123 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.883010    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9c3897a7-ac96-41d8-95fe-8101c9933ddf-sys-fs\") pod \"aws-ebs-csi-driver-node-n5hwh\" (UID: \"9c3897a7-ac96-41d8-95fe-8101c9933ddf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5hwh"
Jul 01 20:46:23.884123 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.883038    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6cb9481e-9331-4e53-abc4-3336c04034c2-konnectivity-ca\") pod \"konnectivity-agent-njzrp\" (UID: \"6cb9481e-9331-4e53-abc4-3336c04034c2\") " pod="kube-system/konnectivity-agent-njzrp"
Jul 01 20:46:23.884123 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.883109    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/95b88ca8-8085-428f-a64d-b141a08f26ea-sys\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.884123 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.883145    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/95b88ca8-8085-428f-a64d-b141a08f26ea-etc-systemd\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.884123 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.883181    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/95b88ca8-8085-428f-a64d-b141a08f26ea-run\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.884123 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.883269    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/95b88ca8-8085-428f-a64d-b141a08f26ea-etc-modprobe-d\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.884123 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.883899    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/66cd16df-9b52-4df7-ac4c-ac525b188137-iptables-alerter-script\") pod \"iptables-alerter-lhmmx\" (UID: \"66cd16df-9b52-4df7-ac4c-ac525b188137\") " pod="openshift-network-operator/iptables-alerter-lhmmx"
Jul 01 20:46:23.884123 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:23.884015    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 20:46:23.884123 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:23.884078    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3779f941-e08c-4e16-a1ab-be86f76ee83d-metrics-certs podName:3779f941-e08c-4e16-a1ab-be86f76ee83d nodeName:}" failed. No retries permitted until 2026-07-01 20:46:24.384056898 +0000 UTC m=+2.896532087 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3779f941-e08c-4e16-a1ab-be86f76ee83d-metrics-certs") pod "network-metrics-daemon-cthwx" (UID: "3779f941-e08c-4e16-a1ab-be86f76ee83d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 20:46:23.887741 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.887717    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95b88ca8-8085-428f-a64d-b141a08f26ea-tmp\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.888137 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.888112    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/95b88ca8-8085-428f-a64d-b141a08f26ea-etc-tuned\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.892594 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.892570    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsjjg\" (UniqueName: \"kubernetes.io/projected/95b88ca8-8085-428f-a64d-b141a08f26ea-kube-api-access-lsjjg\") pod \"tuned-4x2pr\" (UID: \"95b88ca8-8085-428f-a64d-b141a08f26ea\") " pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:23.893089 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.893066    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mddc\" (UniqueName: \"kubernetes.io/projected/66cd16df-9b52-4df7-ac4c-ac525b188137-kube-api-access-8mddc\") pod \"iptables-alerter-lhmmx\" (UID: \"66cd16df-9b52-4df7-ac4c-ac525b188137\") " pod="openshift-network-operator/iptables-alerter-lhmmx"
Jul 01 20:46:23.894386 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.894363    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mjzp\" (UniqueName: \"kubernetes.io/projected/3779f941-e08c-4e16-a1ab-be86f76ee83d-kube-api-access-8mjzp\") pod \"network-metrics-daemon-cthwx\" (UID: \"3779f941-e08c-4e16-a1ab-be86f76ee83d\") " pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:46:23.987054 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.987026    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-ovnkube-config\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.987167 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.987070    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-multus-socket-dir-parent\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.987167 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.987097    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bc52f915-112e-4617-b476-01b645a52eb3-multus-daemon-config\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.987167 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.987122    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-host-kubelet\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.987167 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.987148    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-systemd-units\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.987409 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.987174    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-host-slash\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.987409 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.987200    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-system-cni-dir\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.987409 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.987226    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-host-run-netns\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.987409 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.987264    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj4fn\" (UniqueName: \"kubernetes.io/projected/a02110cb-ee26-4f3e-a854-ececca41783e-kube-api-access-cj4fn\") pod \"network-check-target-fz5ld\" (UID: \"a02110cb-ee26-4f3e-a854-ececca41783e\") " pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:46:23.987409 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.987289    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-cnibin\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.987409 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.987313    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/111e269e-a761-4eee-9a78-3fd554431c39-serviceca\") pod \"node-ca-swv7c\" (UID: \"111e269e-a761-4eee-9a78-3fd554431c39\") " pod="openshift-image-registry/node-ca-swv7c"
Jul 01 20:46:23.987409 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.987339    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-var-lib-openvswitch\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.987409 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.987366    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9jkh\" (UniqueName: \"kubernetes.io/projected/9c3897a7-ac96-41d8-95fe-8101c9933ddf-kube-api-access-b9jkh\") pod \"aws-ebs-csi-driver-node-n5hwh\" (UID: \"9c3897a7-ac96-41d8-95fe-8101c9933ddf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5hwh"
Jul 01 20:46:23.987409 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.987390    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-host-var-lib-cni-bin\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.987871 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.987414    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a45d9e0-af21-4075-9578-09d1913a1082-system-cni-dir\") pod \"multus-additional-cni-plugins-qbnbc\" (UID: \"9a45d9e0-af21-4075-9578-09d1913a1082\") " pod="openshift-multus/multus-additional-cni-plugins-qbnbc"
Jul 01 20:46:23.987871 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.987443    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9a45d9e0-af21-4075-9578-09d1913a1082-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qbnbc\" (UID: \"9a45d9e0-af21-4075-9578-09d1913a1082\") " pod="openshift-multus/multus-additional-cni-plugins-qbnbc"
Jul 01 20:46:23.987871 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.987472    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9jj8\" (UniqueName: \"kubernetes.io/projected/9a45d9e0-af21-4075-9578-09d1913a1082-kube-api-access-r9jj8\") pod \"multus-additional-cni-plugins-qbnbc\" (UID: \"9a45d9e0-af21-4075-9578-09d1913a1082\") " pod="openshift-multus/multus-additional-cni-plugins-qbnbc"
Jul 01 20:46:23.987871 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.987500    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-node-log\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.987871 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.987527    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-ovn-node-metrics-cert\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.987871 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.987553    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-hostroot\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.987871 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.987584    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-host-cni-bin\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.987871 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.987608    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1dfa26e7-23bf-4e56-808d-e59cee52aee7-original-pull-secret\") pod \"global-pull-secret-syncer-t6lw6\" (UID: \"1dfa26e7-23bf-4e56-808d-e59cee52aee7\") " pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:46:23.987871 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.987633    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9c3897a7-ac96-41d8-95fe-8101c9933ddf-etc-selinux\") pod \"aws-ebs-csi-driver-node-n5hwh\" (UID: \"9c3897a7-ac96-41d8-95fe-8101c9933ddf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5hwh"
Jul 01 20:46:23.987871 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.987655    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bc52f915-112e-4617-b476-01b645a52eb3-cni-binary-copy\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.987871 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.987676    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-multus-conf-dir\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.987871 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.987722    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-host-run-multus-certs\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.987871 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.987744    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/111e269e-a761-4eee-9a78-3fd554431c39-host\") pod \"node-ca-swv7c\" (UID: \"111e269e-a761-4eee-9a78-3fd554431c39\") " pod="openshift-image-registry/node-ca-swv7c"
Jul 01 20:46:23.987871 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.987770    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a45d9e0-af21-4075-9578-09d1913a1082-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qbnbc\" (UID: \"9a45d9e0-af21-4075-9578-09d1913a1082\") " pod="openshift-multus/multus-additional-cni-plugins-qbnbc"
Jul 01 20:46:23.987871 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.987796    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9c3897a7-ac96-41d8-95fe-8101c9933ddf-registration-dir\") pod \"aws-ebs-csi-driver-node-n5hwh\" (UID: \"9c3897a7-ac96-41d8-95fe-8101c9933ddf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5hwh"
Jul 01 20:46:23.987871 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.987821    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-etc-kubernetes\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.987871 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.987847    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9a45d9e0-af21-4075-9578-09d1913a1082-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qbnbc\" (UID: \"9a45d9e0-af21-4075-9578-09d1913a1082\") " pod="openshift-multus/multus-additional-cni-plugins-qbnbc"
Jul 01 20:46:23.988502 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.987880    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.988502 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.987924    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1dfa26e7-23bf-4e56-808d-e59cee52aee7-kubelet-config\") pod \"global-pull-secret-syncer-t6lw6\" (UID: \"1dfa26e7-23bf-4e56-808d-e59cee52aee7\") " pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:46:23.988502 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.987952    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-host-var-lib-cni-multus\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.988502 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.987983    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-host-run-netns\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.988502 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.988007    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9c3897a7-ac96-41d8-95fe-8101c9933ddf-sys-fs\") pod \"aws-ebs-csi-driver-node-n5hwh\" (UID: \"9c3897a7-ac96-41d8-95fe-8101c9933ddf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5hwh"
Jul 01 20:46:23.988502 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.988032    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6cb9481e-9331-4e53-abc4-3336c04034c2-konnectivity-ca\") pod \"konnectivity-agent-njzrp\" (UID: \"6cb9481e-9331-4e53-abc4-3336c04034c2\") " pod="kube-system/konnectivity-agent-njzrp"
Jul 01 20:46:23.988502 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.988060    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-run-openvswitch\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.988502 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.988086    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9c3897a7-ac96-41d8-95fe-8101c9933ddf-device-dir\") pod \"aws-ebs-csi-driver-node-n5hwh\" (UID: \"9c3897a7-ac96-41d8-95fe-8101c9933ddf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5hwh"
Jul 01 20:46:23.988502 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.988116    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-multus-cni-dir\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.988502 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.988141    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-host-run-k8s-cni-cncf-io\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.988502 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.988166    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9a45d9e0-af21-4075-9578-09d1913a1082-os-release\") pod \"multus-additional-cni-plugins-qbnbc\" (UID: \"9a45d9e0-af21-4075-9578-09d1913a1082\") " pod="openshift-multus/multus-additional-cni-plugins-qbnbc"
Jul 01 20:46:23.988502 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.988197    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-ovnkube-script-lib\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.988502 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.988261    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9c3897a7-ac96-41d8-95fe-8101c9933ddf-kubelet-dir\") pod \"aws-ebs-csi-driver-node-n5hwh\" (UID: \"9c3897a7-ac96-41d8-95fe-8101c9933ddf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5hwh"
Jul 01 20:46:23.988502 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.988271    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-ovnkube-config\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.988502 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.988287    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1dfa26e7-23bf-4e56-808d-e59cee52aee7-dbus\") pod \"global-pull-secret-syncer-t6lw6\" (UID: \"1dfa26e7-23bf-4e56-808d-e59cee52aee7\") " pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:46:23.988502 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.988314    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6cb9481e-9331-4e53-abc4-3336c04034c2-agent-certs\") pod \"konnectivity-agent-njzrp\" (UID: \"6cb9481e-9331-4e53-abc4-3336c04034c2\") " pod="kube-system/konnectivity-agent-njzrp"
Jul 01 20:46:23.988502 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.988341    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-os-release\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.989231 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.988369    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6zb8\" (UniqueName: \"kubernetes.io/projected/bc52f915-112e-4617-b476-01b645a52eb3-kube-api-access-m6zb8\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.989231 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.988394    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a45d9e0-af21-4075-9578-09d1913a1082-cnibin\") pod \"multus-additional-cni-plugins-qbnbc\" (UID: \"9a45d9e0-af21-4075-9578-09d1913a1082\") " pod="openshift-multus/multus-additional-cni-plugins-qbnbc"
Jul 01 20:46:23.989231 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.988419    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eaa37651-7fb2-4625-a15d-e7427a744196-tmp-dir\") pod \"node-resolver-ncfhm\" (UID: \"eaa37651-7fb2-4625-a15d-e7427a744196\") " pod="openshift-dns/node-resolver-ncfhm"
Jul 01 20:46:23.989231 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.988451    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-etc-openvswitch\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.989231 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.988476    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-env-overrides\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.989231 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.988504    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5czp7\" (UniqueName: \"kubernetes.io/projected/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-kube-api-access-5czp7\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.989231 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.988532    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ws6hp\" (UniqueName: \"kubernetes.io/projected/111e269e-a761-4eee-9a78-3fd554431c39-kube-api-access-ws6hp\") pod \"node-ca-swv7c\" (UID: \"111e269e-a761-4eee-9a78-3fd554431c39\") " pod="openshift-image-registry/node-ca-swv7c"
Jul 01 20:46:23.989231 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.988558    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9a45d9e0-af21-4075-9578-09d1913a1082-cni-binary-copy\") pod \"multus-additional-cni-plugins-qbnbc\" (UID: \"9a45d9e0-af21-4075-9578-09d1913a1082\") " pod="openshift-multus/multus-additional-cni-plugins-qbnbc"
Jul 01 20:46:23.989231 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.988591    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-run-systemd\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.989231 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.988616    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-host-cni-netd\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.989231 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.988643    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9c3897a7-ac96-41d8-95fe-8101c9933ddf-socket-dir\") pod \"aws-ebs-csi-driver-node-n5hwh\" (UID: \"9c3897a7-ac96-41d8-95fe-8101c9933ddf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5hwh"
Jul 01 20:46:23.989231 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.988671    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/eaa37651-7fb2-4625-a15d-e7427a744196-hosts-file\") pod \"node-resolver-ncfhm\" (UID: \"eaa37651-7fb2-4625-a15d-e7427a744196\") " pod="openshift-dns/node-resolver-ncfhm"
Jul 01 20:46:23.989231 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.988718    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-host-var-lib-kubelet\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.989231 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.988748    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbmgs\" (UniqueName: \"kubernetes.io/projected/eaa37651-7fb2-4625-a15d-e7427a744196-kube-api-access-qbmgs\") pod \"node-resolver-ncfhm\" (UID: \"eaa37651-7fb2-4625-a15d-e7427a744196\") " pod="openshift-dns/node-resolver-ncfhm"
Jul 01 20:46:23.989231 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.988780    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-run-ovn\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.989231 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.988792    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bc52f915-112e-4617-b476-01b645a52eb3-multus-daemon-config\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.989231 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.988806    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-log-socket\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.989838 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.988833    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-host-run-ovn-kubernetes\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.989838 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.988848    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-host-kubelet\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.989838 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.988949    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-var-lib-openvswitch\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.989838 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.988982    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-hostroot\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.989838 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.989018    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-host-cni-bin\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.989838 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:23.989097    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 20:46:23.989838 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.989143    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-host-var-lib-cni-bin\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.989838 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:23.989157    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dfa26e7-23bf-4e56-808d-e59cee52aee7-original-pull-secret podName:1dfa26e7-23bf-4e56-808d-e59cee52aee7 nodeName:}" failed. No retries permitted until 2026-07-01 20:46:24.489137211 +0000 UTC m=+3.001612395 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1dfa26e7-23bf-4e56-808d-e59cee52aee7-original-pull-secret") pod "global-pull-secret-syncer-t6lw6" (UID: "1dfa26e7-23bf-4e56-808d-e59cee52aee7") : object "kube-system"/"original-pull-secret" not registered
Jul 01 20:46:23.989838 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.989182    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a45d9e0-af21-4075-9578-09d1913a1082-system-cni-dir\") pod \"multus-additional-cni-plugins-qbnbc\" (UID: \"9a45d9e0-af21-4075-9578-09d1913a1082\") " pod="openshift-multus/multus-additional-cni-plugins-qbnbc"
Jul 01 20:46:23.989838 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.989252    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9c3897a7-ac96-41d8-95fe-8101c9933ddf-etc-selinux\") pod \"aws-ebs-csi-driver-node-n5hwh\" (UID: \"9c3897a7-ac96-41d8-95fe-8101c9933ddf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5hwh"
Jul 01 20:46:23.989838 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.989428    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-multus-socket-dir-parent\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.989838 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.989680    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9a45d9e0-af21-4075-9578-09d1913a1082-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qbnbc\" (UID: \"9a45d9e0-af21-4075-9578-09d1913a1082\") " pod="openshift-multus/multus-additional-cni-plugins-qbnbc"
Jul 01 20:46:23.989838 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.989801    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-os-release\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.990291 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.990013    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a45d9e0-af21-4075-9578-09d1913a1082-cnibin\") pod \"multus-additional-cni-plugins-qbnbc\" (UID: \"9a45d9e0-af21-4075-9578-09d1913a1082\") " pod="openshift-multus/multus-additional-cni-plugins-qbnbc"
Jul 01 20:46:23.990291 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.990268    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eaa37651-7fb2-4625-a15d-e7427a744196-tmp-dir\") pod \"node-resolver-ncfhm\" (UID: \"eaa37651-7fb2-4625-a15d-e7427a744196\") " pod="openshift-dns/node-resolver-ncfhm"
Jul 01 20:46:23.990377 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.990290    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-etc-openvswitch\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.990377 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.990314    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-run-ovn\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.990377 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.990358    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-log-socket\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.990493 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.990392    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-host-run-ovn-kubernetes\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.990493 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.990429    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-systemd-units\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.990493 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.990464    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-host-slash\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.990644 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.990513    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-system-cni-dir\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.990644 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.990549    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-host-run-netns\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.990766 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.990652    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-env-overrides\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.990818 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.990775    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1dfa26e7-23bf-4e56-808d-e59cee52aee7-kubelet-config\") pod \"global-pull-secret-syncer-t6lw6\" (UID: \"1dfa26e7-23bf-4e56-808d-e59cee52aee7\") " pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:46:23.990870 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.990824    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-node-log\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.990870 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.990832    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-cnibin\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.990969 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.990910    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a45d9e0-af21-4075-9578-09d1913a1082-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qbnbc\" (UID: \"9a45d9e0-af21-4075-9578-09d1913a1082\") " pod="openshift-multus/multus-additional-cni-plugins-qbnbc"
Jul 01 20:46:23.990969 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.990912    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-multus-conf-dir\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.991068 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.990966    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9c3897a7-ac96-41d8-95fe-8101c9933ddf-registration-dir\") pod \"aws-ebs-csi-driver-node-n5hwh\" (UID: \"9c3897a7-ac96-41d8-95fe-8101c9933ddf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5hwh"
Jul 01 20:46:23.991068 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.990963    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-host-run-multus-certs\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.991068 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.991007    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-etc-kubernetes\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.991068 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.991015    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9c3897a7-ac96-41d8-95fe-8101c9933ddf-device-dir\") pod \"aws-ebs-csi-driver-node-n5hwh\" (UID: \"9c3897a7-ac96-41d8-95fe-8101c9933ddf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5hwh"
Jul 01 20:46:23.991068 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.991061    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-host-var-lib-cni-multus\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.991243 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.991125    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-host-run-netns\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.991243 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.991175    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9c3897a7-ac96-41d8-95fe-8101c9933ddf-sys-fs\") pod \"aws-ebs-csi-driver-node-n5hwh\" (UID: \"9c3897a7-ac96-41d8-95fe-8101c9933ddf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5hwh"
Jul 01 20:46:23.991243 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.991175    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/111e269e-a761-4eee-9a78-3fd554431c39-serviceca\") pod \"node-ca-swv7c\" (UID: \"111e269e-a761-4eee-9a78-3fd554431c39\") " pod="openshift-image-registry/node-ca-swv7c"
Jul 01 20:46:23.991243 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.991207    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/111e269e-a761-4eee-9a78-3fd554431c39-host\") pod \"node-ca-swv7c\" (UID: \"111e269e-a761-4eee-9a78-3fd554431c39\") " pod="openshift-image-registry/node-ca-swv7c"
Jul 01 20:46:23.991374 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.991272    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9a45d9e0-af21-4075-9578-09d1913a1082-os-release\") pod \"multus-additional-cni-plugins-qbnbc\" (UID: \"9a45d9e0-af21-4075-9578-09d1913a1082\") " pod="openshift-multus/multus-additional-cni-plugins-qbnbc"
Jul 01 20:46:23.991374 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.991298    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bc52f915-112e-4617-b476-01b645a52eb3-cni-binary-copy\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.991374 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.991325    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-multus-cni-dir\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.991374 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.991361    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-host-run-k8s-cni-cncf-io\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.991504 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.991403    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-run-openvswitch\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.991504 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.991446    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.991504 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.991463    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9a45d9e0-af21-4075-9578-09d1913a1082-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qbnbc\" (UID: \"9a45d9e0-af21-4075-9578-09d1913a1082\") " pod="openshift-multus/multus-additional-cni-plugins-qbnbc"
Jul 01 20:46:23.991504 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.991487    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-host-cni-netd\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.991645 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.991519    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9c3897a7-ac96-41d8-95fe-8101c9933ddf-kubelet-dir\") pod \"aws-ebs-csi-driver-node-n5hwh\" (UID: \"9c3897a7-ac96-41d8-95fe-8101c9933ddf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5hwh"
Jul 01 20:46:23.995741 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.992861    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-ovnkube-script-lib\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.995741 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.993081    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6cb9481e-9331-4e53-abc4-3336c04034c2-konnectivity-ca\") pod \"konnectivity-agent-njzrp\" (UID: \"6cb9481e-9331-4e53-abc4-3336c04034c2\") " pod="kube-system/konnectivity-agent-njzrp"
Jul 01 20:46:23.995741 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.995381    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6cb9481e-9331-4e53-abc4-3336c04034c2-agent-certs\") pod \"konnectivity-agent-njzrp\" (UID: \"6cb9481e-9331-4e53-abc4-3336c04034c2\") " pod="kube-system/konnectivity-agent-njzrp"
Jul 01 20:46:23.996502 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.996474    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-run-systemd\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:23.996661 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.996614    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/eaa37651-7fb2-4625-a15d-e7427a744196-hosts-file\") pod \"node-resolver-ncfhm\" (UID: \"eaa37651-7fb2-4625-a15d-e7427a744196\") " pod="openshift-dns/node-resolver-ncfhm"
Jul 01 20:46:23.996796 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.996773    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9c3897a7-ac96-41d8-95fe-8101c9933ddf-socket-dir\") pod \"aws-ebs-csi-driver-node-n5hwh\" (UID: \"9c3897a7-ac96-41d8-95fe-8101c9933ddf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5hwh"
Jul 01 20:46:23.996865 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.996848    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc52f915-112e-4617-b476-01b645a52eb3-host-var-lib-kubelet\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:23.996938 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.996916    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9a45d9e0-af21-4075-9578-09d1913a1082-cni-binary-copy\") pod \"multus-additional-cni-plugins-qbnbc\" (UID: \"9a45d9e0-af21-4075-9578-09d1913a1082\") " pod="openshift-multus/multus-additional-cni-plugins-qbnbc"
Jul 01 20:46:23.997015 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:23.996996    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1dfa26e7-23bf-4e56-808d-e59cee52aee7-dbus\") pod \"global-pull-secret-syncer-t6lw6\" (UID: \"1dfa26e7-23bf-4e56-808d-e59cee52aee7\") " pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:46:24.000687 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:24.000609    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 20:46:24.000687 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:24.000649    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 20:46:24.000687 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:24.000666    2571 projected.go:196] Error preparing data for projected volume kube-api-access-cj4fn for pod openshift-network-diagnostics/network-check-target-fz5ld: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 20:46:24.002071 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:24.001244    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a02110cb-ee26-4f3e-a854-ececca41783e-kube-api-access-cj4fn podName:a02110cb-ee26-4f3e-a854-ececca41783e nodeName:}" failed. No retries permitted until 2026-07-01 20:46:24.50122066 +0000 UTC m=+3.013695854 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-cj4fn" (UniqueName: "kubernetes.io/projected/a02110cb-ee26-4f3e-a854-ececca41783e-kube-api-access-cj4fn") pod "network-check-target-fz5ld" (UID: "a02110cb-ee26-4f3e-a854-ececca41783e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 20:46:24.002071 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:24.001649    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-ovn-node-metrics-cert\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:24.006143 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:24.006120    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9jkh\" (UniqueName: \"kubernetes.io/projected/9c3897a7-ac96-41d8-95fe-8101c9933ddf-kube-api-access-b9jkh\") pod \"aws-ebs-csi-driver-node-n5hwh\" (UID: \"9c3897a7-ac96-41d8-95fe-8101c9933ddf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5hwh"
Jul 01 20:46:24.007124 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:24.006869    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbmgs\" (UniqueName: \"kubernetes.io/projected/eaa37651-7fb2-4625-a15d-e7427a744196-kube-api-access-qbmgs\") pod \"node-resolver-ncfhm\" (UID: \"eaa37651-7fb2-4625-a15d-e7427a744196\") " pod="openshift-dns/node-resolver-ncfhm"
Jul 01 20:46:24.007124 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:24.007091    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6zb8\" (UniqueName: \"kubernetes.io/projected/bc52f915-112e-4617-b476-01b645a52eb3-kube-api-access-m6zb8\") pod \"multus-m8mqn\" (UID: \"bc52f915-112e-4617-b476-01b645a52eb3\") " pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:24.007834 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:24.007815    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ws6hp\" (UniqueName: \"kubernetes.io/projected/111e269e-a761-4eee-9a78-3fd554431c39-kube-api-access-ws6hp\") pod \"node-ca-swv7c\" (UID: \"111e269e-a761-4eee-9a78-3fd554431c39\") " pod="openshift-image-registry/node-ca-swv7c"
Jul 01 20:46:24.007927 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:24.007871    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5czp7\" (UniqueName: \"kubernetes.io/projected/2c340476-3fcc-4aa9-9e02-7d78b0b5858c-kube-api-access-5czp7\") pod \"ovnkube-node-sw9rx\" (UID: \"2c340476-3fcc-4aa9-9e02-7d78b0b5858c\") " pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:24.009522 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:24.009495    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9jj8\" (UniqueName: \"kubernetes.io/projected/9a45d9e0-af21-4075-9578-09d1913a1082-kube-api-access-r9jj8\") pod \"multus-additional-cni-plugins-qbnbc\" (UID: \"9a45d9e0-af21-4075-9578-09d1913a1082\") " pod="openshift-multus/multus-additional-cni-plugins-qbnbc"
Jul 01 20:46:24.015770 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:24.015746    2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 20:46:24.082202 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:24.082176    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lhmmx"
Jul 01 20:46:24.094843 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:24.094696    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4x2pr"
Jul 01 20:46:24.095609 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:24.095583    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66cd16df_9b52_4df7_ac4c_ac525b188137.slice/crio-ab72aab14bde4ded93df34a37317d8f33ca17ee0698fe5cdd0389634cd808af0 WatchSource:0}: Error finding container ab72aab14bde4ded93df34a37317d8f33ca17ee0698fe5cdd0389634cd808af0: Status 404 returned error can't find the container with id ab72aab14bde4ded93df34a37317d8f33ca17ee0698fe5cdd0389634cd808af0
Jul 01 20:46:24.104195 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:24.104171    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ncfhm"
Jul 01 20:46:24.110724 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:24.110701    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95b88ca8_8085_428f_a64d_b141a08f26ea.slice/crio-081b169a96ad8656cfa85e05dea4a9ec32527d902bfda20dd28681d2cd38288d WatchSource:0}: Error finding container 081b169a96ad8656cfa85e05dea4a9ec32527d902bfda20dd28681d2cd38288d: Status 404 returned error can't find the container with id 081b169a96ad8656cfa85e05dea4a9ec32527d902bfda20dd28681d2cd38288d
Jul 01 20:46:24.111622 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:24.111585    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-njzrp"
Jul 01 20:46:24.115568 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:24.115545    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaa37651_7fb2_4625_a15d_e7427a744196.slice/crio-579ce4cd271392742ee8237161394bb047032b0048938541ab6aff95f5589a70 WatchSource:0}: Error finding container 579ce4cd271392742ee8237161394bb047032b0048938541ab6aff95f5589a70: Status 404 returned error can't find the container with id 579ce4cd271392742ee8237161394bb047032b0048938541ab6aff95f5589a70
Jul 01 20:46:24.119192 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:24.119170    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-m8mqn"
Jul 01 20:46:24.129696 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:24.129505    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-swv7c"
Jul 01 20:46:24.138471 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:24.138457    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:24.140034 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:24.139972    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc52f915_112e_4617_b476_01b645a52eb3.slice/crio-5a95be0541bbaa1618b20d2a0004a3fdcd5c30f3cc9416cb64f547f49a050f55 WatchSource:0}: Error finding container 5a95be0541bbaa1618b20d2a0004a3fdcd5c30f3cc9416cb64f547f49a050f55: Status 404 returned error can't find the container with id 5a95be0541bbaa1618b20d2a0004a3fdcd5c30f3cc9416cb64f547f49a050f55
Jul 01 20:46:24.144842 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:24.144823    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod111e269e_a761_4eee_9a78_3fd554431c39.slice/crio-5e0a7948128e4aead7a343becb49102098660fc21738b90b3d35c5bc927b360e WatchSource:0}: Error finding container 5e0a7948128e4aead7a343becb49102098660fc21738b90b3d35c5bc927b360e: Status 404 returned error can't find the container with id 5e0a7948128e4aead7a343becb49102098660fc21738b90b3d35c5bc927b360e
Jul 01 20:46:24.149040 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:24.149005    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5hwh"
Jul 01 20:46:24.154113 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:24.154088    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c340476_3fcc_4aa9_9e02_7d78b0b5858c.slice/crio-0a74cabd8b19ff50904da3c14c1fcc039a67cf3368cad0188fe28a31e8a8cf7e WatchSource:0}: Error finding container 0a74cabd8b19ff50904da3c14c1fcc039a67cf3368cad0188fe28a31e8a8cf7e: Status 404 returned error can't find the container with id 0a74cabd8b19ff50904da3c14c1fcc039a67cf3368cad0188fe28a31e8a8cf7e
Jul 01 20:46:24.155669 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:24.155645    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qbnbc"
Jul 01 20:46:24.163338 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:24.163315    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c3897a7_ac96_41d8_95fe_8101c9933ddf.slice/crio-f8b5f24cdc51b996f8f901b45883adfb14a0acd62031d8fdd067d5508e1efa24 WatchSource:0}: Error finding container f8b5f24cdc51b996f8f901b45883adfb14a0acd62031d8fdd067d5508e1efa24: Status 404 returned error can't find the container with id f8b5f24cdc51b996f8f901b45883adfb14a0acd62031d8fdd067d5508e1efa24
Jul 01 20:46:24.168939 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:46:24.168913    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a45d9e0_af21_4075_9578_09d1913a1082.slice/crio-8db7cad69f57ba0d7cfce9ee68b488c4cb2769b38a4f7ddab5712744e3a46dac WatchSource:0}: Error finding container 8db7cad69f57ba0d7cfce9ee68b488c4cb2769b38a4f7ddab5712744e3a46dac: Status 404 returned error can't find the container with id 8db7cad69f57ba0d7cfce9ee68b488c4cb2769b38a4f7ddab5712744e3a46dac
Jul 01 20:46:24.392018 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:24.391921    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3779f941-e08c-4e16-a1ab-be86f76ee83d-metrics-certs\") pod \"network-metrics-daemon-cthwx\" (UID: \"3779f941-e08c-4e16-a1ab-be86f76ee83d\") " pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:46:24.392168 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:24.392068    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 20:46:24.392168 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:24.392133    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3779f941-e08c-4e16-a1ab-be86f76ee83d-metrics-certs podName:3779f941-e08c-4e16-a1ab-be86f76ee83d nodeName:}" failed. No retries permitted until 2026-07-01 20:46:25.392112778 +0000 UTC m=+3.904587952 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3779f941-e08c-4e16-a1ab-be86f76ee83d-metrics-certs") pod "network-metrics-daemon-cthwx" (UID: "3779f941-e08c-4e16-a1ab-be86f76ee83d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 20:46:24.492788 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:24.492756    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1dfa26e7-23bf-4e56-808d-e59cee52aee7-original-pull-secret\") pod \"global-pull-secret-syncer-t6lw6\" (UID: \"1dfa26e7-23bf-4e56-808d-e59cee52aee7\") " pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:46:24.492944 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:24.492916    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 20:46:24.492991 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:24.492977    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dfa26e7-23bf-4e56-808d-e59cee52aee7-original-pull-secret podName:1dfa26e7-23bf-4e56-808d-e59cee52aee7 nodeName:}" failed. No retries permitted until 2026-07-01 20:46:25.492957898 +0000 UTC m=+4.005433071 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1dfa26e7-23bf-4e56-808d-e59cee52aee7-original-pull-secret") pod "global-pull-secret-syncer-t6lw6" (UID: "1dfa26e7-23bf-4e56-808d-e59cee52aee7") : object "kube-system"/"original-pull-secret" not registered
Jul 01 20:46:24.593943 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:24.593904    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj4fn\" (UniqueName: \"kubernetes.io/projected/a02110cb-ee26-4f3e-a854-ececca41783e-kube-api-access-cj4fn\") pod \"network-check-target-fz5ld\" (UID: \"a02110cb-ee26-4f3e-a854-ececca41783e\") " pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:46:24.594118 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:24.594080    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 20:46:24.594118 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:24.594107    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 20:46:24.594227 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:24.594121    2571 projected.go:196] Error preparing data for projected volume kube-api-access-cj4fn for pod openshift-network-diagnostics/network-check-target-fz5ld: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 20:46:24.594227 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:24.594195    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a02110cb-ee26-4f3e-a854-ececca41783e-kube-api-access-cj4fn podName:a02110cb-ee26-4f3e-a854-ececca41783e nodeName:}" failed. No retries permitted until 2026-07-01 20:46:25.594172247 +0000 UTC m=+4.106647438 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-cj4fn" (UniqueName: "kubernetes.io/projected/a02110cb-ee26-4f3e-a854-ececca41783e-kube-api-access-cj4fn") pod "network-check-target-fz5ld" (UID: "a02110cb-ee26-4f3e-a854-ececca41783e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 20:46:24.822739 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:24.822695    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 20:41:24 +0000 UTC" deadline="2028-03-10 05:02:56.429428054 +0000 UTC"
Jul 01 20:46:24.822739 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:24.822736    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14816h16m31.606698055s"
Jul 01 20:46:24.833254 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:24.833227    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:46:24.833390 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:24.833263    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:46:24.833390 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:24.833374    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cthwx" podUID="3779f941-e08c-4e16-a1ab-be86f76ee83d"
Jul 01 20:46:24.833540 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:24.833516    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lw6" podUID="1dfa26e7-23bf-4e56-808d-e59cee52aee7"
Jul 01 20:46:24.849130 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:24.849091    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qbnbc" event={"ID":"9a45d9e0-af21-4075-9578-09d1913a1082","Type":"ContainerStarted","Data":"8db7cad69f57ba0d7cfce9ee68b488c4cb2769b38a4f7ddab5712744e3a46dac"}
Jul 01 20:46:24.850314 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:24.850281    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5hwh" event={"ID":"9c3897a7-ac96-41d8-95fe-8101c9933ddf","Type":"ContainerStarted","Data":"f8b5f24cdc51b996f8f901b45883adfb14a0acd62031d8fdd067d5508e1efa24"}
Jul 01 20:46:24.851591 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:24.851559    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-m8mqn" event={"ID":"bc52f915-112e-4617-b476-01b645a52eb3","Type":"ContainerStarted","Data":"5a95be0541bbaa1618b20d2a0004a3fdcd5c30f3cc9416cb64f547f49a050f55"}
Jul 01 20:46:24.852914 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:24.852890    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-njzrp" event={"ID":"6cb9481e-9331-4e53-abc4-3336c04034c2","Type":"ContainerStarted","Data":"eaf26e672235ad2ee894ab6a95e164abf71263ea7fecda32ff384cdd17e1c75b"}
Jul 01 20:46:24.854140 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:24.854106    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ncfhm" event={"ID":"eaa37651-7fb2-4625-a15d-e7427a744196","Type":"ContainerStarted","Data":"579ce4cd271392742ee8237161394bb047032b0048938541ab6aff95f5589a70"}
Jul 01 20:46:24.855423 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:24.855391    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4x2pr" event={"ID":"95b88ca8-8085-428f-a64d-b141a08f26ea","Type":"ContainerStarted","Data":"081b169a96ad8656cfa85e05dea4a9ec32527d902bfda20dd28681d2cd38288d"}
Jul 01 20:46:24.856896 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:24.856807    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx" event={"ID":"2c340476-3fcc-4aa9-9e02-7d78b0b5858c","Type":"ContainerStarted","Data":"0a74cabd8b19ff50904da3c14c1fcc039a67cf3368cad0188fe28a31e8a8cf7e"}
Jul 01 20:46:24.858239 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:24.858214    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-swv7c" event={"ID":"111e269e-a761-4eee-9a78-3fd554431c39","Type":"ContainerStarted","Data":"5e0a7948128e4aead7a343becb49102098660fc21738b90b3d35c5bc927b360e"}
Jul 01 20:46:24.861028 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:24.861004    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lhmmx" event={"ID":"66cd16df-9b52-4df7-ac4c-ac525b188137","Type":"ContainerStarted","Data":"ab72aab14bde4ded93df34a37317d8f33ca17ee0698fe5cdd0389634cd808af0"}
Jul 01 20:46:25.104090 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:25.103791    2571 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 20:46:25.400045 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:25.399964    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3779f941-e08c-4e16-a1ab-be86f76ee83d-metrics-certs\") pod \"network-metrics-daemon-cthwx\" (UID: \"3779f941-e08c-4e16-a1ab-be86f76ee83d\") " pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:46:25.400212 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:25.400172    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 20:46:25.400281 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:25.400248    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3779f941-e08c-4e16-a1ab-be86f76ee83d-metrics-certs podName:3779f941-e08c-4e16-a1ab-be86f76ee83d nodeName:}" failed. No retries permitted until 2026-07-01 20:46:27.400229423 +0000 UTC m=+5.912704595 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3779f941-e08c-4e16-a1ab-be86f76ee83d-metrics-certs") pod "network-metrics-daemon-cthwx" (UID: "3779f941-e08c-4e16-a1ab-be86f76ee83d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 20:46:25.501401 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:25.501360    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1dfa26e7-23bf-4e56-808d-e59cee52aee7-original-pull-secret\") pod \"global-pull-secret-syncer-t6lw6\" (UID: \"1dfa26e7-23bf-4e56-808d-e59cee52aee7\") " pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:46:25.501596 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:25.501531    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 20:46:25.501674 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:25.501597    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dfa26e7-23bf-4e56-808d-e59cee52aee7-original-pull-secret podName:1dfa26e7-23bf-4e56-808d-e59cee52aee7 nodeName:}" failed. No retries permitted until 2026-07-01 20:46:27.501577094 +0000 UTC m=+6.014052289 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1dfa26e7-23bf-4e56-808d-e59cee52aee7-original-pull-secret") pod "global-pull-secret-syncer-t6lw6" (UID: "1dfa26e7-23bf-4e56-808d-e59cee52aee7") : object "kube-system"/"original-pull-secret" not registered
Jul 01 20:46:25.603290 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:25.602511    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj4fn\" (UniqueName: \"kubernetes.io/projected/a02110cb-ee26-4f3e-a854-ececca41783e-kube-api-access-cj4fn\") pod \"network-check-target-fz5ld\" (UID: \"a02110cb-ee26-4f3e-a854-ececca41783e\") " pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:46:25.603290 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:25.602713    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 20:46:25.603290 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:25.602738    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 20:46:25.603290 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:25.602753    2571 projected.go:196] Error preparing data for projected volume kube-api-access-cj4fn for pod openshift-network-diagnostics/network-check-target-fz5ld: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 20:46:25.603290 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:25.602821    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a02110cb-ee26-4f3e-a854-ececca41783e-kube-api-access-cj4fn podName:a02110cb-ee26-4f3e-a854-ececca41783e nodeName:}" failed. No retries permitted until 2026-07-01 20:46:27.6027991 +0000 UTC m=+6.115274284 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-cj4fn" (UniqueName: "kubernetes.io/projected/a02110cb-ee26-4f3e-a854-ececca41783e-kube-api-access-cj4fn") pod "network-check-target-fz5ld" (UID: "a02110cb-ee26-4f3e-a854-ececca41783e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 20:46:25.823590 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:25.823544    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 20:41:24 +0000 UTC" deadline="2027-12-12 21:23:31.403253226 +0000 UTC"
Jul 01 20:46:25.823590 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:25.823588    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12696h37m5.57967037s"
Jul 01 20:46:25.835573 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:25.835547    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:46:25.835730 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:25.835672    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fz5ld" podUID="a02110cb-ee26-4f3e-a854-ececca41783e"
Jul 01 20:46:26.833789 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:26.833752    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:46:26.834261 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:26.833872    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cthwx" podUID="3779f941-e08c-4e16-a1ab-be86f76ee83d"
Jul 01 20:46:26.834261 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:26.834251    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:46:26.834402 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:26.834345    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lw6" podUID="1dfa26e7-23bf-4e56-808d-e59cee52aee7"
Jul 01 20:46:26.886785 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:26.886273    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-104.ec2.internal" event={"ID":"289aec27ab6274a41b4614fd9786336a","Type":"ContainerStarted","Data":"cdf10688fd8d442929016532fddbfc337c0d5574126ea76a7d581241a6a3a289"}
Jul 01 20:46:27.418607 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:27.418564    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3779f941-e08c-4e16-a1ab-be86f76ee83d-metrics-certs\") pod \"network-metrics-daemon-cthwx\" (UID: \"3779f941-e08c-4e16-a1ab-be86f76ee83d\") " pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:46:27.418818 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:27.418732    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 20:46:27.418818 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:27.418797    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3779f941-e08c-4e16-a1ab-be86f76ee83d-metrics-certs podName:3779f941-e08c-4e16-a1ab-be86f76ee83d nodeName:}" failed. No retries permitted until 2026-07-01 20:46:31.418778374 +0000 UTC m=+9.931253543 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3779f941-e08c-4e16-a1ab-be86f76ee83d-metrics-certs") pod "network-metrics-daemon-cthwx" (UID: "3779f941-e08c-4e16-a1ab-be86f76ee83d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 20:46:27.519518 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:27.519429    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1dfa26e7-23bf-4e56-808d-e59cee52aee7-original-pull-secret\") pod \"global-pull-secret-syncer-t6lw6\" (UID: \"1dfa26e7-23bf-4e56-808d-e59cee52aee7\") " pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:46:27.519676 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:27.519633    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 20:46:27.519819 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:27.519723    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dfa26e7-23bf-4e56-808d-e59cee52aee7-original-pull-secret podName:1dfa26e7-23bf-4e56-808d-e59cee52aee7 nodeName:}" failed. No retries permitted until 2026-07-01 20:46:31.519702968 +0000 UTC m=+10.032178139 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1dfa26e7-23bf-4e56-808d-e59cee52aee7-original-pull-secret") pod "global-pull-secret-syncer-t6lw6" (UID: "1dfa26e7-23bf-4e56-808d-e59cee52aee7") : object "kube-system"/"original-pull-secret" not registered
Jul 01 20:46:27.620206 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:27.620168    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj4fn\" (UniqueName: \"kubernetes.io/projected/a02110cb-ee26-4f3e-a854-ececca41783e-kube-api-access-cj4fn\") pod \"network-check-target-fz5ld\" (UID: \"a02110cb-ee26-4f3e-a854-ececca41783e\") " pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:46:27.620385 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:27.620355    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 20:46:27.620385 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:27.620377    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 20:46:27.620505 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:27.620390    2571 projected.go:196] Error preparing data for projected volume kube-api-access-cj4fn for pod openshift-network-diagnostics/network-check-target-fz5ld: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 20:46:27.620505 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:27.620465    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a02110cb-ee26-4f3e-a854-ececca41783e-kube-api-access-cj4fn podName:a02110cb-ee26-4f3e-a854-ececca41783e nodeName:}" failed. No retries permitted until 2026-07-01 20:46:31.620445251 +0000 UTC m=+10.132920425 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-cj4fn" (UniqueName: "kubernetes.io/projected/a02110cb-ee26-4f3e-a854-ececca41783e-kube-api-access-cj4fn") pod "network-check-target-fz5ld" (UID: "a02110cb-ee26-4f3e-a854-ececca41783e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 20:46:27.836434 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:27.836352    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:46:27.836943 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:27.836493    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fz5ld" podUID="a02110cb-ee26-4f3e-a854-ececca41783e"
Jul 01 20:46:27.905126 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:27.905073    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-104.ec2.internal" event={"ID":"23a6143e19ae3dec90500595f50e9ee3","Type":"ContainerStarted","Data":"c215aceec6af68c5ac58948649ce2f9a2b04802dba2bf32f98b54316677e78af"}
Jul 01 20:46:27.923118 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:27.923058    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-104.ec2.internal" podStartSLOduration=4.923035267 podStartE2EDuration="4.923035267s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 20:46:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:46:26.905681734 +0000 UTC m=+5.418156933" watchObservedRunningTime="2026-07-01 20:46:27.923035267 +0000 UTC m=+6.435510465"
Jul 01 20:46:28.833898 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:28.833864    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:46:28.834068 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:28.834000    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cthwx" podUID="3779f941-e08c-4e16-a1ab-be86f76ee83d"
Jul 01 20:46:28.834375 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:28.834351    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:46:28.834476 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:28.834454    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lw6" podUID="1dfa26e7-23bf-4e56-808d-e59cee52aee7"
Jul 01 20:46:29.835960 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:29.835449    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:46:29.835960 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:29.835564    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fz5ld" podUID="a02110cb-ee26-4f3e-a854-ececca41783e"
Jul 01 20:46:30.833789 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:30.833761    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:46:30.833926 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:30.833894    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cthwx" podUID="3779f941-e08c-4e16-a1ab-be86f76ee83d"
Jul 01 20:46:30.834003 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:30.833950    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:46:30.834084 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:30.834056    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lw6" podUID="1dfa26e7-23bf-4e56-808d-e59cee52aee7"
Jul 01 20:46:31.451674 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:31.451637    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3779f941-e08c-4e16-a1ab-be86f76ee83d-metrics-certs\") pod \"network-metrics-daemon-cthwx\" (UID: \"3779f941-e08c-4e16-a1ab-be86f76ee83d\") " pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:46:31.452126 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:31.451773    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 20:46:31.452126 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:31.451830    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3779f941-e08c-4e16-a1ab-be86f76ee83d-metrics-certs podName:3779f941-e08c-4e16-a1ab-be86f76ee83d nodeName:}" failed. No retries permitted until 2026-07-01 20:46:39.451811432 +0000 UTC m=+17.964286608 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3779f941-e08c-4e16-a1ab-be86f76ee83d-metrics-certs") pod "network-metrics-daemon-cthwx" (UID: "3779f941-e08c-4e16-a1ab-be86f76ee83d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 20:46:31.552445 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:31.552417    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1dfa26e7-23bf-4e56-808d-e59cee52aee7-original-pull-secret\") pod \"global-pull-secret-syncer-t6lw6\" (UID: \"1dfa26e7-23bf-4e56-808d-e59cee52aee7\") " pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:46:31.552599 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:31.552569    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 20:46:31.552693 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:31.552653    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dfa26e7-23bf-4e56-808d-e59cee52aee7-original-pull-secret podName:1dfa26e7-23bf-4e56-808d-e59cee52aee7 nodeName:}" failed. No retries permitted until 2026-07-01 20:46:39.552613628 +0000 UTC m=+18.065088802 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1dfa26e7-23bf-4e56-808d-e59cee52aee7-original-pull-secret") pod "global-pull-secret-syncer-t6lw6" (UID: "1dfa26e7-23bf-4e56-808d-e59cee52aee7") : object "kube-system"/"original-pull-secret" not registered
Jul 01 20:46:31.653700 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:31.653663    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj4fn\" (UniqueName: \"kubernetes.io/projected/a02110cb-ee26-4f3e-a854-ececca41783e-kube-api-access-cj4fn\") pod \"network-check-target-fz5ld\" (UID: \"a02110cb-ee26-4f3e-a854-ececca41783e\") " pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:46:31.653873 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:31.653854    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 20:46:31.653938 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:31.653881    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 20:46:31.653938 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:31.653894    2571 projected.go:196] Error preparing data for projected volume kube-api-access-cj4fn for pod openshift-network-diagnostics/network-check-target-fz5ld: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 20:46:31.654039 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:31.653954    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a02110cb-ee26-4f3e-a854-ececca41783e-kube-api-access-cj4fn podName:a02110cb-ee26-4f3e-a854-ececca41783e nodeName:}" failed. No retries permitted until 2026-07-01 20:46:39.653935472 +0000 UTC m=+18.166410643 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-cj4fn" (UniqueName: "kubernetes.io/projected/a02110cb-ee26-4f3e-a854-ececca41783e-kube-api-access-cj4fn") pod "network-check-target-fz5ld" (UID: "a02110cb-ee26-4f3e-a854-ececca41783e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 20:46:31.839271 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:31.838685    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:46:31.839271 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:31.838776    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fz5ld" podUID="a02110cb-ee26-4f3e-a854-ececca41783e"
Jul 01 20:46:31.839271 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:31.839086    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:46:31.839271 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:31.839190    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cthwx" podUID="3779f941-e08c-4e16-a1ab-be86f76ee83d"
Jul 01 20:46:32.834060 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:32.833975    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:46:32.834492 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:32.834115    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lw6" podUID="1dfa26e7-23bf-4e56-808d-e59cee52aee7"
Jul 01 20:46:32.917450 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:32.916674    2571 generic.go:356] "Generic (PLEG): container finished" podID="23a6143e19ae3dec90500595f50e9ee3" containerID="c215aceec6af68c5ac58948649ce2f9a2b04802dba2bf32f98b54316677e78af" exitCode=0
Jul 01 20:46:32.917450 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:32.916728    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-104.ec2.internal" event={"ID":"23a6143e19ae3dec90500595f50e9ee3","Type":"ContainerDied","Data":"c215aceec6af68c5ac58948649ce2f9a2b04802dba2bf32f98b54316677e78af"}
Jul 01 20:46:33.833519 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:33.833488    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:46:33.833724 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:33.833612    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fz5ld" podUID="a02110cb-ee26-4f3e-a854-ececca41783e"
Jul 01 20:46:33.834011 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:33.833983    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:46:33.834128 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:33.834099    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cthwx" podUID="3779f941-e08c-4e16-a1ab-be86f76ee83d"
Jul 01 20:46:34.833544 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:34.833510    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:46:34.833749 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:34.833666    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lw6" podUID="1dfa26e7-23bf-4e56-808d-e59cee52aee7"
Jul 01 20:46:35.833904 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:35.833871    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:46:35.833904 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:35.833887    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:46:35.834294 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:35.833969    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fz5ld" podUID="a02110cb-ee26-4f3e-a854-ececca41783e"
Jul 01 20:46:35.834294 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:35.834102    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cthwx" podUID="3779f941-e08c-4e16-a1ab-be86f76ee83d"
Jul 01 20:46:36.833351 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:36.833308    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:46:36.833531 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:36.833444    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lw6" podUID="1dfa26e7-23bf-4e56-808d-e59cee52aee7"
Jul 01 20:46:37.834115 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:37.834083    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:46:37.834115 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:37.834105    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:46:37.834613 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:37.834207    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fz5ld" podUID="a02110cb-ee26-4f3e-a854-ececca41783e"
Jul 01 20:46:37.834613 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:37.834343    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cthwx" podUID="3779f941-e08c-4e16-a1ab-be86f76ee83d"
Jul 01 20:46:38.833901 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:38.833861    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:46:38.834121 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:38.833998    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lw6" podUID="1dfa26e7-23bf-4e56-808d-e59cee52aee7"
Jul 01 20:46:39.517154 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:39.517111    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3779f941-e08c-4e16-a1ab-be86f76ee83d-metrics-certs\") pod \"network-metrics-daemon-cthwx\" (UID: \"3779f941-e08c-4e16-a1ab-be86f76ee83d\") " pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:46:39.517646 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:39.517299    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 20:46:39.517646 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:39.517383    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3779f941-e08c-4e16-a1ab-be86f76ee83d-metrics-certs podName:3779f941-e08c-4e16-a1ab-be86f76ee83d nodeName:}" failed. No retries permitted until 2026-07-01 20:46:55.517365504 +0000 UTC m=+34.029840683 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3779f941-e08c-4e16-a1ab-be86f76ee83d-metrics-certs") pod "network-metrics-daemon-cthwx" (UID: "3779f941-e08c-4e16-a1ab-be86f76ee83d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 20:46:39.618032 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:39.617998    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1dfa26e7-23bf-4e56-808d-e59cee52aee7-original-pull-secret\") pod \"global-pull-secret-syncer-t6lw6\" (UID: \"1dfa26e7-23bf-4e56-808d-e59cee52aee7\") " pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:46:39.618192 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:39.618172    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 20:46:39.618265 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:39.618255    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dfa26e7-23bf-4e56-808d-e59cee52aee7-original-pull-secret podName:1dfa26e7-23bf-4e56-808d-e59cee52aee7 nodeName:}" failed. No retries permitted until 2026-07-01 20:46:55.618233249 +0000 UTC m=+34.130708426 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1dfa26e7-23bf-4e56-808d-e59cee52aee7-original-pull-secret") pod "global-pull-secret-syncer-t6lw6" (UID: "1dfa26e7-23bf-4e56-808d-e59cee52aee7") : object "kube-system"/"original-pull-secret" not registered
Jul 01 20:46:39.718439 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:39.718405    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj4fn\" (UniqueName: \"kubernetes.io/projected/a02110cb-ee26-4f3e-a854-ececca41783e-kube-api-access-cj4fn\") pod \"network-check-target-fz5ld\" (UID: \"a02110cb-ee26-4f3e-a854-ececca41783e\") " pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:46:39.718647 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:39.718602    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 20:46:39.718647 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:39.718645    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 20:46:39.718761 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:39.718659    2571 projected.go:196] Error preparing data for projected volume kube-api-access-cj4fn for pod openshift-network-diagnostics/network-check-target-fz5ld: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 20:46:39.718761 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:39.718723    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a02110cb-ee26-4f3e-a854-ececca41783e-kube-api-access-cj4fn podName:a02110cb-ee26-4f3e-a854-ececca41783e nodeName:}" failed. No retries permitted until 2026-07-01 20:46:55.718704464 +0000 UTC m=+34.231179635 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-cj4fn" (UniqueName: "kubernetes.io/projected/a02110cb-ee26-4f3e-a854-ececca41783e-kube-api-access-cj4fn") pod "network-check-target-fz5ld" (UID: "a02110cb-ee26-4f3e-a854-ececca41783e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 20:46:39.834070 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:39.833974    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:46:39.834241 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:39.834094    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fz5ld" podUID="a02110cb-ee26-4f3e-a854-ececca41783e"
Jul 01 20:46:39.834241 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:39.834142    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:46:39.834342 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:39.834248    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cthwx" podUID="3779f941-e08c-4e16-a1ab-be86f76ee83d"
Jul 01 20:46:40.833671 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:40.833636    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:46:40.834154 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:40.833764    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lw6" podUID="1dfa26e7-23bf-4e56-808d-e59cee52aee7"
Jul 01 20:46:41.834759 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:41.834724    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:46:41.834759 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:41.834761    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:46:41.835266 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:41.834831    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fz5ld" podUID="a02110cb-ee26-4f3e-a854-ececca41783e"
Jul 01 20:46:41.835266 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:41.834874    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cthwx" podUID="3779f941-e08c-4e16-a1ab-be86f76ee83d"
Jul 01 20:46:42.833751 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:42.833712    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:46:42.833921 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:42.833837    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lw6" podUID="1dfa26e7-23bf-4e56-808d-e59cee52aee7"
Jul 01 20:46:43.833731 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:43.833689    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:46:43.834272 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:43.833815    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fz5ld" podUID="a02110cb-ee26-4f3e-a854-ececca41783e"
Jul 01 20:46:43.834272 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:43.833900    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:46:43.834272 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:43.833999    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cthwx" podUID="3779f941-e08c-4e16-a1ab-be86f76ee83d"
Jul 01 20:46:44.833452 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:44.833277    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:46:44.833603 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:44.833535    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lw6" podUID="1dfa26e7-23bf-4e56-808d-e59cee52aee7"
Jul 01 20:46:44.940762 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:44.940723    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ncfhm" event={"ID":"eaa37651-7fb2-4625-a15d-e7427a744196","Type":"ContainerStarted","Data":"a7b1526334817a5eacbd03c53dc892927d14ce56f3ca57d2e32f691123370551"}
Jul 01 20:46:44.942256 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:44.942233    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4x2pr" event={"ID":"95b88ca8-8085-428f-a64d-b141a08f26ea","Type":"ContainerStarted","Data":"c78efb27a82ce2c6d70946dd2577f1197408e4c37fa4cbd5f634e99caba5af8f"}
Jul 01 20:46:44.944146 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:44.944112    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx" event={"ID":"2c340476-3fcc-4aa9-9e02-7d78b0b5858c","Type":"ContainerStarted","Data":"bd9049c107d092674841de271851689db612b667f52469fb28a9c1053db630d1"}
Jul 01 20:46:44.944251 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:44.944185    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx" event={"ID":"2c340476-3fcc-4aa9-9e02-7d78b0b5858c","Type":"ContainerStarted","Data":"e07d941e3374cb4499201f195138f110d73c1f12d3a5c4a269b9045ebdf6e03c"}
Jul 01 20:46:44.944306 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:44.944248    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx" event={"ID":"2c340476-3fcc-4aa9-9e02-7d78b0b5858c","Type":"ContainerStarted","Data":"69a0675e112aae298b3f82364b8ba364f9bd1fd9f882454ac08ee6a2bd1a1e3f"}
Jul 01 20:46:44.945399 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:44.945376    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-swv7c" event={"ID":"111e269e-a761-4eee-9a78-3fd554431c39","Type":"ContainerStarted","Data":"14b61cc57c403726ff971fafc4a89d525a501c73f4724f62c7f8210bff1bd062"}
Jul 01 20:46:44.947021 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:44.947001    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-104.ec2.internal" event={"ID":"23a6143e19ae3dec90500595f50e9ee3","Type":"ContainerStarted","Data":"f18a3e22915cf3525c379927cd3ea410695ff3f0da0f4edabcd448ad67563955"}
Jul 01 20:46:44.948313 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:44.948289    2571 generic.go:356] "Generic (PLEG): container finished" podID="9a45d9e0-af21-4075-9578-09d1913a1082" containerID="d53b785e6f907b988ca8389766abd3dd15c564fafeb1005f18f3f412c9f5676c" exitCode=0
Jul 01 20:46:44.948389 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:44.948358    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qbnbc" event={"ID":"9a45d9e0-af21-4075-9578-09d1913a1082","Type":"ContainerDied","Data":"d53b785e6f907b988ca8389766abd3dd15c564fafeb1005f18f3f412c9f5676c"}
Jul 01 20:46:44.949937 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:44.949816    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5hwh" event={"ID":"9c3897a7-ac96-41d8-95fe-8101c9933ddf","Type":"ContainerStarted","Data":"d58998aeae9d557a0ba86789b993473f90fdccd7fdc297abdf2584e666542453"}
Jul 01 20:46:44.952881 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:44.952857    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-m8mqn" event={"ID":"bc52f915-112e-4617-b476-01b645a52eb3","Type":"ContainerStarted","Data":"e31c57ed949ae26ff94e65ad9933e1962046b58d707728456f31a06bfc270460"}
Jul 01 20:46:44.954532 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:44.954513    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-njzrp" event={"ID":"6cb9481e-9331-4e53-abc4-3336c04034c2","Type":"ContainerStarted","Data":"43c1c0081cf1cda406d50a074515ffe87fd0efe6b104bf5780eb2c485d837f14"}
Jul 01 20:46:44.956469 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:44.956432    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ncfhm" podStartSLOduration=2.637245779 podStartE2EDuration="22.956422158s" totalImagesPullingTime="20.319176379s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 20:46:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:46:44.95604879 +0000 UTC m=+23.468523982" watchObservedRunningTime="2026-07-01 20:46:44.956422158 +0000 UTC m=+23.468897347"
Jul 01 20:46:44.973129 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:44.973050    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4x2pr" podStartSLOduration=2.65009203 podStartE2EDuration="22.973033758s" totalImagesPullingTime="20.322941728s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 20:46:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:46:44.972591441 +0000 UTC m=+23.485066632" watchObservedRunningTime="2026-07-01 20:46:44.973033758 +0000 UTC m=+23.485508947"
Jul 01 20:46:44.988008 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:44.987960    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-njzrp" podStartSLOduration=3.011989274 podStartE2EDuration="22.987943513s" totalImagesPullingTime="19.975954239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:46:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:46:44.987619592 +0000 UTC m=+23.500094782" watchObservedRunningTime="2026-07-01 20:46:44.987943513 +0000 UTC m=+23.500418703"
Jul 01 20:46:45.006016 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:45.005973    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-m8mqn" podStartSLOduration=2.560411664 podStartE2EDuration="23.00596055s" totalImagesPullingTime="20.445548886s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 20:46:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:46:45.005733817 +0000 UTC m=+23.518209007" watchObservedRunningTime="2026-07-01 20:46:45.00596055 +0000 UTC m=+23.518435740"
Jul 01 20:46:45.020598 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:45.020544    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-swv7c" podStartSLOduration=2.732557147 podStartE2EDuration="23.020528833s" totalImagesPullingTime="20.287971686s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 20:46:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:46:45.02022224 +0000 UTC m=+23.532697430" watchObservedRunningTime="2026-07-01 20:46:45.020528833 +0000 UTC m=+23.533004022"
Jul 01 20:46:45.037679 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:45.037602    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-104.ec2.internal" podStartSLOduration=22.037581508 podStartE2EDuration="22.037581508s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 20:46:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:46:45.037008613 +0000 UTC m=+23.549483805" watchObservedRunningTime="2026-07-01 20:46:45.037581508 +0000 UTC m=+23.550056701"
Jul 01 20:46:45.658130 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:45.658087    2571 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 01 20:46:45.834126 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:45.834100    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:46:45.834324 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:45.834200    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fz5ld" podUID="a02110cb-ee26-4f3e-a854-ececca41783e"
Jul 01 20:46:45.834324 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:45.834279    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:46:45.834458 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:45.834364    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cthwx" podUID="3779f941-e08c-4e16-a1ab-be86f76ee83d"
Jul 01 20:46:45.836548 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:45.836478    2571 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-01T20:46:45.658125625Z","UUID":"c9ee0045-7b29-431c-86e0-dd20bec0cbf9","Handler":null,"Name":"","Endpoint":""}
Jul 01 20:46:45.837933 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:45.837910    2571 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 01 20:46:45.838006 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:45.837942    2571 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 01 20:46:45.960040 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:45.959959    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx" event={"ID":"2c340476-3fcc-4aa9-9e02-7d78b0b5858c","Type":"ContainerStarted","Data":"3215e273397397535b063213471df79220119e8e2c36cac4b8998962824dd9ca"}
Jul 01 20:46:45.960524 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:45.960044    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx" event={"ID":"2c340476-3fcc-4aa9-9e02-7d78b0b5858c","Type":"ContainerStarted","Data":"d6801608cc7790016247e435afe5022a2c62f7f558e63ec42a24dc042a624a6c"}
Jul 01 20:46:45.960524 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:45.960088    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx" event={"ID":"2c340476-3fcc-4aa9-9e02-7d78b0b5858c","Type":"ContainerStarted","Data":"1d0c1205c978268368830658d78134153c9930df3ac979c86faeb54565cecc68"}
Jul 01 20:46:45.961654 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:45.961608    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lhmmx" event={"ID":"66cd16df-9b52-4df7-ac4c-ac525b188137","Type":"ContainerStarted","Data":"43996db47486ef3b52d2a12d435977d75657f06f6ec4b626c802e26b0d0a6ea7"}
Jul 01 20:46:45.963376 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:45.963344    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5hwh" event={"ID":"9c3897a7-ac96-41d8-95fe-8101c9933ddf","Type":"ContainerStarted","Data":"7a9acfcf54bf534f538ad8678c039992533e4bc055ec397e9857483207038970"}
Jul 01 20:46:45.978297 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:45.978242    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-lhmmx" podStartSLOduration=3.661876777 podStartE2EDuration="23.978225647s" totalImagesPullingTime="20.31634887s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 20:46:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:46:45.977269975 +0000 UTC m=+24.489745166" watchObservedRunningTime="2026-07-01 20:46:45.978225647 +0000 UTC m=+24.490700838"
Jul 01 20:46:46.833690 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:46.833661    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:46:46.833958 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:46.833776    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lw6" podUID="1dfa26e7-23bf-4e56-808d-e59cee52aee7"
Jul 01 20:46:47.833241 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:47.833022    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:46:47.833674 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:47.833028    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:46:47.833674 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:47.833349    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fz5ld" podUID="a02110cb-ee26-4f3e-a854-ececca41783e"
Jul 01 20:46:47.833674 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:47.833467    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cthwx" podUID="3779f941-e08c-4e16-a1ab-be86f76ee83d"
Jul 01 20:46:47.970068 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:47.970030    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5hwh" event={"ID":"9c3897a7-ac96-41d8-95fe-8101c9933ddf","Type":"ContainerStarted","Data":"4411f5fa3ab8df482559dd75ffdbadb83bd57901f76d93346383298598a4a39e"}
Jul 01 20:46:47.973574 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:47.973550    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx" event={"ID":"2c340476-3fcc-4aa9-9e02-7d78b0b5858c","Type":"ContainerStarted","Data":"70898c94066850bc6eb92c1c97f4ee5d5858626c2b11ada58ca7eec98e2a7565"}
Jul 01 20:46:47.989687 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:47.989614    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5hwh" podStartSLOduration=3.5956843320000003 podStartE2EDuration="25.989600449s" totalImagesPullingTime="22.393916117s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 20:46:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:46:47.989285613 +0000 UTC m=+26.501760803" watchObservedRunningTime="2026-07-01 20:46:47.989600449 +0000 UTC m=+26.502075639"
Jul 01 20:46:48.563069 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:48.563036    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-njzrp"
Jul 01 20:46:48.563703 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:48.563682    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-njzrp"
Jul 01 20:46:48.833870 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:48.833788    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:46:48.834360 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:48.833915    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lw6" podUID="1dfa26e7-23bf-4e56-808d-e59cee52aee7"
Jul 01 20:46:48.975920 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:48.975889    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-njzrp"
Jul 01 20:46:48.976178 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:48.976158    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-njzrp"
Jul 01 20:46:49.836535 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:49.836380    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:46:49.837053 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:49.836427    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:46:49.837053 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:49.836618    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cthwx" podUID="3779f941-e08c-4e16-a1ab-be86f76ee83d"
Jul 01 20:46:49.837053 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:49.836691    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fz5ld" podUID="a02110cb-ee26-4f3e-a854-ececca41783e"
Jul 01 20:46:49.981226 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:49.981198    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx" event={"ID":"2c340476-3fcc-4aa9-9e02-7d78b0b5858c","Type":"ContainerStarted","Data":"1dd2d5019ef7dabe61712822653de1e6b0f90dd26ba46122e0f37075b777aa4c"}
Jul 01 20:46:49.981479 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:49.981459    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:49.983132 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:49.983084    2571 generic.go:356] "Generic (PLEG): container finished" podID="9a45d9e0-af21-4075-9578-09d1913a1082" containerID="21aedc41fff32d9da6e176e7724fdc80f37f887a0891d6c9661a0e072a4c1ec9" exitCode=0
Jul 01 20:46:49.983132 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:49.983116    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qbnbc" event={"ID":"9a45d9e0-af21-4075-9578-09d1913a1082","Type":"ContainerDied","Data":"21aedc41fff32d9da6e176e7724fdc80f37f887a0891d6c9661a0e072a4c1ec9"}
Jul 01 20:46:49.998143 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:49.998126    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:50.018781 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:50.018738    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx" podStartSLOduration=7.707449485 podStartE2EDuration="28.018724614s" totalImagesPullingTime="20.311275129s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 20:46:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:46:50.018327036 +0000 UTC m=+28.530802226" watchObservedRunningTime="2026-07-01 20:46:50.018724614 +0000 UTC m=+28.531199804"
Jul 01 20:46:50.833398 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:50.833359    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:46:50.833607 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:50.833472    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lw6" podUID="1dfa26e7-23bf-4e56-808d-e59cee52aee7"
Jul 01 20:46:50.985218 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:50.985193    2571 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 01 20:46:50.985650 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:50.985613    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:51.001057 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:51.001035    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:51.816749 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:51.816717    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-t6lw6"]
Jul 01 20:46:51.816912 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:51.816811    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:46:51.816960 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:51.816913    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lw6" podUID="1dfa26e7-23bf-4e56-808d-e59cee52aee7"
Jul 01 20:46:51.818968 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:51.818947    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cthwx"]
Jul 01 20:46:51.819057 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:51.819044    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:46:51.819170 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:51.819148    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cthwx" podUID="3779f941-e08c-4e16-a1ab-be86f76ee83d"
Jul 01 20:46:51.831887 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:51.831866    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fz5ld"]
Jul 01 20:46:51.831970 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:51.831941    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:46:51.832019 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:51.831998    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fz5ld" podUID="a02110cb-ee26-4f3e-a854-ececca41783e"
Jul 01 20:46:51.988596 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:51.988418    2571 generic.go:356] "Generic (PLEG): container finished" podID="9a45d9e0-af21-4075-9578-09d1913a1082" containerID="5ecbb2f0446fa2de2886c983229d48716c02ab8ca346f1dea567f9181e4e9860" exitCode=0
Jul 01 20:46:51.988953 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:51.988556    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qbnbc" event={"ID":"9a45d9e0-af21-4075-9578-09d1913a1082","Type":"ContainerDied","Data":"5ecbb2f0446fa2de2886c983229d48716c02ab8ca346f1dea567f9181e4e9860"}
Jul 01 20:46:51.988953 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:51.988756    2571 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 01 20:46:52.990776 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:52.990748    2571 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 01 20:46:53.833343 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:53.833256    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:46:53.833490 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:53.833256    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:46:53.833490 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:53.833372    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fz5ld" podUID="a02110cb-ee26-4f3e-a854-ececca41783e"
Jul 01 20:46:53.833490 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:53.833274    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:46:53.833490 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:53.833427    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lw6" podUID="1dfa26e7-23bf-4e56-808d-e59cee52aee7"
Jul 01 20:46:53.833730 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:53.833529    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cthwx" podUID="3779f941-e08c-4e16-a1ab-be86f76ee83d"
Jul 01 20:46:53.995114 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:53.995082    2571 generic.go:356] "Generic (PLEG): container finished" podID="9a45d9e0-af21-4075-9578-09d1913a1082" containerID="add7db6487ec220fe8ea1d1dd701f2bac8280242852afdfdbe70cea8509bbcaf" exitCode=0
Jul 01 20:46:53.995710 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:53.995142    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qbnbc" event={"ID":"9a45d9e0-af21-4075-9578-09d1913a1082","Type":"ContainerDied","Data":"add7db6487ec220fe8ea1d1dd701f2bac8280242852afdfdbe70cea8509bbcaf"}
Jul 01 20:46:53.998646 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:53.998608    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:46:55.349023 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:55.348991    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ncfhm_eaa37651-7fb2-4625-a15d-e7427a744196/dns-node-resolver/0.log"
Jul 01 20:46:55.531807 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:55.531769    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3779f941-e08c-4e16-a1ab-be86f76ee83d-metrics-certs\") pod \"network-metrics-daemon-cthwx\" (UID: \"3779f941-e08c-4e16-a1ab-be86f76ee83d\") " pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:46:55.531963 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:55.531870    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 20:46:55.531963 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:55.531942    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3779f941-e08c-4e16-a1ab-be86f76ee83d-metrics-certs podName:3779f941-e08c-4e16-a1ab-be86f76ee83d nodeName:}" failed. No retries permitted until 2026-07-01 20:47:27.531920304 +0000 UTC m=+66.044395485 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3779f941-e08c-4e16-a1ab-be86f76ee83d-metrics-certs") pod "network-metrics-daemon-cthwx" (UID: "3779f941-e08c-4e16-a1ab-be86f76ee83d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 20:46:55.632789 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:55.632718    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1dfa26e7-23bf-4e56-808d-e59cee52aee7-original-pull-secret\") pod \"global-pull-secret-syncer-t6lw6\" (UID: \"1dfa26e7-23bf-4e56-808d-e59cee52aee7\") " pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:46:55.632920 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:55.632838    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 20:46:55.632920 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:55.632891    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dfa26e7-23bf-4e56-808d-e59cee52aee7-original-pull-secret podName:1dfa26e7-23bf-4e56-808d-e59cee52aee7 nodeName:}" failed. No retries permitted until 2026-07-01 20:47:27.632874834 +0000 UTC m=+66.145350004 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1dfa26e7-23bf-4e56-808d-e59cee52aee7-original-pull-secret") pod "global-pull-secret-syncer-t6lw6" (UID: "1dfa26e7-23bf-4e56-808d-e59cee52aee7") : object "kube-system"/"original-pull-secret" not registered
Jul 01 20:46:55.733016 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:55.732991    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj4fn\" (UniqueName: \"kubernetes.io/projected/a02110cb-ee26-4f3e-a854-ececca41783e-kube-api-access-cj4fn\") pod \"network-check-target-fz5ld\" (UID: \"a02110cb-ee26-4f3e-a854-ececca41783e\") " pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:46:55.733181 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:55.733163    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 20:46:55.733257 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:55.733191    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 20:46:55.733257 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:55.733204    2571 projected.go:196] Error preparing data for projected volume kube-api-access-cj4fn for pod openshift-network-diagnostics/network-check-target-fz5ld: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 20:46:55.733362 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:55.733267    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a02110cb-ee26-4f3e-a854-ececca41783e-kube-api-access-cj4fn podName:a02110cb-ee26-4f3e-a854-ececca41783e nodeName:}" failed. No retries permitted until 2026-07-01 20:47:27.733247511 +0000 UTC m=+66.245722692 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-cj4fn" (UniqueName: "kubernetes.io/projected/a02110cb-ee26-4f3e-a854-ececca41783e-kube-api-access-cj4fn") pod "network-check-target-fz5ld" (UID: "a02110cb-ee26-4f3e-a854-ececca41783e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 20:46:55.833433 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:55.833407    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:46:55.833589 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:55.833518    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fz5ld" podUID="a02110cb-ee26-4f3e-a854-ececca41783e"
Jul 01 20:46:55.833589 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:55.833542    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:46:55.833788 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:55.833670    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cthwx" podUID="3779f941-e08c-4e16-a1ab-be86f76ee83d"
Jul 01 20:46:55.833788 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:55.833408    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:46:55.833788 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:55.833766    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lw6" podUID="1dfa26e7-23bf-4e56-808d-e59cee52aee7"
Jul 01 20:46:56.332403 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:56.332377    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-swv7c_111e269e-a761-4eee-9a78-3fd554431c39/node-ca/0.log"
Jul 01 20:46:57.835570 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:57.833818    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:46:57.835570 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:57.833834    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:46:57.835570 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:57.833947    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lw6" podUID="1dfa26e7-23bf-4e56-808d-e59cee52aee7"
Jul 01 20:46:57.835570 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:57.834164    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fz5ld" podUID="a02110cb-ee26-4f3e-a854-ececca41783e"
Jul 01 20:46:57.835570 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:57.834500    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:46:57.835570 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:57.834608    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cthwx" podUID="3779f941-e08c-4e16-a1ab-be86f76ee83d"
Jul 01 20:46:59.008923 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:59.008885    2571 generic.go:356] "Generic (PLEG): container finished" podID="9a45d9e0-af21-4075-9578-09d1913a1082" containerID="c182cdef492113b02deb99c164d649d88d4bbd9d9917fe05f336a69d51837534" exitCode=0
Jul 01 20:46:59.009270 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:59.008945    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qbnbc" event={"ID":"9a45d9e0-af21-4075-9578-09d1913a1082","Type":"ContainerDied","Data":"c182cdef492113b02deb99c164d649d88d4bbd9d9917fe05f336a69d51837534"}
Jul 01 20:46:59.833342 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:59.833305    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:46:59.833483 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:59.833348    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:46:59.833483 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:59.833438    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cthwx" podUID="3779f941-e08c-4e16-a1ab-be86f76ee83d"
Jul 01 20:46:59.833483 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:59.833466    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fz5ld" podUID="a02110cb-ee26-4f3e-a854-ececca41783e"
Jul 01 20:46:59.833605 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:46:59.833502    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:46:59.833605 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:46:59.833577    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lw6" podUID="1dfa26e7-23bf-4e56-808d-e59cee52aee7"
Jul 01 20:47:00.013600 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:00.013576    2571 generic.go:356] "Generic (PLEG): container finished" podID="9a45d9e0-af21-4075-9578-09d1913a1082" containerID="ebc3133e6f4d2cb750124806ea0be0b1a7a863e800f5102cf31149706046add3" exitCode=0
Jul 01 20:47:00.013926 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:00.013649    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qbnbc" event={"ID":"9a45d9e0-af21-4075-9578-09d1913a1082","Type":"ContainerDied","Data":"ebc3133e6f4d2cb750124806ea0be0b1a7a863e800f5102cf31149706046add3"}
Jul 01 20:47:01.019280 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:01.019117    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qbnbc" event={"ID":"9a45d9e0-af21-4075-9578-09d1913a1082","Type":"ContainerStarted","Data":"cdb1a99bb1e6e52fd4d95faf3f4432eeb68bdb76ba24ed8edc9458600998ba54"}
Jul 01 20:47:01.045402 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:01.045362    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qbnbc" podStartSLOduration=7.793480045 podStartE2EDuration="39.045347994s" totalImagesPullingTime="31.251867949s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 20:46:22 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:47:01.045333618 +0000 UTC m=+39.557808789" watchObservedRunningTime="2026-07-01 20:47:01.045347994 +0000 UTC m=+39.557823175"
Jul 01 20:47:01.833969 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:01.833939    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:47:01.834113 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:47:01.834014    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lw6" podUID="1dfa26e7-23bf-4e56-808d-e59cee52aee7"
Jul 01 20:47:01.834113 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:01.834067    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:47:01.834223 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:47:01.834154    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cthwx" podUID="3779f941-e08c-4e16-a1ab-be86f76ee83d"
Jul 01 20:47:01.834223 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:01.834177    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:47:01.834317 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:47:01.834283    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fz5ld" podUID="a02110cb-ee26-4f3e-a854-ececca41783e"
Jul 01 20:47:03.833660 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:03.833615    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:47:03.834121 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:03.833615    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:47:03.834121 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:03.833621    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:47:03.834121 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:47:03.833803    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lw6" podUID="1dfa26e7-23bf-4e56-808d-e59cee52aee7"
Jul 01 20:47:03.834121 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:47:03.833717    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fz5ld" podUID="a02110cb-ee26-4f3e-a854-ececca41783e"
Jul 01 20:47:03.834121 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:47:03.833901    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cthwx" podUID="3779f941-e08c-4e16-a1ab-be86f76ee83d"
Jul 01 20:47:05.833515 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:05.833486    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:47:05.833515 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:05.833505    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:47:05.833941 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:05.833486    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:47:05.833941 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:47:05.833588    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lw6" podUID="1dfa26e7-23bf-4e56-808d-e59cee52aee7"
Jul 01 20:47:05.833941 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:47:05.833701    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cthwx" podUID="3779f941-e08c-4e16-a1ab-be86f76ee83d"
Jul 01 20:47:05.833941 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:47:05.833770    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fz5ld" podUID="a02110cb-ee26-4f3e-a854-ececca41783e"
Jul 01 20:47:07.833790 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:07.833759    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:47:07.834285 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:07.833806    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:47:07.834285 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:07.833912    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:47:07.834285 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:47:07.833926    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lw6" podUID="1dfa26e7-23bf-4e56-808d-e59cee52aee7"
Jul 01 20:47:07.834285 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:47:07.834007    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fz5ld" podUID="a02110cb-ee26-4f3e-a854-ececca41783e"
Jul 01 20:47:07.834285 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:47:07.834104    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cthwx" podUID="3779f941-e08c-4e16-a1ab-be86f76ee83d"
Jul 01 20:47:09.833276 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:09.833244    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:47:09.833276 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:09.833263    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:47:09.833699 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:09.833269    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:47:09.833699 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:47:09.833342    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lw6" podUID="1dfa26e7-23bf-4e56-808d-e59cee52aee7"
Jul 01 20:47:09.833699 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:47:09.833413    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fz5ld" podUID="a02110cb-ee26-4f3e-a854-ececca41783e"
Jul 01 20:47:09.833699 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:47:09.833486    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cthwx" podUID="3779f941-e08c-4e16-a1ab-be86f76ee83d"
Jul 01 20:47:11.834807 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:11.834773    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:47:11.835200 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:47:11.834894    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lw6" podUID="1dfa26e7-23bf-4e56-808d-e59cee52aee7"
Jul 01 20:47:11.835200 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:11.834944    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:47:11.835200 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:47:11.835019    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fz5ld" podUID="a02110cb-ee26-4f3e-a854-ececca41783e"
Jul 01 20:47:11.835200 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:11.835038    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:47:11.835200 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:47:11.835168    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cthwx" podUID="3779f941-e08c-4e16-a1ab-be86f76ee83d"
Jul 01 20:47:13.833201 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:13.833170    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:47:13.833602 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:47:13.833266    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fz5ld" podUID="a02110cb-ee26-4f3e-a854-ececca41783e"
Jul 01 20:47:13.833602 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:13.833170    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:47:13.833602 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:47:13.833371    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lw6" podUID="1dfa26e7-23bf-4e56-808d-e59cee52aee7"
Jul 01 20:47:13.833602 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:13.833389    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:47:13.833602 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:47:13.833459    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cthwx" podUID="3779f941-e08c-4e16-a1ab-be86f76ee83d"
Jul 01 20:47:15.833355 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:15.833331    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:47:15.833798 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:15.833341    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:47:15.833798 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:47:15.833436    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cthwx" podUID="3779f941-e08c-4e16-a1ab-be86f76ee83d"
Jul 01 20:47:15.833798 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:47:15.833504    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lw6" podUID="1dfa26e7-23bf-4e56-808d-e59cee52aee7"
Jul 01 20:47:15.833798 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:15.833560    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:47:15.833798 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:47:15.833616    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fz5ld" podUID="a02110cb-ee26-4f3e-a854-ececca41783e"
Jul 01 20:47:17.833381 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:17.833350    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:47:17.833771 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:17.833360    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:47:17.833771 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:17.833478    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:47:17.833771 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:47:17.833468    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lw6" podUID="1dfa26e7-23bf-4e56-808d-e59cee52aee7"
Jul 01 20:47:17.833771 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:47:17.833553    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fz5ld" podUID="a02110cb-ee26-4f3e-a854-ececca41783e"
Jul 01 20:47:17.833771 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:47:17.833613    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cthwx" podUID="3779f941-e08c-4e16-a1ab-be86f76ee83d"
Jul 01 20:47:19.833270 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:19.833229    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:47:19.833270 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:19.833240    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:47:19.833853 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:47:19.833368    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lw6" podUID="1dfa26e7-23bf-4e56-808d-e59cee52aee7"
Jul 01 20:47:19.833853 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:47:19.833422    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fz5ld" podUID="a02110cb-ee26-4f3e-a854-ececca41783e"
Jul 01 20:47:19.833853 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:19.833472    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:47:19.833853 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:47:19.833549    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cthwx" podUID="3779f941-e08c-4e16-a1ab-be86f76ee83d"
Jul 01 20:47:21.276899 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.276872    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-104.ec2.internal" event="NodeReady"
Jul 01 20:47:21.277269 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.276991    2571 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 01 20:47:21.332115 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.332086    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-54dbbfbdcd-xq7ll"]
Jul 01 20:47:21.333805 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.333790    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54dbbfbdcd-xq7ll"
Jul 01 20:47:21.341001 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.340966    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 01 20:47:21.341161 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.341144    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 01 20:47:21.341345 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.341327    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 01 20:47:21.341451 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.341437    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-txm48\""
Jul 01 20:47:21.347783 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.347757    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 01 20:47:21.358272 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.358180    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-54dbbfbdcd-xq7ll"]
Jul 01 20:47:21.363264 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.363240    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6xhrz"]
Jul 01 20:47:21.365173 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.365159    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6xhrz"
Jul 01 20:47:21.369671 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.369649    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 01 20:47:21.369774 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.369707    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-f5twz\""
Jul 01 20:47:21.369893 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.369872    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 01 20:47:21.373066 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.373046    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-wpqxs"]
Jul 01 20:47:21.374933 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.374918    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wpqxs"
Jul 01 20:47:21.380431 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.380415    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 01 20:47:21.380676 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.380654    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 01 20:47:21.380977 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.380791    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9fsnr\""
Jul 01 20:47:21.380977 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.380820    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 01 20:47:21.380977 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.380887    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 01 20:47:21.382165 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.382142    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6xhrz"]
Jul 01 20:47:21.395949 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.395915    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wpqxs"]
Jul 01 20:47:21.452714 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.452676    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-r64tx"]
Jul 01 20:47:21.454668 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.454647    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-r64tx"
Jul 01 20:47:21.458318 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.458300    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 01 20:47:21.458561 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.458546    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7ck8j\""
Jul 01 20:47:21.458670 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.458599    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 01 20:47:21.458670 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.458643    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 01 20:47:21.465088 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.465066    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-r64tx"]
Jul 01 20:47:21.520394 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.520353    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6ac76942-02a5-4206-97e1-3329bcea5c93-metrics-tls\") pod \"dns-default-6xhrz\" (UID: \"6ac76942-02a5-4206-97e1-3329bcea5c93\") " pod="openshift-dns/dns-default-6xhrz"
Jul 01 20:47:21.520394 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.520389    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qq4jj\" (UniqueName: \"kubernetes.io/projected/6ac76942-02a5-4206-97e1-3329bcea5c93-kube-api-access-qq4jj\") pod \"dns-default-6xhrz\" (UID: \"6ac76942-02a5-4206-97e1-3329bcea5c93\") " pod="openshift-dns/dns-default-6xhrz"
Jul 01 20:47:21.520620 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.520406    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/da6e12d2-79f9-4aef-a545-2329fae14c68-data-volume\") pod \"insights-runtime-extractor-wpqxs\" (UID: \"da6e12d2-79f9-4aef-a545-2329fae14c68\") " pod="openshift-insights/insights-runtime-extractor-wpqxs"
Jul 01 20:47:21.520620 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.520449    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5dec1da6-cda0-4fb5-832a-36cea048f4d4-bound-sa-token\") pod \"image-registry-54dbbfbdcd-xq7ll\" (UID: \"5dec1da6-cda0-4fb5-832a-36cea048f4d4\") " pod="openshift-image-registry/image-registry-54dbbfbdcd-xq7ll"
Jul 01 20:47:21.520620 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.520471    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5dec1da6-cda0-4fb5-832a-36cea048f4d4-image-registry-private-configuration\") pod \"image-registry-54dbbfbdcd-xq7ll\" (UID: \"5dec1da6-cda0-4fb5-832a-36cea048f4d4\") " pod="openshift-image-registry/image-registry-54dbbfbdcd-xq7ll"
Jul 01 20:47:21.520620 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.520487    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dec1da6-cda0-4fb5-832a-36cea048f4d4-registry-tls\") pod \"image-registry-54dbbfbdcd-xq7ll\" (UID: \"5dec1da6-cda0-4fb5-832a-36cea048f4d4\") " pod="openshift-image-registry/image-registry-54dbbfbdcd-xq7ll"
Jul 01 20:47:21.520620 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.520508    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5dec1da6-cda0-4fb5-832a-36cea048f4d4-ca-trust-extracted\") pod \"image-registry-54dbbfbdcd-xq7ll\" (UID: \"5dec1da6-cda0-4fb5-832a-36cea048f4d4\") " pod="openshift-image-registry/image-registry-54dbbfbdcd-xq7ll"
Jul 01 20:47:21.520620 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.520561    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5dec1da6-cda0-4fb5-832a-36cea048f4d4-trusted-ca\") pod \"image-registry-54dbbfbdcd-xq7ll\" (UID: \"5dec1da6-cda0-4fb5-832a-36cea048f4d4\") " pod="openshift-image-registry/image-registry-54dbbfbdcd-xq7ll"
Jul 01 20:47:21.520620 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.520579    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5dec1da6-cda0-4fb5-832a-36cea048f4d4-installation-pull-secrets\") pod \"image-registry-54dbbfbdcd-xq7ll\" (UID: \"5dec1da6-cda0-4fb5-832a-36cea048f4d4\") " pod="openshift-image-registry/image-registry-54dbbfbdcd-xq7ll"
Jul 01 20:47:21.520620 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.520601    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6wzs\" (UniqueName: \"kubernetes.io/projected/5dec1da6-cda0-4fb5-832a-36cea048f4d4-kube-api-access-r6wzs\") pod \"image-registry-54dbbfbdcd-xq7ll\" (UID: \"5dec1da6-cda0-4fb5-832a-36cea048f4d4\") " pod="openshift-image-registry/image-registry-54dbbfbdcd-xq7ll"
Jul 01 20:47:21.520620 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.520617    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6ac76942-02a5-4206-97e1-3329bcea5c93-config-volume\") pod \"dns-default-6xhrz\" (UID: \"6ac76942-02a5-4206-97e1-3329bcea5c93\") " pod="openshift-dns/dns-default-6xhrz"
Jul 01 20:47:21.520983 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.520697    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mltx\" (UniqueName: \"kubernetes.io/projected/da6e12d2-79f9-4aef-a545-2329fae14c68-kube-api-access-2mltx\") pod \"insights-runtime-extractor-wpqxs\" (UID: \"da6e12d2-79f9-4aef-a545-2329fae14c68\") " pod="openshift-insights/insights-runtime-extractor-wpqxs"
Jul 01 20:47:21.520983 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.520746    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ac76942-02a5-4206-97e1-3329bcea5c93-tmp-dir\") pod \"dns-default-6xhrz\" (UID: \"6ac76942-02a5-4206-97e1-3329bcea5c93\") " pod="openshift-dns/dns-default-6xhrz"
Jul 01 20:47:21.520983 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.520767    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/da6e12d2-79f9-4aef-a545-2329fae14c68-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wpqxs\" (UID: \"da6e12d2-79f9-4aef-a545-2329fae14c68\") " pod="openshift-insights/insights-runtime-extractor-wpqxs"
Jul 01 20:47:21.520983 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.520787    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/da6e12d2-79f9-4aef-a545-2329fae14c68-crio-socket\") pod \"insights-runtime-extractor-wpqxs\" (UID: \"da6e12d2-79f9-4aef-a545-2329fae14c68\") " pod="openshift-insights/insights-runtime-extractor-wpqxs"
Jul 01 20:47:21.520983 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.520806    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/da6e12d2-79f9-4aef-a545-2329fae14c68-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wpqxs\" (UID: \"da6e12d2-79f9-4aef-a545-2329fae14c68\") " pod="openshift-insights/insights-runtime-extractor-wpqxs"
Jul 01 20:47:21.520983 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.520902    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5dec1da6-cda0-4fb5-832a-36cea048f4d4-registry-certificates\") pod \"image-registry-54dbbfbdcd-xq7ll\" (UID: \"5dec1da6-cda0-4fb5-832a-36cea048f4d4\") " pod="openshift-image-registry/image-registry-54dbbfbdcd-xq7ll"
Jul 01 20:47:21.621843 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.621740    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5dec1da6-cda0-4fb5-832a-36cea048f4d4-trusted-ca\") pod \"image-registry-54dbbfbdcd-xq7ll\" (UID: \"5dec1da6-cda0-4fb5-832a-36cea048f4d4\") " pod="openshift-image-registry/image-registry-54dbbfbdcd-xq7ll"
Jul 01 20:47:21.621843 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.621791    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5dec1da6-cda0-4fb5-832a-36cea048f4d4-installation-pull-secrets\") pod \"image-registry-54dbbfbdcd-xq7ll\" (UID: \"5dec1da6-cda0-4fb5-832a-36cea048f4d4\") " pod="openshift-image-registry/image-registry-54dbbfbdcd-xq7ll"
Jul 01 20:47:21.621843 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.621817    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6wzs\" (UniqueName: \"kubernetes.io/projected/5dec1da6-cda0-4fb5-832a-36cea048f4d4-kube-api-access-r6wzs\") pod \"image-registry-54dbbfbdcd-xq7ll\" (UID: \"5dec1da6-cda0-4fb5-832a-36cea048f4d4\") " pod="openshift-image-registry/image-registry-54dbbfbdcd-xq7ll"
Jul 01 20:47:21.621843 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.621842    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6ac76942-02a5-4206-97e1-3329bcea5c93-config-volume\") pod \"dns-default-6xhrz\" (UID: \"6ac76942-02a5-4206-97e1-3329bcea5c93\") " pod="openshift-dns/dns-default-6xhrz"
Jul 01 20:47:21.622169 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.621869    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mltx\" (UniqueName: \"kubernetes.io/projected/da6e12d2-79f9-4aef-a545-2329fae14c68-kube-api-access-2mltx\") pod \"insights-runtime-extractor-wpqxs\" (UID: \"da6e12d2-79f9-4aef-a545-2329fae14c68\") " pod="openshift-insights/insights-runtime-extractor-wpqxs"
Jul 01 20:47:21.622169 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.621912    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ac76942-02a5-4206-97e1-3329bcea5c93-tmp-dir\") pod \"dns-default-6xhrz\" (UID: \"6ac76942-02a5-4206-97e1-3329bcea5c93\") " pod="openshift-dns/dns-default-6xhrz"
Jul 01 20:47:21.622169 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.621936    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/da6e12d2-79f9-4aef-a545-2329fae14c68-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wpqxs\" (UID: \"da6e12d2-79f9-4aef-a545-2329fae14c68\") " pod="openshift-insights/insights-runtime-extractor-wpqxs"
Jul 01 20:47:21.622169 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.622074    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/da6e12d2-79f9-4aef-a545-2329fae14c68-crio-socket\") pod \"insights-runtime-extractor-wpqxs\" (UID: \"da6e12d2-79f9-4aef-a545-2329fae14c68\") " pod="openshift-insights/insights-runtime-extractor-wpqxs"
Jul 01 20:47:21.622169 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.622121    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/da6e12d2-79f9-4aef-a545-2329fae14c68-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wpqxs\" (UID: \"da6e12d2-79f9-4aef-a545-2329fae14c68\") " pod="openshift-insights/insights-runtime-extractor-wpqxs"
Jul 01 20:47:21.622169 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.622140    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/da6e12d2-79f9-4aef-a545-2329fae14c68-crio-socket\") pod \"insights-runtime-extractor-wpqxs\" (UID: \"da6e12d2-79f9-4aef-a545-2329fae14c68\") " pod="openshift-insights/insights-runtime-extractor-wpqxs"
Jul 01 20:47:21.622423 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.622182    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5dec1da6-cda0-4fb5-832a-36cea048f4d4-registry-certificates\") pod \"image-registry-54dbbfbdcd-xq7ll\" (UID: \"5dec1da6-cda0-4fb5-832a-36cea048f4d4\") " pod="openshift-image-registry/image-registry-54dbbfbdcd-xq7ll"
Jul 01 20:47:21.622423 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.622227    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6ac76942-02a5-4206-97e1-3329bcea5c93-metrics-tls\") pod \"dns-default-6xhrz\" (UID: \"6ac76942-02a5-4206-97e1-3329bcea5c93\") " pod="openshift-dns/dns-default-6xhrz"
Jul 01 20:47:21.622423 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.622249    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qq4jj\" (UniqueName: \"kubernetes.io/projected/6ac76942-02a5-4206-97e1-3329bcea5c93-kube-api-access-qq4jj\") pod \"dns-default-6xhrz\" (UID: \"6ac76942-02a5-4206-97e1-3329bcea5c93\") " pod="openshift-dns/dns-default-6xhrz"
Jul 01 20:47:21.622423 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.622270    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/da6e12d2-79f9-4aef-a545-2329fae14c68-data-volume\") pod \"insights-runtime-extractor-wpqxs\" (UID: \"da6e12d2-79f9-4aef-a545-2329fae14c68\") " pod="openshift-insights/insights-runtime-extractor-wpqxs"
Jul 01 20:47:21.622423 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.622280    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ac76942-02a5-4206-97e1-3329bcea5c93-tmp-dir\") pod \"dns-default-6xhrz\" (UID: \"6ac76942-02a5-4206-97e1-3329bcea5c93\") " pod="openshift-dns/dns-default-6xhrz"
Jul 01 20:47:21.622423 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.622303    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5dec1da6-cda0-4fb5-832a-36cea048f4d4-bound-sa-token\") pod \"image-registry-54dbbfbdcd-xq7ll\" (UID: \"5dec1da6-cda0-4fb5-832a-36cea048f4d4\") " pod="openshift-image-registry/image-registry-54dbbfbdcd-xq7ll"
Jul 01 20:47:21.622423 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.622333    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5dec1da6-cda0-4fb5-832a-36cea048f4d4-image-registry-private-configuration\") pod \"image-registry-54dbbfbdcd-xq7ll\" (UID: \"5dec1da6-cda0-4fb5-832a-36cea048f4d4\") " pod="openshift-image-registry/image-registry-54dbbfbdcd-xq7ll"
Jul 01 20:47:21.622423 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.622356    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dec1da6-cda0-4fb5-832a-36cea048f4d4-registry-tls\") pod \"image-registry-54dbbfbdcd-xq7ll\" (UID: \"5dec1da6-cda0-4fb5-832a-36cea048f4d4\") " pod="openshift-image-registry/image-registry-54dbbfbdcd-xq7ll"
Jul 01 20:47:21.622423 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.622382    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5dec1da6-cda0-4fb5-832a-36cea048f4d4-ca-trust-extracted\") pod \"image-registry-54dbbfbdcd-xq7ll\" (UID: \"5dec1da6-cda0-4fb5-832a-36cea048f4d4\") " pod="openshift-image-registry/image-registry-54dbbfbdcd-xq7ll"
Jul 01 20:47:21.622423 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.622409    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5df26038-9551-4259-af17-826c1c3480f7-cert\") pod \"ingress-canary-r64tx\" (UID: \"5df26038-9551-4259-af17-826c1c3480f7\") " pod="openshift-ingress-canary/ingress-canary-r64tx"
Jul 01 20:47:21.622885 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.622430    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-842g6\" (UniqueName: \"kubernetes.io/projected/5df26038-9551-4259-af17-826c1c3480f7-kube-api-access-842g6\") pod \"ingress-canary-r64tx\" (UID: \"5df26038-9551-4259-af17-826c1c3480f7\") " pod="openshift-ingress-canary/ingress-canary-r64tx"
Jul 01 20:47:21.622885 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.622582    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6ac76942-02a5-4206-97e1-3329bcea5c93-config-volume\") pod \"dns-default-6xhrz\" (UID: \"6ac76942-02a5-4206-97e1-3329bcea5c93\") " pod="openshift-dns/dns-default-6xhrz"
Jul 01 20:47:21.622885 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.622799    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/da6e12d2-79f9-4aef-a545-2329fae14c68-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wpqxs\" (UID: \"da6e12d2-79f9-4aef-a545-2329fae14c68\") " pod="openshift-insights/insights-runtime-extractor-wpqxs"
Jul 01 20:47:21.622885 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.622811    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/da6e12d2-79f9-4aef-a545-2329fae14c68-data-volume\") pod \"insights-runtime-extractor-wpqxs\" (UID: \"da6e12d2-79f9-4aef-a545-2329fae14c68\") " pod="openshift-insights/insights-runtime-extractor-wpqxs"
Jul 01 20:47:21.623128 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.623087    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5dec1da6-cda0-4fb5-832a-36cea048f4d4-ca-trust-extracted\") pod \"image-registry-54dbbfbdcd-xq7ll\" (UID: \"5dec1da6-cda0-4fb5-832a-36cea048f4d4\") " pod="openshift-image-registry/image-registry-54dbbfbdcd-xq7ll"
Jul 01 20:47:21.623347 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.623325    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5dec1da6-cda0-4fb5-832a-36cea048f4d4-registry-certificates\") pod \"image-registry-54dbbfbdcd-xq7ll\" (UID: \"5dec1da6-cda0-4fb5-832a-36cea048f4d4\") " pod="openshift-image-registry/image-registry-54dbbfbdcd-xq7ll"
Jul 01 20:47:21.623575 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.623553    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5dec1da6-cda0-4fb5-832a-36cea048f4d4-trusted-ca\") pod \"image-registry-54dbbfbdcd-xq7ll\" (UID: \"5dec1da6-cda0-4fb5-832a-36cea048f4d4\") " pod="openshift-image-registry/image-registry-54dbbfbdcd-xq7ll"
Jul 01 20:47:21.626352 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.626324    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6ac76942-02a5-4206-97e1-3329bcea5c93-metrics-tls\") pod \"dns-default-6xhrz\" (UID: \"6ac76942-02a5-4206-97e1-3329bcea5c93\") " pod="openshift-dns/dns-default-6xhrz"
Jul 01 20:47:21.626444 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.626369    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/da6e12d2-79f9-4aef-a545-2329fae14c68-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wpqxs\" (UID: \"da6e12d2-79f9-4aef-a545-2329fae14c68\") " pod="openshift-insights/insights-runtime-extractor-wpqxs"
Jul 01 20:47:21.626444 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.626369    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5dec1da6-cda0-4fb5-832a-36cea048f4d4-image-registry-private-configuration\") pod \"image-registry-54dbbfbdcd-xq7ll\" (UID: \"5dec1da6-cda0-4fb5-832a-36cea048f4d4\") " pod="openshift-image-registry/image-registry-54dbbfbdcd-xq7ll"
Jul 01 20:47:21.626444 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.626418    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dec1da6-cda0-4fb5-832a-36cea048f4d4-registry-tls\") pod \"image-registry-54dbbfbdcd-xq7ll\" (UID: \"5dec1da6-cda0-4fb5-832a-36cea048f4d4\") " pod="openshift-image-registry/image-registry-54dbbfbdcd-xq7ll"
Jul 01 20:47:21.626444 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.626436    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5dec1da6-cda0-4fb5-832a-36cea048f4d4-installation-pull-secrets\") pod \"image-registry-54dbbfbdcd-xq7ll\" (UID: \"5dec1da6-cda0-4fb5-832a-36cea048f4d4\") " pod="openshift-image-registry/image-registry-54dbbfbdcd-xq7ll"
Jul 01 20:47:21.631614 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.631585    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5dec1da6-cda0-4fb5-832a-36cea048f4d4-bound-sa-token\") pod \"image-registry-54dbbfbdcd-xq7ll\" (UID: \"5dec1da6-cda0-4fb5-832a-36cea048f4d4\") " pod="openshift-image-registry/image-registry-54dbbfbdcd-xq7ll"
Jul 01 20:47:21.634564 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.634537    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mltx\" (UniqueName: \"kubernetes.io/projected/da6e12d2-79f9-4aef-a545-2329fae14c68-kube-api-access-2mltx\") pod \"insights-runtime-extractor-wpqxs\" (UID: \"da6e12d2-79f9-4aef-a545-2329fae14c68\") " pod="openshift-insights/insights-runtime-extractor-wpqxs"
Jul 01 20:47:21.634828 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.634812    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6wzs\" (UniqueName: \"kubernetes.io/projected/5dec1da6-cda0-4fb5-832a-36cea048f4d4-kube-api-access-r6wzs\") pod \"image-registry-54dbbfbdcd-xq7ll\" (UID: \"5dec1da6-cda0-4fb5-832a-36cea048f4d4\") " pod="openshift-image-registry/image-registry-54dbbfbdcd-xq7ll"
Jul 01 20:47:21.635726 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.635704    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qq4jj\" (UniqueName: \"kubernetes.io/projected/6ac76942-02a5-4206-97e1-3329bcea5c93-kube-api-access-qq4jj\") pod \"dns-default-6xhrz\" (UID: \"6ac76942-02a5-4206-97e1-3329bcea5c93\") " pod="openshift-dns/dns-default-6xhrz"
Jul 01 20:47:21.642555 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.642537    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54dbbfbdcd-xq7ll"
Jul 01 20:47:21.674606 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.674566    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6xhrz"
Jul 01 20:47:21.683405 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.683373    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wpqxs"
Jul 01 20:47:21.724521 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.723519    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5df26038-9551-4259-af17-826c1c3480f7-cert\") pod \"ingress-canary-r64tx\" (UID: \"5df26038-9551-4259-af17-826c1c3480f7\") " pod="openshift-ingress-canary/ingress-canary-r64tx"
Jul 01 20:47:21.724521 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.723572    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-842g6\" (UniqueName: \"kubernetes.io/projected/5df26038-9551-4259-af17-826c1c3480f7-kube-api-access-842g6\") pod \"ingress-canary-r64tx\" (UID: \"5df26038-9551-4259-af17-826c1c3480f7\") " pod="openshift-ingress-canary/ingress-canary-r64tx"
Jul 01 20:47:21.726840 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.726813    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5df26038-9551-4259-af17-826c1c3480f7-cert\") pod \"ingress-canary-r64tx\" (UID: \"5df26038-9551-4259-af17-826c1c3480f7\") " pod="openshift-ingress-canary/ingress-canary-r64tx"
Jul 01 20:47:21.733717 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.733660    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-842g6\" (UniqueName: \"kubernetes.io/projected/5df26038-9551-4259-af17-826c1c3480f7-kube-api-access-842g6\") pod \"ingress-canary-r64tx\" (UID: \"5df26038-9551-4259-af17-826c1c3480f7\") " pod="openshift-ingress-canary/ingress-canary-r64tx"
Jul 01 20:47:21.773700 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.769538    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7ck8j\""
Jul 01 20:47:21.773700 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.773686    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-r64tx"
Jul 01 20:47:21.834816 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.833655    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:47:21.834816 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.833830    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:47:21.834816 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.833662    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:47:21.840730 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.840202    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 01 20:47:21.840730 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.840553    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v6s46\""
Jul 01 20:47:21.840730 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.840567    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 01 20:47:21.840730 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.840585    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 01 20:47:21.841018 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.840807    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 01 20:47:21.841090 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.841072    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4g6n6\""
Jul 01 20:47:21.843151 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.843112    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-54dbbfbdcd-xq7ll"]
Jul 01 20:47:21.850139 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:47:21.850113    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dec1da6_cda0_4fb5_832a_36cea048f4d4.slice/crio-64ffb1ab07b52c103e17cd54f356cad6a41a6ad51079d62628b37caa89a7739a WatchSource:0}: Error finding container 64ffb1ab07b52c103e17cd54f356cad6a41a6ad51079d62628b37caa89a7739a: Status 404 returned error can't find the container with id 64ffb1ab07b52c103e17cd54f356cad6a41a6ad51079d62628b37caa89a7739a
Jul 01 20:47:21.861858 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.861696    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6xhrz"]
Jul 01 20:47:21.884145 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:21.884112    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wpqxs"]
Jul 01 20:47:21.888313 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:47:21.888283    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda6e12d2_79f9_4aef_a545_2329fae14c68.slice/crio-c5b70961e3814b275f24279dcd335a4b94af6094c05290afeacd2ef19381ac27 WatchSource:0}: Error finding container c5b70961e3814b275f24279dcd335a4b94af6094c05290afeacd2ef19381ac27: Status 404 returned error can't find the container with id c5b70961e3814b275f24279dcd335a4b94af6094c05290afeacd2ef19381ac27
Jul 01 20:47:22.062376 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:22.062332    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wpqxs" event={"ID":"da6e12d2-79f9-4aef-a545-2329fae14c68","Type":"ContainerStarted","Data":"c5b70961e3814b275f24279dcd335a4b94af6094c05290afeacd2ef19381ac27"}
Jul 01 20:47:22.063383 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:22.063360    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6xhrz" event={"ID":"6ac76942-02a5-4206-97e1-3329bcea5c93","Type":"ContainerStarted","Data":"b55f5c4d8ae07fece8e83b6f612b7961748ab0df459e36c1a769ffd5172021f8"}
Jul 01 20:47:22.064813 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:22.064793    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54dbbfbdcd-xq7ll" event={"ID":"5dec1da6-cda0-4fb5-832a-36cea048f4d4","Type":"ContainerStarted","Data":"c420a17b9c81caff8c2fc1fe731b39f62923bbafded47e2cc426562329ccd406"}
Jul 01 20:47:22.064896 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:22.064826    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54dbbfbdcd-xq7ll" event={"ID":"5dec1da6-cda0-4fb5-832a-36cea048f4d4","Type":"ContainerStarted","Data":"64ffb1ab07b52c103e17cd54f356cad6a41a6ad51079d62628b37caa89a7739a"}
Jul 01 20:47:22.064953 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:22.064943    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-54dbbfbdcd-xq7ll"
Jul 01 20:47:22.084864 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:22.084805    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-54dbbfbdcd-xq7ll" podStartSLOduration=2.084781691 podStartE2EDuration="2.084781691s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:47:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:47:22.084288568 +0000 UTC m=+60.596763759" watchObservedRunningTime="2026-07-01 20:47:22.084781691 +0000 UTC m=+60.597256881"
Jul 01 20:47:22.156830 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:22.156801    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-r64tx"]
Jul 01 20:47:22.160730 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:47:22.160701    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5df26038_9551_4259_af17_826c1c3480f7.slice/crio-35b94c8f9f0db02cad8afdc09290a75e3e4fe09f0c3d996e8091964b975f3e00 WatchSource:0}: Error finding container 35b94c8f9f0db02cad8afdc09290a75e3e4fe09f0c3d996e8091964b975f3e00: Status 404 returned error can't find the container with id 35b94c8f9f0db02cad8afdc09290a75e3e4fe09f0c3d996e8091964b975f3e00
Jul 01 20:47:23.070687 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:23.070586    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-r64tx" event={"ID":"5df26038-9551-4259-af17-826c1c3480f7","Type":"ContainerStarted","Data":"35b94c8f9f0db02cad8afdc09290a75e3e4fe09f0c3d996e8091964b975f3e00"}
Jul 01 20:47:24.019349 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:24.019316    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sw9rx"
Jul 01 20:47:27.565124 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:27.565045    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3779f941-e08c-4e16-a1ab-be86f76ee83d-metrics-certs\") pod \"network-metrics-daemon-cthwx\" (UID: \"3779f941-e08c-4e16-a1ab-be86f76ee83d\") " pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:47:27.567760 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:27.567729    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3779f941-e08c-4e16-a1ab-be86f76ee83d-metrics-certs\") pod \"network-metrics-daemon-cthwx\" (UID: \"3779f941-e08c-4e16-a1ab-be86f76ee83d\") " pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:47:27.666041 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:27.666012    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1dfa26e7-23bf-4e56-808d-e59cee52aee7-original-pull-secret\") pod \"global-pull-secret-syncer-t6lw6\" (UID: \"1dfa26e7-23bf-4e56-808d-e59cee52aee7\") " pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:47:27.668242 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:27.668215    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1dfa26e7-23bf-4e56-808d-e59cee52aee7-original-pull-secret\") pod \"global-pull-secret-syncer-t6lw6\" (UID: \"1dfa26e7-23bf-4e56-808d-e59cee52aee7\") " pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:47:27.766889 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:27.766862    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj4fn\" (UniqueName: \"kubernetes.io/projected/a02110cb-ee26-4f3e-a854-ececca41783e-kube-api-access-cj4fn\") pod \"network-check-target-fz5ld\" (UID: \"a02110cb-ee26-4f3e-a854-ececca41783e\") " pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:47:27.769166 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:27.769141    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cj4fn\" (UniqueName: \"kubernetes.io/projected/a02110cb-ee26-4f3e-a854-ececca41783e-kube-api-access-cj4fn\") pod \"network-check-target-fz5ld\" (UID: \"a02110cb-ee26-4f3e-a854-ececca41783e\") " pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:47:27.850103 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:27.850056    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cthwx"
Jul 01 20:47:27.860229 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:27.860205    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:47:27.869416 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:27.869393    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lw6"
Jul 01 20:47:28.008194 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:28.008167    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cthwx"]
Jul 01 20:47:28.026333 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:28.026304    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fz5ld"]
Jul 01 20:47:28.038523 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:28.038499    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-t6lw6"]
Jul 01 20:47:28.041791 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:47:28.041769    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3779f941_e08c_4e16_a1ab_be86f76ee83d.slice/crio-ebe81fbc310763f056c346eca9cd2a8f1683ec59b6a406060b87753e92906bb5 WatchSource:0}: Error finding container ebe81fbc310763f056c346eca9cd2a8f1683ec59b6a406060b87753e92906bb5: Status 404 returned error can't find the container with id ebe81fbc310763f056c346eca9cd2a8f1683ec59b6a406060b87753e92906bb5
Jul 01 20:47:28.042386 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:47:28.042361    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dfa26e7_23bf_4e56_808d_e59cee52aee7.slice/crio-cb71712be571271084381bc476016fd8ad51085e594dd5c19351c4e27242b0b6 WatchSource:0}: Error finding container cb71712be571271084381bc476016fd8ad51085e594dd5c19351c4e27242b0b6: Status 404 returned error can't find the container with id cb71712be571271084381bc476016fd8ad51085e594dd5c19351c4e27242b0b6
Jul 01 20:47:28.043333 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:47:28.043315    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda02110cb_ee26_4f3e_a854_ececca41783e.slice/crio-97125cec9a8c6b4e4f95a1f3df09447c284ec81326365b27107579330df18933 WatchSource:0}: Error finding container 97125cec9a8c6b4e4f95a1f3df09447c284ec81326365b27107579330df18933: Status 404 returned error can't find the container with id 97125cec9a8c6b4e4f95a1f3df09447c284ec81326365b27107579330df18933
Jul 01 20:47:28.085609 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:28.085585    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6xhrz" event={"ID":"6ac76942-02a5-4206-97e1-3329bcea5c93","Type":"ContainerStarted","Data":"2980324e46e0650a6bc889376759b645a1fdf201a9e272f817c2c0612e3e082a"}
Jul 01 20:47:28.085703 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:28.085648    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6xhrz" event={"ID":"6ac76942-02a5-4206-97e1-3329bcea5c93","Type":"ContainerStarted","Data":"c0f4d46dcde562700149ee6040a9c3d04c982238632a58b3efafe4c1cbc8217e"}
Jul 01 20:47:28.085762 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:28.085710    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6xhrz"
Jul 01 20:47:28.087050 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:28.087031    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-r64tx" event={"ID":"5df26038-9551-4259-af17-826c1c3480f7","Type":"ContainerStarted","Data":"9d2da468b52b83789e4bfb35497a8ee820d5f8c8bbdb04ccbfddd5e5ff64b6fe"}
Jul 01 20:47:28.088389 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:28.088370    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wpqxs" event={"ID":"da6e12d2-79f9-4aef-a545-2329fae14c68","Type":"ContainerStarted","Data":"73d5f98877b084bcbf9a9d37519c72225bfd118108952f85aaa9f7cfa0260e6c"}
Jul 01 20:47:28.089490 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:28.089464    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fz5ld" event={"ID":"a02110cb-ee26-4f3e-a854-ececca41783e","Type":"ContainerStarted","Data":"97125cec9a8c6b4e4f95a1f3df09447c284ec81326365b27107579330df18933"}
Jul 01 20:47:28.090529 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:28.090510    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cthwx" event={"ID":"3779f941-e08c-4e16-a1ab-be86f76ee83d","Type":"ContainerStarted","Data":"ebe81fbc310763f056c346eca9cd2a8f1683ec59b6a406060b87753e92906bb5"}
Jul 01 20:47:28.091642 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:28.091599    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-t6lw6" event={"ID":"1dfa26e7-23bf-4e56-808d-e59cee52aee7","Type":"ContainerStarted","Data":"cb71712be571271084381bc476016fd8ad51085e594dd5c19351c4e27242b0b6"}
Jul 01 20:47:28.107437 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:28.107372    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-6xhrz" podStartSLOduration=1.696428167 podStartE2EDuration="7.107359771s" totalImagesPullingTime="5.410931604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:47:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:47:28.107055866 +0000 UTC m=+66.619531057" watchObservedRunningTime="2026-07-01 20:47:28.107359771 +0000 UTC m=+66.619834939"
Jul 01 20:47:28.124693 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:28.124649    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-r64tx" podStartSLOduration=1.998363991 podStartE2EDuration="7.124608331s" totalImagesPullingTime="5.12624434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:47:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:47:28.124267685 +0000 UTC m=+66.636742875" watchObservedRunningTime="2026-07-01 20:47:28.124608331 +0000 UTC m=+66.637083521"
Jul 01 20:47:29.100246 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:29.100206    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wpqxs" event={"ID":"da6e12d2-79f9-4aef-a545-2329fae14c68","Type":"ContainerStarted","Data":"b0d853bbcf7fee6c872fa71aa487406bcd4f11a571b4312f0ce5cdca5ccec29c"}
Jul 01 20:47:30.107349 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:30.107304    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cthwx" event={"ID":"3779f941-e08c-4e16-a1ab-be86f76ee83d","Type":"ContainerStarted","Data":"f1a4d262aedc6f9a5a63f008898ff71558db70d2038332926ff29caa95881ad4"}
Jul 01 20:47:30.107799 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:30.107377    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cthwx" event={"ID":"3779f941-e08c-4e16-a1ab-be86f76ee83d","Type":"ContainerStarted","Data":"b2de448456224c8dbaea2bae831e397bd462d7233ad5f3e5d0f59a72b197f6dd"}
Jul 01 20:47:30.128755 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:30.128701    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-cthwx" podStartSLOduration=66.913522916 podStartE2EDuration="1m8.128681503s" totalImagesPullingTime="1.215158587s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:46:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:47:30.126212334 +0000 UTC m=+68.638687530" watchObservedRunningTime="2026-07-01 20:47:30.128681503 +0000 UTC m=+68.641156697"
Jul 01 20:47:34.122370 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:34.122286    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fz5ld" event={"ID":"a02110cb-ee26-4f3e-a854-ececca41783e","Type":"ContainerStarted","Data":"7b5975156eb18dc805fc84be70a8c4883f9ad83a1faf9e2d3c198b226511eced"}
Jul 01 20:47:34.122810 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:34.122446    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:47:34.123683 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:34.123654    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-t6lw6" event={"ID":"1dfa26e7-23bf-4e56-808d-e59cee52aee7","Type":"ContainerStarted","Data":"ac271b2cbd31bde9837daecdee42e0b50aa8942749954cf48f72cd1fab2f706e"}
Jul 01 20:47:34.125639 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:34.125601    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wpqxs" event={"ID":"da6e12d2-79f9-4aef-a545-2329fae14c68","Type":"ContainerStarted","Data":"a7c288924233104ce68aa937d37c9451ba4a506e9bd360752949c2d4500ff05c"}
Jul 01 20:47:34.141123 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:34.141078    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fz5ld" podStartSLOduration=66.364232027 podStartE2EDuration="1m12.141067051s" totalImagesPullingTime="5.776835024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:46:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:47:34.139918442 +0000 UTC m=+72.652393632" watchObservedRunningTime="2026-07-01 20:47:34.141067051 +0000 UTC m=+72.653542241"
Jul 01 20:47:34.161309 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:34.161269    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-wpqxs" podStartSLOduration=1.344467995 podStartE2EDuration="13.161247662s" totalImagesPullingTime="11.816779667s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 20:47:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:47:34.160792356 +0000 UTC m=+72.673267546" watchObservedRunningTime="2026-07-01 20:47:34.161247662 +0000 UTC m=+72.673722851"
Jul 01 20:47:34.179003 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:34.178965    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-t6lw6" podStartSLOduration=66.400745081 podStartE2EDuration="1m12.178953605s" totalImagesPullingTime="5.778208524s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 20:46:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:47:34.177719198 +0000 UTC m=+72.690194388" watchObservedRunningTime="2026-07-01 20:47:34.178953605 +0000 UTC m=+72.691428795"
Jul 01 20:47:36.945014 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:36.944978    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-f7ddq"]
Jul 01 20:47:36.948295 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:36.948271    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-f7ddq"
Jul 01 20:47:36.951065 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:36.951045    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 01 20:47:36.951183 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:36.951049    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dh9nf\""
Jul 01 20:47:36.952566 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:36.952545    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 01 20:47:36.952618 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:36.952545    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 01 20:47:36.952678 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:36.952665    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 01 20:47:36.952801 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:36.952784    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 01 20:47:36.952840 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:36.952801    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 01 20:47:37.029012 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.028989    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/639b6494-9238-4659-a094-d1fe4b13021f-metrics-client-ca\") pod \"node-exporter-f7ddq\" (UID: \"639b6494-9238-4659-a094-d1fe4b13021f\") " pod="openshift-monitoring/node-exporter-f7ddq"
Jul 01 20:47:37.029114 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.029016    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vr7zx\" (UniqueName: \"kubernetes.io/projected/639b6494-9238-4659-a094-d1fe4b13021f-kube-api-access-vr7zx\") pod \"node-exporter-f7ddq\" (UID: \"639b6494-9238-4659-a094-d1fe4b13021f\") " pod="openshift-monitoring/node-exporter-f7ddq"
Jul 01 20:47:37.029114 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.029036    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/639b6494-9238-4659-a094-d1fe4b13021f-node-exporter-textfile\") pod \"node-exporter-f7ddq\" (UID: \"639b6494-9238-4659-a094-d1fe4b13021f\") " pod="openshift-monitoring/node-exporter-f7ddq"
Jul 01 20:47:37.029114 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.029054    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/639b6494-9238-4659-a094-d1fe4b13021f-node-exporter-accelerators-collector-config\") pod \"node-exporter-f7ddq\" (UID: \"639b6494-9238-4659-a094-d1fe4b13021f\") " pod="openshift-monitoring/node-exporter-f7ddq"
Jul 01 20:47:37.029114 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.029080    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/639b6494-9238-4659-a094-d1fe4b13021f-root\") pod \"node-exporter-f7ddq\" (UID: \"639b6494-9238-4659-a094-d1fe4b13021f\") " pod="openshift-monitoring/node-exporter-f7ddq"
Jul 01 20:47:37.029258 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.029134    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/639b6494-9238-4659-a094-d1fe4b13021f-node-exporter-wtmp\") pod \"node-exporter-f7ddq\" (UID: \"639b6494-9238-4659-a094-d1fe4b13021f\") " pod="openshift-monitoring/node-exporter-f7ddq"
Jul 01 20:47:37.029258 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.029193    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/639b6494-9238-4659-a094-d1fe4b13021f-sys\") pod \"node-exporter-f7ddq\" (UID: \"639b6494-9238-4659-a094-d1fe4b13021f\") " pod="openshift-monitoring/node-exporter-f7ddq"
Jul 01 20:47:37.029258 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.029224    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/639b6494-9238-4659-a094-d1fe4b13021f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f7ddq\" (UID: \"639b6494-9238-4659-a094-d1fe4b13021f\") " pod="openshift-monitoring/node-exporter-f7ddq"
Jul 01 20:47:37.029373 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.029270    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/639b6494-9238-4659-a094-d1fe4b13021f-node-exporter-tls\") pod \"node-exporter-f7ddq\" (UID: \"639b6494-9238-4659-a094-d1fe4b13021f\") " pod="openshift-monitoring/node-exporter-f7ddq"
Jul 01 20:47:37.129763 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.129735    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/639b6494-9238-4659-a094-d1fe4b13021f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f7ddq\" (UID: \"639b6494-9238-4659-a094-d1fe4b13021f\") " pod="openshift-monitoring/node-exporter-f7ddq"
Jul 01 20:47:37.129848 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.129770    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/639b6494-9238-4659-a094-d1fe4b13021f-node-exporter-tls\") pod \"node-exporter-f7ddq\" (UID: \"639b6494-9238-4659-a094-d1fe4b13021f\") " pod="openshift-monitoring/node-exporter-f7ddq"
Jul 01 20:47:37.129848 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.129788    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/639b6494-9238-4659-a094-d1fe4b13021f-metrics-client-ca\") pod \"node-exporter-f7ddq\" (UID: \"639b6494-9238-4659-a094-d1fe4b13021f\") " pod="openshift-monitoring/node-exporter-f7ddq"
Jul 01 20:47:37.129848 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.129802    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vr7zx\" (UniqueName: \"kubernetes.io/projected/639b6494-9238-4659-a094-d1fe4b13021f-kube-api-access-vr7zx\") pod \"node-exporter-f7ddq\" (UID: \"639b6494-9238-4659-a094-d1fe4b13021f\") " pod="openshift-monitoring/node-exporter-f7ddq"
Jul 01 20:47:37.129848 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.129820    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/639b6494-9238-4659-a094-d1fe4b13021f-node-exporter-textfile\") pod \"node-exporter-f7ddq\" (UID: \"639b6494-9238-4659-a094-d1fe4b13021f\") " pod="openshift-monitoring/node-exporter-f7ddq"
Jul 01 20:47:37.129848 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.129836    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/639b6494-9238-4659-a094-d1fe4b13021f-node-exporter-accelerators-collector-config\") pod \"node-exporter-f7ddq\" (UID: \"639b6494-9238-4659-a094-d1fe4b13021f\") " pod="openshift-monitoring/node-exporter-f7ddq"
Jul 01 20:47:37.130073 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.130008    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/639b6494-9238-4659-a094-d1fe4b13021f-root\") pod \"node-exporter-f7ddq\" (UID: \"639b6494-9238-4659-a094-d1fe4b13021f\") " pod="openshift-monitoring/node-exporter-f7ddq"
Jul 01 20:47:37.130129 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.130088    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/639b6494-9238-4659-a094-d1fe4b13021f-node-exporter-wtmp\") pod \"node-exporter-f7ddq\" (UID: \"639b6494-9238-4659-a094-d1fe4b13021f\") " pod="openshift-monitoring/node-exporter-f7ddq"
Jul 01 20:47:37.130185 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.130131    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/639b6494-9238-4659-a094-d1fe4b13021f-root\") pod \"node-exporter-f7ddq\" (UID: \"639b6494-9238-4659-a094-d1fe4b13021f\") " pod="openshift-monitoring/node-exporter-f7ddq"
Jul 01 20:47:37.130185 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.130134    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/639b6494-9238-4659-a094-d1fe4b13021f-sys\") pod \"node-exporter-f7ddq\" (UID: \"639b6494-9238-4659-a094-d1fe4b13021f\") " pod="openshift-monitoring/node-exporter-f7ddq"
Jul 01 20:47:37.130272 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.130181    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/639b6494-9238-4659-a094-d1fe4b13021f-sys\") pod \"node-exporter-f7ddq\" (UID: \"639b6494-9238-4659-a094-d1fe4b13021f\") " pod="openshift-monitoring/node-exporter-f7ddq"
Jul 01 20:47:37.130327 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.130292    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/639b6494-9238-4659-a094-d1fe4b13021f-node-exporter-wtmp\") pod \"node-exporter-f7ddq\" (UID: \"639b6494-9238-4659-a094-d1fe4b13021f\") " pod="openshift-monitoring/node-exporter-f7ddq"
Jul 01 20:47:37.130376 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.130342    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/639b6494-9238-4659-a094-d1fe4b13021f-node-exporter-textfile\") pod \"node-exporter-f7ddq\" (UID: \"639b6494-9238-4659-a094-d1fe4b13021f\") " pod="openshift-monitoring/node-exporter-f7ddq"
Jul 01 20:47:37.130569 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.130504    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/639b6494-9238-4659-a094-d1fe4b13021f-metrics-client-ca\") pod \"node-exporter-f7ddq\" (UID: \"639b6494-9238-4659-a094-d1fe4b13021f\") " pod="openshift-monitoring/node-exporter-f7ddq"
Jul 01 20:47:37.130569 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.130580    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/639b6494-9238-4659-a094-d1fe4b13021f-node-exporter-accelerators-collector-config\") pod \"node-exporter-f7ddq\" (UID: \"639b6494-9238-4659-a094-d1fe4b13021f\") " pod="openshift-monitoring/node-exporter-f7ddq"
Jul 01 20:47:37.132331 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.132305    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/639b6494-9238-4659-a094-d1fe4b13021f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f7ddq\" (UID: \"639b6494-9238-4659-a094-d1fe4b13021f\") " pod="openshift-monitoring/node-exporter-f7ddq"
Jul 01 20:47:37.132407 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.132364    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/639b6494-9238-4659-a094-d1fe4b13021f-node-exporter-tls\") pod \"node-exporter-f7ddq\" (UID: \"639b6494-9238-4659-a094-d1fe4b13021f\") " pod="openshift-monitoring/node-exporter-f7ddq"
Jul 01 20:47:37.139392 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.139367    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vr7zx\" (UniqueName: \"kubernetes.io/projected/639b6494-9238-4659-a094-d1fe4b13021f-kube-api-access-vr7zx\") pod \"node-exporter-f7ddq\" (UID: \"639b6494-9238-4659-a094-d1fe4b13021f\") " pod="openshift-monitoring/node-exporter-f7ddq"
Jul 01 20:47:37.257434 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.257381    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-f7ddq"
Jul 01 20:47:37.268781 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:47:37.268758    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod639b6494_9238_4659_a094_d1fe4b13021f.slice/crio-714936dca854071501e687c4e9ba7b3c61498a5198b28fb38829f3a0039cac9e WatchSource:0}: Error finding container 714936dca854071501e687c4e9ba7b3c61498a5198b28fb38829f3a0039cac9e: Status 404 returned error can't find the container with id 714936dca854071501e687c4e9ba7b3c61498a5198b28fb38829f3a0039cac9e
Jul 01 20:47:37.989959 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.989927    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 20:47:37.993513 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.993488    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:37.998827 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.998644    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-rv6ch\""
Jul 01 20:47:37.998827 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.998681    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 01 20:47:37.998827 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.998789    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 01 20:47:37.999058 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.998814    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 01 20:47:37.999058 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.998936    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 01 20:47:37.999058 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.998955    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 01 20:47:37.999058 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.998981    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 01 20:47:37.999238 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.999119    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 01 20:47:37.999385 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:37.999370    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 01 20:47:38.002613 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.002594    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 01 20:47:38.022576 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.022555    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 20:47:38.036010 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.035992    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.036113 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.036021    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.036113 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.036041    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-web-config\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.036113 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.036067    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnvrj\" (UniqueName: \"kubernetes.io/projected/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-kube-api-access-xnvrj\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.036113 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.036104    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.036291 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.036128    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.036291 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.036146    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.036291 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.036218    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-config-volume\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.036291 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.036266    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.036439 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.036321    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-config-out\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.036439 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.036344    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.036439 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.036368    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.036439 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.036389    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.103082 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.103058    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6xhrz"
Jul 01 20:47:38.136980 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.136928    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.136980 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.136975    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.137216 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.136998    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.137216 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.137022    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.137216 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.137042    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-web-config\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.137216 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.137069    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnvrj\" (UniqueName: \"kubernetes.io/projected/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-kube-api-access-xnvrj\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.137216 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.137102    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.137216 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.137130    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.137216 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.137158    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.137216 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.137233    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-config-volume\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.137661 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.137257    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.137661 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.137293    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-config-out\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.137661 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.137350    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.138173 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:47:38.137962    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-alertmanager-trusted-ca-bundle podName:8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd nodeName:}" failed. No retries permitted until 2026-07-01 20:47:38.637936638 +0000 UTC m=+77.150411823 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd") : configmap references non-existent config key: ca-bundle.crt
Jul 01 20:47:38.138311 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.138204    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.138384 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:47:38.138312    2571 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 01 20:47:38.138384 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:47:38.138360    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-secret-alertmanager-main-tls podName:8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd nodeName:}" failed. No retries permitted until 2026-07-01 20:47:38.638343166 +0000 UTC m=+77.150818336 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd") : secret "alertmanager-main-tls" not found
Jul 01 20:47:38.138611 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.138530    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.141437 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.140929    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-config-out\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.141437 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.141030    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.141437 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.141194    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-config-volume\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.141437 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.141392    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.141732 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.141492    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-web-config\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.141941 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.141919    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.142505 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.142489    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.142749 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.142730    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.143231 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.143212    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f7ddq" event={"ID":"639b6494-9238-4659-a094-d1fe4b13021f","Type":"ContainerStarted","Data":"714936dca854071501e687c4e9ba7b3c61498a5198b28fb38829f3a0039cac9e"}
Jul 01 20:47:38.157699 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.157674    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnvrj\" (UniqueName: \"kubernetes.io/projected/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-kube-api-access-xnvrj\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.640879 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.640804    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.640879 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.640874    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.641565 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.641541    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.643194 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.643170    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:38.905036 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:38.905011    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:47:39.048900 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:39.048875    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 20:47:39.049472 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:47:39.049449    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ff8e7a4_df4a_42d9_82f6_8a3ad4d3a8cd.slice/crio-26c682b00368c080b29d9ed449c3c10d5a0c94feddbfb37c7c01170ddf34be8e WatchSource:0}: Error finding container 26c682b00368c080b29d9ed449c3c10d5a0c94feddbfb37c7c01170ddf34be8e: Status 404 returned error can't find the container with id 26c682b00368c080b29d9ed449c3c10d5a0c94feddbfb37c7c01170ddf34be8e
Jul 01 20:47:39.147897 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:39.147868    2571 generic.go:356] "Generic (PLEG): container finished" podID="639b6494-9238-4659-a094-d1fe4b13021f" containerID="399bb5b6bd9ad5b9feb4e5ee15f9332119ccaffe533d612337eff2ed112081c2" exitCode=0
Jul 01 20:47:39.148011 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:39.147959    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f7ddq" event={"ID":"639b6494-9238-4659-a094-d1fe4b13021f","Type":"ContainerDied","Data":"399bb5b6bd9ad5b9feb4e5ee15f9332119ccaffe533d612337eff2ed112081c2"}
Jul 01 20:47:39.149276 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:39.149256    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd","Type":"ContainerStarted","Data":"26c682b00368c080b29d9ed449c3c10d5a0c94feddbfb37c7c01170ddf34be8e"}
Jul 01 20:47:40.155700 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:40.155655    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f7ddq" event={"ID":"639b6494-9238-4659-a094-d1fe4b13021f","Type":"ContainerStarted","Data":"11435d03c0bc463a231a4e03506720f05dcfa80a24c688b36e2648187e53a22e"}
Jul 01 20:47:40.156155 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:40.155727    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f7ddq" event={"ID":"639b6494-9238-4659-a094-d1fe4b13021f","Type":"ContainerStarted","Data":"1b8fab5b93e407427fcc490366cb3c3c94262643fe2f88f7bac9d6c4fd04f369"}
Jul 01 20:47:40.178416 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:40.178361    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-f7ddq" podStartSLOduration=3.090580926 podStartE2EDuration="4.178344027s" totalImagesPullingTime="1.019391111s" totalInitContainerRuntime="68.37199ms" isStatefulPod=true podCreationTimestamp="2026-07-01 20:47:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:47:40.176901546 +0000 UTC m=+78.689376738" watchObservedRunningTime="2026-07-01 20:47:40.178344027 +0000 UTC m=+78.690819216"
Jul 01 20:47:41.161248 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:41.161220    2571 generic.go:356] "Generic (PLEG): container finished" podID="8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" containerID="6511a98b388249ad8b638d4071824dd3ba510ee6620f1976965ae26cef709768" exitCode=0
Jul 01 20:47:41.161556 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:41.161294    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd","Type":"ContainerDied","Data":"6511a98b388249ad8b638d4071824dd3ba510ee6620f1976965ae26cef709768"}
Jul 01 20:47:43.075468 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.075446    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-54dbbfbdcd-xq7ll"
Jul 01 20:47:43.171308 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.171282    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd","Type":"ContainerStarted","Data":"028319b7b8bf5e73c45179050fd55b829b21a08122a8189f113956a7462f05b4"}
Jul 01 20:47:43.171400 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.171343    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd","Type":"ContainerStarted","Data":"04f430778bb64f54c84a0b9855cd01b32e2a2592619759b13d105b7939df3bf4"}
Jul 01 20:47:43.171400 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.171378    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd","Type":"ContainerStarted","Data":"bcaa2f6bbfc7dd7f2bed82d88710c58bc56c0bd7e9ee4e5e608737f392ed0666"}
Jul 01 20:47:43.175526 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.175502    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 01 20:47:43.178781 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.178764    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.181580 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.181562    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 01 20:47:43.181842 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.181706    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 01 20:47:43.181842 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.181762    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 01 20:47:43.181842 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.181787    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 01 20:47:43.181842 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.181825    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 01 20:47:43.182045 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.181939    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 01 20:47:43.182116 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.182097    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 01 20:47:43.182258 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.182242    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 01 20:47:43.183203 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.183171    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 01 20:47:43.183203 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.183173    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-39a23vgovq5u0\""
Jul 01 20:47:43.183386 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.183275    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 01 20:47:43.183386 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.183175    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 01 20:47:43.184451 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.184295    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 01 20:47:43.184451 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.184300    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-v79nx\""
Jul 01 20:47:43.184564 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.184546    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 01 20:47:43.186621 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.186561    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 01 20:47:43.189239 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.189220    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 01 20:47:43.213899 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.213875    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 01 20:47:43.274987 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.274964    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.275101 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.274997    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.275101 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.275014    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.275101 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.275036    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.275101 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.275080    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.275228 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.275110    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b9fcac3d-2ec9-4947-ba90-5caabfa17576-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.275228 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.275131    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.275228 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.275153    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.275228 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.275170    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-config\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.275228 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.275198    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b9fcac3d-2ec9-4947-ba90-5caabfa17576-config-out\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.275228 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.275223    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.275389 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.275261    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-web-config\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.275389 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.275276    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7brxr\" (UniqueName: \"kubernetes.io/projected/b9fcac3d-2ec9-4947-ba90-5caabfa17576-kube-api-access-7brxr\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.275389 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.275314    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.275389 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.275334    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.275389 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.275352    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.275389 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.275374    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.275603 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.275389    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.275603 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.275411    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.275603 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.275460    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b9fcac3d-2ec9-4947-ba90-5caabfa17576-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.376522 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.376482    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.376654 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.376542    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b9fcac3d-2ec9-4947-ba90-5caabfa17576-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.376654 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.376579    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.376654 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.376609    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.376654 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.376650    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.376863 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.376676    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.377264 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.376700    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.377264 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.376988    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b9fcac3d-2ec9-4947-ba90-5caabfa17576-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.377264 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.377210    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b9fcac3d-2ec9-4947-ba90-5caabfa17576-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.377457 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.377390    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.377619 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.377548    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.377789 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.377239    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.377904 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.377886    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.378014 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.378000    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-config\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.378133 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.378119    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b9fcac3d-2ec9-4947-ba90-5caabfa17576-config-out\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.378224 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.378210    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.378335 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.378322    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-web-config\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.378428 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.378415    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7brxr\" (UniqueName: \"kubernetes.io/projected/b9fcac3d-2ec9-4947-ba90-5caabfa17576-kube-api-access-7brxr\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.378531 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.378517    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.378653 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.378613    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.378769 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.378754    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.378863 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.378849    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.378953 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.378939    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.379597 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.379576    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.379728 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.379646    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.379813 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.379786    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.381153 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.380431    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.381153 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.380521    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.381153 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.380846    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.381798 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.381448    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.382133 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.382052    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.382133 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.378848    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.382317 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.382177    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b9fcac3d-2ec9-4947-ba90-5caabfa17576-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.382612 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.382591    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b9fcac3d-2ec9-4947-ba90-5caabfa17576-config-out\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.382832 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.382813    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-web-config\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.382986 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.382964    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.383356 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.383333    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.383461 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.383447    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-config\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.384487 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.384468    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.392506 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.392485    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7brxr\" (UniqueName: \"kubernetes.io/projected/b9fcac3d-2ec9-4947-ba90-5caabfa17576-kube-api-access-7brxr\") pod \"prometheus-k8s-0\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.489852 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.489793    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:47:43.626955 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:43.626891    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 01 20:47:43.629326 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:47:43.629295    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9fcac3d_2ec9_4947_ba90_5caabfa17576.slice/crio-d80ec6190acfff37ed5ba75e0d5e0443d5087cc2ad5c04bda5be0dea7619a7f1 WatchSource:0}: Error finding container d80ec6190acfff37ed5ba75e0d5e0443d5087cc2ad5c04bda5be0dea7619a7f1: Status 404 returned error can't find the container with id d80ec6190acfff37ed5ba75e0d5e0443d5087cc2ad5c04bda5be0dea7619a7f1
Jul 01 20:47:44.177599 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:44.177564    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd","Type":"ContainerStarted","Data":"733a4df0fe41171b3b2caea41c10a980d3d9dbf7cf7a688ad639e20208a26f61"}
Jul 01 20:47:44.177975 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:44.177665    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd","Type":"ContainerStarted","Data":"438557ee09d4681b8678525a369b2f3453b931806fe9d9f26b2d018cbc1e8dd4"}
Jul 01 20:47:44.178969 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:44.178946    2571 generic.go:356] "Generic (PLEG): container finished" podID="b9fcac3d-2ec9-4947-ba90-5caabfa17576" containerID="8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03" exitCode=0
Jul 01 20:47:44.179079 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:44.178986    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b9fcac3d-2ec9-4947-ba90-5caabfa17576","Type":"ContainerDied","Data":"8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03"}
Jul 01 20:47:44.179079 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:44.179029    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b9fcac3d-2ec9-4947-ba90-5caabfa17576","Type":"ContainerStarted","Data":"d80ec6190acfff37ed5ba75e0d5e0443d5087cc2ad5c04bda5be0dea7619a7f1"}
Jul 01 20:47:45.188418 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:45.188380    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd","Type":"ContainerStarted","Data":"18a60199ef951266ec177e956c856be93984274058f12259b4d55b8ed8145b5c"}
Jul 01 20:47:45.218594 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:45.218547    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.4599476510000002 podStartE2EDuration="8.218532946s" totalImagesPullingTime="4.742515557s" totalInitContainerRuntime="16.069738ms" isStatefulPod=false podCreationTimestamp="2026-07-01 20:47:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:47:45.216851648 +0000 UTC m=+83.729326865" watchObservedRunningTime="2026-07-01 20:47:45.218532946 +0000 UTC m=+83.731008136"
Jul 01 20:47:48.202760 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:48.202715    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b9fcac3d-2ec9-4947-ba90-5caabfa17576","Type":"ContainerStarted","Data":"23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370"}
Jul 01 20:47:48.203119 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:48.202796    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b9fcac3d-2ec9-4947-ba90-5caabfa17576","Type":"ContainerStarted","Data":"f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce"}
Jul 01 20:47:50.213330 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:50.213295    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b9fcac3d-2ec9-4947-ba90-5caabfa17576","Type":"ContainerStarted","Data":"e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11"}
Jul 01 20:47:50.213725 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:50.213358    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b9fcac3d-2ec9-4947-ba90-5caabfa17576","Type":"ContainerStarted","Data":"ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2"}
Jul 01 20:47:50.213725 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:50.213396    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b9fcac3d-2ec9-4947-ba90-5caabfa17576","Type":"ContainerStarted","Data":"dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378"}
Jul 01 20:47:50.213725 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:50.213424    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b9fcac3d-2ec9-4947-ba90-5caabfa17576","Type":"ContainerStarted","Data":"ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6"}
Jul 01 20:47:50.243932 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:50.243877    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.834158615 podStartE2EDuration="7.243858168s" totalImagesPullingTime="5.388220935s" totalInitContainerRuntime="21.478618ms" isStatefulPod=false podCreationTimestamp="2026-07-01 20:47:43 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:47:50.242263913 +0000 UTC m=+88.754739128" watchObservedRunningTime="2026-07-01 20:47:50.243858168 +0000 UTC m=+88.756333361"
Jul 01 20:47:53.490100 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:47:53.490064    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:48:05.132290 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:05.132255    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fz5ld"
Jul 01 20:48:43.490842 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:43.490800    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:48:43.511096 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:43.511068    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:48:44.422759 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:44.422737    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:48:57.153803 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:57.153753    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 20:48:57.154224 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:57.154155    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" containerName="alertmanager" containerID="cri-o://bcaa2f6bbfc7dd7f2bed82d88710c58bc56c0bd7e9ee4e5e608737f392ed0666" gracePeriod=120
Jul 01 20:48:57.154224 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:57.154212    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" containerName="kube-rbac-proxy-web" containerID="cri-o://028319b7b8bf5e73c45179050fd55b829b21a08122a8189f113956a7462f05b4" gracePeriod=120
Jul 01 20:48:57.154341 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:57.154244    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" containerName="prom-label-proxy" containerID="cri-o://18a60199ef951266ec177e956c856be93984274058f12259b4d55b8ed8145b5c" gracePeriod=120
Jul 01 20:48:57.154393 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:57.154315    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" containerName="kube-rbac-proxy" containerID="cri-o://438557ee09d4681b8678525a369b2f3453b931806fe9d9f26b2d018cbc1e8dd4" gracePeriod=120
Jul 01 20:48:57.154445 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:57.154265    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" containerName="config-reloader" containerID="cri-o://04f430778bb64f54c84a0b9855cd01b32e2a2592619759b13d105b7939df3bf4" gracePeriod=120
Jul 01 20:48:57.154445 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:57.154180    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" containerName="kube-rbac-proxy-metric" containerID="cri-o://733a4df0fe41171b3b2caea41c10a980d3d9dbf7cf7a688ad639e20208a26f61" gracePeriod=120
Jul 01 20:48:57.456718 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:57.456662    2571 generic.go:356] "Generic (PLEG): container finished" podID="8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" containerID="18a60199ef951266ec177e956c856be93984274058f12259b4d55b8ed8145b5c" exitCode=0
Jul 01 20:48:57.456718 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:57.456685    2571 generic.go:356] "Generic (PLEG): container finished" podID="8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" containerID="438557ee09d4681b8678525a369b2f3453b931806fe9d9f26b2d018cbc1e8dd4" exitCode=0
Jul 01 20:48:57.456718 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:57.456692    2571 generic.go:356] "Generic (PLEG): container finished" podID="8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" containerID="04f430778bb64f54c84a0b9855cd01b32e2a2592619759b13d105b7939df3bf4" exitCode=0
Jul 01 20:48:57.456718 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:57.456697    2571 generic.go:356] "Generic (PLEG): container finished" podID="8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" containerID="bcaa2f6bbfc7dd7f2bed82d88710c58bc56c0bd7e9ee4e5e608737f392ed0666" exitCode=0
Jul 01 20:48:57.456890 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:57.456729    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd","Type":"ContainerDied","Data":"18a60199ef951266ec177e956c856be93984274058f12259b4d55b8ed8145b5c"}
Jul 01 20:48:57.456890 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:57.456792    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd","Type":"ContainerDied","Data":"438557ee09d4681b8678525a369b2f3453b931806fe9d9f26b2d018cbc1e8dd4"}
Jul 01 20:48:57.456890 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:57.456817    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd","Type":"ContainerDied","Data":"04f430778bb64f54c84a0b9855cd01b32e2a2592619759b13d105b7939df3bf4"}
Jul 01 20:48:57.456890 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:57.456842    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd","Type":"ContainerDied","Data":"bcaa2f6bbfc7dd7f2bed82d88710c58bc56c0bd7e9ee4e5e608737f392ed0666"}
Jul 01 20:48:58.409179 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.409160    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:58.464050 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.464023    2571 generic.go:356] "Generic (PLEG): container finished" podID="8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" containerID="733a4df0fe41171b3b2caea41c10a980d3d9dbf7cf7a688ad639e20208a26f61" exitCode=0
Jul 01 20:48:58.464050 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.464045    2571 generic.go:356] "Generic (PLEG): container finished" podID="8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" containerID="028319b7b8bf5e73c45179050fd55b829b21a08122a8189f113956a7462f05b4" exitCode=0
Jul 01 20:48:58.464228 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.464101    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd","Type":"ContainerDied","Data":"733a4df0fe41171b3b2caea41c10a980d3d9dbf7cf7a688ad639e20208a26f61"}
Jul 01 20:48:58.464228 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.464138    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:58.464228 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.464168    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd","Type":"ContainerDied","Data":"028319b7b8bf5e73c45179050fd55b829b21a08122a8189f113956a7462f05b4"}
Jul 01 20:48:58.464228 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.464197    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd","Type":"ContainerDied","Data":"26c682b00368c080b29d9ed449c3c10d5a0c94feddbfb37c7c01170ddf34be8e"}
Jul 01 20:48:58.464348 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.464247    2571 scope.go:117] "RemoveContainer" containerID="18a60199ef951266ec177e956c856be93984274058f12259b4d55b8ed8145b5c"
Jul 01 20:48:58.471593 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.471573    2571 scope.go:117] "RemoveContainer" containerID="733a4df0fe41171b3b2caea41c10a980d3d9dbf7cf7a688ad639e20208a26f61"
Jul 01 20:48:58.478234 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.478218    2571 scope.go:117] "RemoveContainer" containerID="438557ee09d4681b8678525a369b2f3453b931806fe9d9f26b2d018cbc1e8dd4"
Jul 01 20:48:58.484978 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.484963    2571 scope.go:117] "RemoveContainer" containerID="028319b7b8bf5e73c45179050fd55b829b21a08122a8189f113956a7462f05b4"
Jul 01 20:48:58.491719 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.491702    2571 scope.go:117] "RemoveContainer" containerID="04f430778bb64f54c84a0b9855cd01b32e2a2592619759b13d105b7939df3bf4"
Jul 01 20:48:58.498353 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.498330    2571 scope.go:117] "RemoveContainer" containerID="bcaa2f6bbfc7dd7f2bed82d88710c58bc56c0bd7e9ee4e5e608737f392ed0666"
Jul 01 20:48:58.505275 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.505261    2571 scope.go:117] "RemoveContainer" containerID="6511a98b388249ad8b638d4071824dd3ba510ee6620f1976965ae26cef709768"
Jul 01 20:48:58.511677 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.511640    2571 scope.go:117] "RemoveContainer" containerID="18a60199ef951266ec177e956c856be93984274058f12259b4d55b8ed8145b5c"
Jul 01 20:48:58.511885 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:48:58.511865    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18a60199ef951266ec177e956c856be93984274058f12259b4d55b8ed8145b5c\": container with ID starting with 18a60199ef951266ec177e956c856be93984274058f12259b4d55b8ed8145b5c not found: ID does not exist" containerID="18a60199ef951266ec177e956c856be93984274058f12259b4d55b8ed8145b5c"
Jul 01 20:48:58.511939 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.511893    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18a60199ef951266ec177e956c856be93984274058f12259b4d55b8ed8145b5c"} err="failed to get container status \"18a60199ef951266ec177e956c856be93984274058f12259b4d55b8ed8145b5c\": rpc error: code = NotFound desc = could not find container \"18a60199ef951266ec177e956c856be93984274058f12259b4d55b8ed8145b5c\": container with ID starting with 18a60199ef951266ec177e956c856be93984274058f12259b4d55b8ed8145b5c not found: ID does not exist"
Jul 01 20:48:58.511939 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.511925    2571 scope.go:117] "RemoveContainer" containerID="733a4df0fe41171b3b2caea41c10a980d3d9dbf7cf7a688ad639e20208a26f61"
Jul 01 20:48:58.512179 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:48:58.512163    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"733a4df0fe41171b3b2caea41c10a980d3d9dbf7cf7a688ad639e20208a26f61\": container with ID starting with 733a4df0fe41171b3b2caea41c10a980d3d9dbf7cf7a688ad639e20208a26f61 not found: ID does not exist" containerID="733a4df0fe41171b3b2caea41c10a980d3d9dbf7cf7a688ad639e20208a26f61"
Jul 01 20:48:58.512219 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.512185    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"733a4df0fe41171b3b2caea41c10a980d3d9dbf7cf7a688ad639e20208a26f61"} err="failed to get container status \"733a4df0fe41171b3b2caea41c10a980d3d9dbf7cf7a688ad639e20208a26f61\": rpc error: code = NotFound desc = could not find container \"733a4df0fe41171b3b2caea41c10a980d3d9dbf7cf7a688ad639e20208a26f61\": container with ID starting with 733a4df0fe41171b3b2caea41c10a980d3d9dbf7cf7a688ad639e20208a26f61 not found: ID does not exist"
Jul 01 20:48:58.512219 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.512202    2571 scope.go:117] "RemoveContainer" containerID="438557ee09d4681b8678525a369b2f3453b931806fe9d9f26b2d018cbc1e8dd4"
Jul 01 20:48:58.512282 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.512269    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-secret-alertmanager-kube-rbac-proxy-web\") pod \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") "
Jul 01 20:48:58.512331 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.512297    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-tls-assets\") pod \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") "
Jul 01 20:48:58.512331 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.512323    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-secret-alertmanager-main-tls\") pod \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") "
Jul 01 20:48:58.512423 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.512361    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-secret-alertmanager-kube-rbac-proxy\") pod \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") "
Jul 01 20:48:58.512423 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.512387    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-config-volume\") pod \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") "
Jul 01 20:48:58.512423 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.512417    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-alertmanager-main-db\") pod \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") "
Jul 01 20:48:58.512573 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.512441    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-cluster-tls-config\") pod \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") "
Jul 01 20:48:58.512573 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.512480    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") "
Jul 01 20:48:58.512573 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:48:58.512485    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"438557ee09d4681b8678525a369b2f3453b931806fe9d9f26b2d018cbc1e8dd4\": container with ID starting with 438557ee09d4681b8678525a369b2f3453b931806fe9d9f26b2d018cbc1e8dd4 not found: ID does not exist" containerID="438557ee09d4681b8678525a369b2f3453b931806fe9d9f26b2d018cbc1e8dd4"
Jul 01 20:48:58.512573 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.512513    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xnvrj\" (UniqueName: \"kubernetes.io/projected/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-kube-api-access-xnvrj\") pod \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") "
Jul 01 20:48:58.512573 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.512516    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"438557ee09d4681b8678525a369b2f3453b931806fe9d9f26b2d018cbc1e8dd4"} err="failed to get container status \"438557ee09d4681b8678525a369b2f3453b931806fe9d9f26b2d018cbc1e8dd4\": rpc error: code = NotFound desc = could not find container \"438557ee09d4681b8678525a369b2f3453b931806fe9d9f26b2d018cbc1e8dd4\": container with ID starting with 438557ee09d4681b8678525a369b2f3453b931806fe9d9f26b2d018cbc1e8dd4 not found: ID does not exist"
Jul 01 20:48:58.512573 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.512539    2571 scope.go:117] "RemoveContainer" containerID="028319b7b8bf5e73c45179050fd55b829b21a08122a8189f113956a7462f05b4"
Jul 01 20:48:58.512573 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.512545    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-web-config\") pod \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") "
Jul 01 20:48:58.512573 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.512570    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-config-out\") pod \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") "
Jul 01 20:48:58.512976 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.512597    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-metrics-client-ca\") pod \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") "
Jul 01 20:48:58.512976 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.512647    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-alertmanager-trusted-ca-bundle\") pod \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\" (UID: \"8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd\") "
Jul 01 20:48:58.513786 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:48:58.513365    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"028319b7b8bf5e73c45179050fd55b829b21a08122a8189f113956a7462f05b4\": container with ID starting with 028319b7b8bf5e73c45179050fd55b829b21a08122a8189f113956a7462f05b4 not found: ID does not exist" containerID="028319b7b8bf5e73c45179050fd55b829b21a08122a8189f113956a7462f05b4"
Jul 01 20:48:58.513786 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.513398    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"028319b7b8bf5e73c45179050fd55b829b21a08122a8189f113956a7462f05b4"} err="failed to get container status \"028319b7b8bf5e73c45179050fd55b829b21a08122a8189f113956a7462f05b4\": rpc error: code = NotFound desc = could not find container \"028319b7b8bf5e73c45179050fd55b829b21a08122a8189f113956a7462f05b4\": container with ID starting with 028319b7b8bf5e73c45179050fd55b829b21a08122a8189f113956a7462f05b4 not found: ID does not exist"
Jul 01 20:48:58.513786 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.513421    2571 scope.go:117] "RemoveContainer" containerID="04f430778bb64f54c84a0b9855cd01b32e2a2592619759b13d105b7939df3bf4"
Jul 01 20:48:58.513786 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.513474    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" (UID: "8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 20:48:58.514046 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.513843    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" (UID: "8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 20:48:58.514046 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.513878    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" (UID: "8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 20:48:58.514438 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:48:58.514408    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04f430778bb64f54c84a0b9855cd01b32e2a2592619759b13d105b7939df3bf4\": container with ID starting with 04f430778bb64f54c84a0b9855cd01b32e2a2592619759b13d105b7939df3bf4 not found: ID does not exist" containerID="04f430778bb64f54c84a0b9855cd01b32e2a2592619759b13d105b7939df3bf4"
Jul 01 20:48:58.514516 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.514446    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04f430778bb64f54c84a0b9855cd01b32e2a2592619759b13d105b7939df3bf4"} err="failed to get container status \"04f430778bb64f54c84a0b9855cd01b32e2a2592619759b13d105b7939df3bf4\": rpc error: code = NotFound desc = could not find container \"04f430778bb64f54c84a0b9855cd01b32e2a2592619759b13d105b7939df3bf4\": container with ID starting with 04f430778bb64f54c84a0b9855cd01b32e2a2592619759b13d105b7939df3bf4 not found: ID does not exist"
Jul 01 20:48:58.514516 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.514466    2571 scope.go:117] "RemoveContainer" containerID="bcaa2f6bbfc7dd7f2bed82d88710c58bc56c0bd7e9ee4e5e608737f392ed0666"
Jul 01 20:48:58.515036 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:48:58.514832    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcaa2f6bbfc7dd7f2bed82d88710c58bc56c0bd7e9ee4e5e608737f392ed0666\": container with ID starting with bcaa2f6bbfc7dd7f2bed82d88710c58bc56c0bd7e9ee4e5e608737f392ed0666 not found: ID does not exist" containerID="bcaa2f6bbfc7dd7f2bed82d88710c58bc56c0bd7e9ee4e5e608737f392ed0666"
Jul 01 20:48:58.515036 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.514864    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcaa2f6bbfc7dd7f2bed82d88710c58bc56c0bd7e9ee4e5e608737f392ed0666"} err="failed to get container status \"bcaa2f6bbfc7dd7f2bed82d88710c58bc56c0bd7e9ee4e5e608737f392ed0666\": rpc error: code = NotFound desc = could not find container \"bcaa2f6bbfc7dd7f2bed82d88710c58bc56c0bd7e9ee4e5e608737f392ed0666\": container with ID starting with bcaa2f6bbfc7dd7f2bed82d88710c58bc56c0bd7e9ee4e5e608737f392ed0666 not found: ID does not exist"
Jul 01 20:48:58.515036 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.514884    2571 scope.go:117] "RemoveContainer" containerID="6511a98b388249ad8b638d4071824dd3ba510ee6620f1976965ae26cef709768"
Jul 01 20:48:58.515705 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.515575    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" (UID: "8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 20:48:58.515799 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:48:58.515776    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6511a98b388249ad8b638d4071824dd3ba510ee6620f1976965ae26cef709768\": container with ID starting with 6511a98b388249ad8b638d4071824dd3ba510ee6620f1976965ae26cef709768 not found: ID does not exist" containerID="6511a98b388249ad8b638d4071824dd3ba510ee6620f1976965ae26cef709768"
Jul 01 20:48:58.515863 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.515802    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6511a98b388249ad8b638d4071824dd3ba510ee6620f1976965ae26cef709768"} err="failed to get container status \"6511a98b388249ad8b638d4071824dd3ba510ee6620f1976965ae26cef709768\": rpc error: code = NotFound desc = could not find container \"6511a98b388249ad8b638d4071824dd3ba510ee6620f1976965ae26cef709768\": container with ID starting with 6511a98b388249ad8b638d4071824dd3ba510ee6620f1976965ae26cef709768 not found: ID does not exist"
Jul 01 20:48:58.515863 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.515823    2571 scope.go:117] "RemoveContainer" containerID="18a60199ef951266ec177e956c856be93984274058f12259b4d55b8ed8145b5c"
Jul 01 20:48:58.515976 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.515949    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" (UID: "8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 20:48:58.516165 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.516110    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" (UID: "8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 20:48:58.516273 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.516235    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18a60199ef951266ec177e956c856be93984274058f12259b4d55b8ed8145b5c"} err="failed to get container status \"18a60199ef951266ec177e956c856be93984274058f12259b4d55b8ed8145b5c\": rpc error: code = NotFound desc = could not find container \"18a60199ef951266ec177e956c856be93984274058f12259b4d55b8ed8145b5c\": container with ID starting with 18a60199ef951266ec177e956c856be93984274058f12259b4d55b8ed8145b5c not found: ID does not exist"
Jul 01 20:48:58.516273 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.516261    2571 scope.go:117] "RemoveContainer" containerID="733a4df0fe41171b3b2caea41c10a980d3d9dbf7cf7a688ad639e20208a26f61"
Jul 01 20:48:58.516491 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.516447    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" (UID: "8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 20:48:58.516491 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.516470    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" (UID: "8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 20:48:58.516664 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.516488    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-config-out" (OuterVolumeSpecName: "config-out") pod "8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" (UID: "8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 20:48:58.516664 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.516532    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"733a4df0fe41171b3b2caea41c10a980d3d9dbf7cf7a688ad639e20208a26f61"} err="failed to get container status \"733a4df0fe41171b3b2caea41c10a980d3d9dbf7cf7a688ad639e20208a26f61\": rpc error: code = NotFound desc = could not find container \"733a4df0fe41171b3b2caea41c10a980d3d9dbf7cf7a688ad639e20208a26f61\": container with ID starting with 733a4df0fe41171b3b2caea41c10a980d3d9dbf7cf7a688ad639e20208a26f61 not found: ID does not exist"
Jul 01 20:48:58.516664 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.516555    2571 scope.go:117] "RemoveContainer" containerID="438557ee09d4681b8678525a369b2f3453b931806fe9d9f26b2d018cbc1e8dd4"
Jul 01 20:48:58.516840 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.516794    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"438557ee09d4681b8678525a369b2f3453b931806fe9d9f26b2d018cbc1e8dd4"} err="failed to get container status \"438557ee09d4681b8678525a369b2f3453b931806fe9d9f26b2d018cbc1e8dd4\": rpc error: code = NotFound desc = could not find container \"438557ee09d4681b8678525a369b2f3453b931806fe9d9f26b2d018cbc1e8dd4\": container with ID starting with 438557ee09d4681b8678525a369b2f3453b931806fe9d9f26b2d018cbc1e8dd4 not found: ID does not exist"
Jul 01 20:48:58.516840 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.516818    2571 scope.go:117] "RemoveContainer" containerID="028319b7b8bf5e73c45179050fd55b829b21a08122a8189f113956a7462f05b4"
Jul 01 20:48:58.517065 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.517046    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"028319b7b8bf5e73c45179050fd55b829b21a08122a8189f113956a7462f05b4"} err="failed to get container status \"028319b7b8bf5e73c45179050fd55b829b21a08122a8189f113956a7462f05b4\": rpc error: code = NotFound desc = could not find container \"028319b7b8bf5e73c45179050fd55b829b21a08122a8189f113956a7462f05b4\": container with ID starting with 028319b7b8bf5e73c45179050fd55b829b21a08122a8189f113956a7462f05b4 not found: ID does not exist"
Jul 01 20:48:58.517144 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.517066    2571 scope.go:117] "RemoveContainer" containerID="04f430778bb64f54c84a0b9855cd01b32e2a2592619759b13d105b7939df3bf4"
Jul 01 20:48:58.517294 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.517276    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04f430778bb64f54c84a0b9855cd01b32e2a2592619759b13d105b7939df3bf4"} err="failed to get container status \"04f430778bb64f54c84a0b9855cd01b32e2a2592619759b13d105b7939df3bf4\": rpc error: code = NotFound desc = could not find container \"04f430778bb64f54c84a0b9855cd01b32e2a2592619759b13d105b7939df3bf4\": container with ID starting with 04f430778bb64f54c84a0b9855cd01b32e2a2592619759b13d105b7939df3bf4 not found: ID does not exist"
Jul 01 20:48:58.517380 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.517296    2571 scope.go:117] "RemoveContainer" containerID="bcaa2f6bbfc7dd7f2bed82d88710c58bc56c0bd7e9ee4e5e608737f392ed0666"
Jul 01 20:48:58.517552 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.517533    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcaa2f6bbfc7dd7f2bed82d88710c58bc56c0bd7e9ee4e5e608737f392ed0666"} err="failed to get container status \"bcaa2f6bbfc7dd7f2bed82d88710c58bc56c0bd7e9ee4e5e608737f392ed0666\": rpc error: code = NotFound desc = could not find container \"bcaa2f6bbfc7dd7f2bed82d88710c58bc56c0bd7e9ee4e5e608737f392ed0666\": container with ID starting with bcaa2f6bbfc7dd7f2bed82d88710c58bc56c0bd7e9ee4e5e608737f392ed0666 not found: ID does not exist"
Jul 01 20:48:58.517654 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.517553    2571 scope.go:117] "RemoveContainer" containerID="6511a98b388249ad8b638d4071824dd3ba510ee6620f1976965ae26cef709768"
Jul 01 20:48:58.517850 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.517824    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-kube-api-access-xnvrj" (OuterVolumeSpecName: "kube-api-access-xnvrj") pod "8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" (UID: "8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd"). InnerVolumeSpecName "kube-api-access-xnvrj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 20:48:58.517957 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.517871    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6511a98b388249ad8b638d4071824dd3ba510ee6620f1976965ae26cef709768"} err="failed to get container status \"6511a98b388249ad8b638d4071824dd3ba510ee6620f1976965ae26cef709768\": rpc error: code = NotFound desc = could not find container \"6511a98b388249ad8b638d4071824dd3ba510ee6620f1976965ae26cef709768\": container with ID starting with 6511a98b388249ad8b638d4071824dd3ba510ee6620f1976965ae26cef709768 not found: ID does not exist"
Jul 01 20:48:58.518030 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.518013    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-config-volume" (OuterVolumeSpecName: "config-volume") pod "8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" (UID: "8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 20:48:58.521174 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.521151    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" (UID: "8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 20:48:58.526600 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.526578    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-web-config" (OuterVolumeSpecName: "web-config") pod "8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" (UID: "8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 20:48:58.613708 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.613678    2571 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 20:48:58.613708 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.613707    2571 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-config-volume\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 20:48:58.613708 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.613718    2571 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-alertmanager-main-db\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 20:48:58.613858 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.613726    2571 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-cluster-tls-config\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 20:48:58.613858 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.613736    2571 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 20:48:58.613858 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.613744    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnvrj\" (UniqueName: \"kubernetes.io/projected/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-kube-api-access-xnvrj\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 20:48:58.613858 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.613753    2571 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-web-config\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 20:48:58.613858 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.613760    2571 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-config-out\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 20:48:58.613858 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.613768    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-metrics-client-ca\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 20:48:58.613858 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.613775    2571 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 20:48:58.613858 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.613784    2571 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 20:48:58.613858 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.613792    2571 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-tls-assets\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 20:48:58.613858 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.613800    2571 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd-secret-alertmanager-main-tls\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 20:48:58.792355 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.792335    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 20:48:58.796885 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.796865    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 20:48:58.823656 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.823221    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 20:48:58.823767 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.823721    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" containerName="kube-rbac-proxy"
Jul 01 20:48:58.823767 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.823739    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" containerName="kube-rbac-proxy"
Jul 01 20:48:58.823767 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.823762    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" containerName="alertmanager"
Jul 01 20:48:58.823908 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.823771    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" containerName="alertmanager"
Jul 01 20:48:58.823908 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.823782    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" containerName="prom-label-proxy"
Jul 01 20:48:58.823908 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.823790    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" containerName="prom-label-proxy"
Jul 01 20:48:58.823908 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.823809    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" containerName="kube-rbac-proxy-metric"
Jul 01 20:48:58.823908 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.823817    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" containerName="kube-rbac-proxy-metric"
Jul 01 20:48:58.823908 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.823830    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" containerName="config-reloader"
Jul 01 20:48:58.823908 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.823838    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" containerName="config-reloader"
Jul 01 20:48:58.823908 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.823868    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" containerName="kube-rbac-proxy-web"
Jul 01 20:48:58.823908 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.823877    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" containerName="kube-rbac-proxy-web"
Jul 01 20:48:58.823908 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.823887    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" containerName="init-config-reloader"
Jul 01 20:48:58.823908 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.823902    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" containerName="init-config-reloader"
Jul 01 20:48:58.824386 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.823993    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" containerName="alertmanager"
Jul 01 20:48:58.824386 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.824008    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" containerName="config-reloader"
Jul 01 20:48:58.824386 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.824024    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" containerName="kube-rbac-proxy-metric"
Jul 01 20:48:58.824386 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.824033    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" containerName="prom-label-proxy"
Jul 01 20:48:58.824386 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.824043    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" containerName="kube-rbac-proxy-web"
Jul 01 20:48:58.824386 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.824060    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" containerName="kube-rbac-proxy"
Jul 01 20:48:58.829803 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.829785    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:58.832536 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.832514    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 01 20:48:58.832663 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.832581    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 01 20:48:58.832663 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.832592    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-rv6ch\""
Jul 01 20:48:58.832773 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.832727    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 01 20:48:58.832773 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.832730    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 01 20:48:58.832868 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.832859    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 01 20:48:58.832919 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.832897    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 01 20:48:58.832919 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.832906    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 01 20:48:58.833015 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.833000    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 01 20:48:58.837543 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.837523    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 01 20:48:58.845287 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.845269    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 20:48:58.916195 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.916171    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-config-out\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:58.916300 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.916206    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:58.916300 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.916235    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:58.916300 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.916283    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:58.916422 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.916303    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:58.916422 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.916331    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:58.916422 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.916352    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:58.916422 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.916367    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-config-volume\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:58.916422 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.916398    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgb45\" (UniqueName: \"kubernetes.io/projected/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-kube-api-access-qgb45\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:58.916602 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.916438    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:58.916602 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.916465    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:58.916602 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.916482    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-web-config\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:58.916602 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:58.916496    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:59.017341 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:59.017313    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:59.017341 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:59.017343    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-web-config\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:59.017525 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:59.017360    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:59.017525 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:59.017379    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-config-out\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:59.017525 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:59.017398    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:59.017525 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:59.017419    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:59.017525 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:59.017454    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:59.017525 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:59.017478    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:59.017525 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:59.017501    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:59.017525 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:59.017524    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:59.017901 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:59.017547    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-config-volume\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:59.017901 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:59.017580    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgb45\" (UniqueName: \"kubernetes.io/projected/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-kube-api-access-qgb45\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:59.017901 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:59.017652    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:59.018812 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:59.018786    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:59.019164 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:59.019145    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:59.020468 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:59.020443    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-config-out\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:59.020765 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:59.020741    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:59.020848 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:59.020772    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:59.020848 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:59.020781    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:59.020930 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:59.020900    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:59.021174 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:59.021145    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:59.021312 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:59.021296    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:59.021380 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:59.021363    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-web-config\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:59.021900 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:59.021856    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:59.022568 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:59.022552    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-config-volume\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:59.030320 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:59.030300    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgb45\" (UniqueName: \"kubernetes.io/projected/eb5351f3-0f90-4da6-846a-11ff1ba5f6bb-kube-api-access-qgb45\") pod \"alertmanager-main-0\" (UID: \"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:59.139157 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:59.139107    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 01 20:48:59.272358 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:59.272300    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 20:48:59.275230 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:48:59.275206    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb5351f3_0f90_4da6_846a_11ff1ba5f6bb.slice/crio-67e7d675bf446a69b3a6422d16a0963b2403b460e0dfd9cf474b433cda589acf WatchSource:0}: Error finding container 67e7d675bf446a69b3a6422d16a0963b2403b460e0dfd9cf474b433cda589acf: Status 404 returned error can't find the container with id 67e7d675bf446a69b3a6422d16a0963b2403b460e0dfd9cf474b433cda589acf
Jul 01 20:48:59.469115 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:59.469087    2571 generic.go:356] "Generic (PLEG): container finished" podID="eb5351f3-0f90-4da6-846a-11ff1ba5f6bb" containerID="74f3c50afb87d7c11a9a402b2a552de82f6c0597fb31f012211c3988f8979db1" exitCode=0
Jul 01 20:48:59.469426 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:59.469178    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb","Type":"ContainerDied","Data":"74f3c50afb87d7c11a9a402b2a552de82f6c0597fb31f012211c3988f8979db1"}
Jul 01 20:48:59.469426 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:59.469246    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb","Type":"ContainerStarted","Data":"67e7d675bf446a69b3a6422d16a0963b2403b460e0dfd9cf474b433cda589acf"}
Jul 01 20:48:59.838024 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:48:59.837989    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd" path="/var/lib/kubelet/pods/8ff8e7a4-df4a-42d9-82f6-8a3ad4d3a8cd/volumes"
Jul 01 20:49:00.477404 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:00.477366    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb","Type":"ContainerStarted","Data":"a7425a591361d31961ef44ff9a2068871875f61f7fbe331a6845692b405462c9"}
Jul 01 20:49:00.477876 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:00.477439    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb","Type":"ContainerStarted","Data":"f0f9cc215d1069316ffc663f8057c0f56d6ebfb37e3d5a4694f45abbee3dfb32"}
Jul 01 20:49:00.477876 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:00.477467    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb","Type":"ContainerStarted","Data":"c084c15bbed8824309bed153a3f728c56ff235eacebfab7a0870814d1b1531c4"}
Jul 01 20:49:00.477876 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:00.477491    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb","Type":"ContainerStarted","Data":"102a1725fba70e4399d5aae0de7c0ff4912afe29608f89e110cca1458c4d1ac6"}
Jul 01 20:49:00.477876 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:00.477516    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb","Type":"ContainerStarted","Data":"785d2db4ac7f1bf0214a0faf7c0e0d15c5d049067cb4e065f2294e4f36a52dc9"}
Jul 01 20:49:00.477876 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:00.477544    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"eb5351f3-0f90-4da6-846a-11ff1ba5f6bb","Type":"ContainerStarted","Data":"464ec66372be01af7f4d3fc9dd2962e91b8fe035677e2368a29c6a6870e23690"}
Jul 01 20:49:00.506284 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:00.506192    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.490637401 podStartE2EDuration="2.506177288s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.539887ms" isStatefulPod=false podCreationTimestamp="2026-07-01 20:48:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:49:00.504172074 +0000 UTC m=+159.016647279" watchObservedRunningTime="2026-07-01 20:49:00.506177288 +0000 UTC m=+159.018652498"
Jul 01 20:49:01.205157 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.205117    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-789f894974-2v8t6"]
Jul 01 20:49:01.208890 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.208863    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-789f894974-2v8t6"
Jul 01 20:49:01.216651 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.215694    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-zr6tq\""
Jul 01 20:49:01.216651 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.215732    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 01 20:49:01.216651 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.216475    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 01 20:49:01.216942 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.216786    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 01 20:49:01.217835 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.217583    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 01 20:49:01.222460 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.222427    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 01 20:49:01.235053 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.235029    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a7e8a91f-7786-4107-8c33-8592e49a15c8-secret-telemeter-client\") pod \"telemeter-client-789f894974-2v8t6\" (UID: \"a7e8a91f-7786-4107-8c33-8592e49a15c8\") " pod="openshift-monitoring/telemeter-client-789f894974-2v8t6"
Jul 01 20:49:01.235130 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.235064    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7e8a91f-7786-4107-8c33-8592e49a15c8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-789f894974-2v8t6\" (UID: \"a7e8a91f-7786-4107-8c33-8592e49a15c8\") " pod="openshift-monitoring/telemeter-client-789f894974-2v8t6"
Jul 01 20:49:01.235130 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.235088    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2r9rx\" (UniqueName: \"kubernetes.io/projected/a7e8a91f-7786-4107-8c33-8592e49a15c8-kube-api-access-2r9rx\") pod \"telemeter-client-789f894974-2v8t6\" (UID: \"a7e8a91f-7786-4107-8c33-8592e49a15c8\") " pod="openshift-monitoring/telemeter-client-789f894974-2v8t6"
Jul 01 20:49:01.235192 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.235139    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a7e8a91f-7786-4107-8c33-8592e49a15c8-federate-client-tls\") pod \"telemeter-client-789f894974-2v8t6\" (UID: \"a7e8a91f-7786-4107-8c33-8592e49a15c8\") " pod="openshift-monitoring/telemeter-client-789f894974-2v8t6"
Jul 01 20:49:01.235192 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.235169    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7e8a91f-7786-4107-8c33-8592e49a15c8-serving-certs-ca-bundle\") pod \"telemeter-client-789f894974-2v8t6\" (UID: \"a7e8a91f-7786-4107-8c33-8592e49a15c8\") " pod="openshift-monitoring/telemeter-client-789f894974-2v8t6"
Jul 01 20:49:01.235253 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.235207    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7e8a91f-7786-4107-8c33-8592e49a15c8-metrics-client-ca\") pod \"telemeter-client-789f894974-2v8t6\" (UID: \"a7e8a91f-7786-4107-8c33-8592e49a15c8\") " pod="openshift-monitoring/telemeter-client-789f894974-2v8t6"
Jul 01 20:49:01.235286 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.235267    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a7e8a91f-7786-4107-8c33-8592e49a15c8-telemeter-client-tls\") pod \"telemeter-client-789f894974-2v8t6\" (UID: \"a7e8a91f-7786-4107-8c33-8592e49a15c8\") " pod="openshift-monitoring/telemeter-client-789f894974-2v8t6"
Jul 01 20:49:01.235319 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.235290    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7e8a91f-7786-4107-8c33-8592e49a15c8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-789f894974-2v8t6\" (UID: \"a7e8a91f-7786-4107-8c33-8592e49a15c8\") " pod="openshift-monitoring/telemeter-client-789f894974-2v8t6"
Jul 01 20:49:01.266023 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.265999    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 01 20:49:01.301127 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.301103    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-789f894974-2v8t6"]
Jul 01 20:49:01.336257 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.336238    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a7e8a91f-7786-4107-8c33-8592e49a15c8-secret-telemeter-client\") pod \"telemeter-client-789f894974-2v8t6\" (UID: \"a7e8a91f-7786-4107-8c33-8592e49a15c8\") " pod="openshift-monitoring/telemeter-client-789f894974-2v8t6"
Jul 01 20:49:01.336357 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.336281    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7e8a91f-7786-4107-8c33-8592e49a15c8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-789f894974-2v8t6\" (UID: \"a7e8a91f-7786-4107-8c33-8592e49a15c8\") " pod="openshift-monitoring/telemeter-client-789f894974-2v8t6"
Jul 01 20:49:01.336434 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.336412    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r9rx\" (UniqueName: \"kubernetes.io/projected/a7e8a91f-7786-4107-8c33-8592e49a15c8-kube-api-access-2r9rx\") pod \"telemeter-client-789f894974-2v8t6\" (UID: \"a7e8a91f-7786-4107-8c33-8592e49a15c8\") " pod="openshift-monitoring/telemeter-client-789f894974-2v8t6"
Jul 01 20:49:01.336485 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.336453    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a7e8a91f-7786-4107-8c33-8592e49a15c8-federate-client-tls\") pod \"telemeter-client-789f894974-2v8t6\" (UID: \"a7e8a91f-7786-4107-8c33-8592e49a15c8\") " pod="openshift-monitoring/telemeter-client-789f894974-2v8t6"
Jul 01 20:49:01.336485 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.336476    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7e8a91f-7786-4107-8c33-8592e49a15c8-serving-certs-ca-bundle\") pod \"telemeter-client-789f894974-2v8t6\" (UID: \"a7e8a91f-7786-4107-8c33-8592e49a15c8\") " pod="openshift-monitoring/telemeter-client-789f894974-2v8t6"
Jul 01 20:49:01.336585 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.336499    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7e8a91f-7786-4107-8c33-8592e49a15c8-metrics-client-ca\") pod \"telemeter-client-789f894974-2v8t6\" (UID: \"a7e8a91f-7786-4107-8c33-8592e49a15c8\") " pod="openshift-monitoring/telemeter-client-789f894974-2v8t6"
Jul 01 20:49:01.336585 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.336537    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a7e8a91f-7786-4107-8c33-8592e49a15c8-telemeter-client-tls\") pod \"telemeter-client-789f894974-2v8t6\" (UID: \"a7e8a91f-7786-4107-8c33-8592e49a15c8\") " pod="openshift-monitoring/telemeter-client-789f894974-2v8t6"
Jul 01 20:49:01.336773 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.336747    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7e8a91f-7786-4107-8c33-8592e49a15c8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-789f894974-2v8t6\" (UID: \"a7e8a91f-7786-4107-8c33-8592e49a15c8\") " pod="openshift-monitoring/telemeter-client-789f894974-2v8t6"
Jul 01 20:49:01.337392 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.337356    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7e8a91f-7786-4107-8c33-8592e49a15c8-serving-certs-ca-bundle\") pod \"telemeter-client-789f894974-2v8t6\" (UID: \"a7e8a91f-7786-4107-8c33-8592e49a15c8\") " pod="openshift-monitoring/telemeter-client-789f894974-2v8t6"
Jul 01 20:49:01.337523 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.337492    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7e8a91f-7786-4107-8c33-8592e49a15c8-metrics-client-ca\") pod \"telemeter-client-789f894974-2v8t6\" (UID: \"a7e8a91f-7786-4107-8c33-8592e49a15c8\") " pod="openshift-monitoring/telemeter-client-789f894974-2v8t6"
Jul 01 20:49:01.337523 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.337518    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7e8a91f-7786-4107-8c33-8592e49a15c8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-789f894974-2v8t6\" (UID: \"a7e8a91f-7786-4107-8c33-8592e49a15c8\") " pod="openshift-monitoring/telemeter-client-789f894974-2v8t6"
Jul 01 20:49:01.339094 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.339070    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a7e8a91f-7786-4107-8c33-8592e49a15c8-federate-client-tls\") pod \"telemeter-client-789f894974-2v8t6\" (UID: \"a7e8a91f-7786-4107-8c33-8592e49a15c8\") " pod="openshift-monitoring/telemeter-client-789f894974-2v8t6"
Jul 01 20:49:01.339171 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.339105    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a7e8a91f-7786-4107-8c33-8592e49a15c8-telemeter-client-tls\") pod \"telemeter-client-789f894974-2v8t6\" (UID: \"a7e8a91f-7786-4107-8c33-8592e49a15c8\") " pod="openshift-monitoring/telemeter-client-789f894974-2v8t6"
Jul 01 20:49:01.339295 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.339272    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7e8a91f-7786-4107-8c33-8592e49a15c8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-789f894974-2v8t6\" (UID: \"a7e8a91f-7786-4107-8c33-8592e49a15c8\") " pod="openshift-monitoring/telemeter-client-789f894974-2v8t6"
Jul 01 20:49:01.339347 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.339333    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a7e8a91f-7786-4107-8c33-8592e49a15c8-secret-telemeter-client\") pod \"telemeter-client-789f894974-2v8t6\" (UID: \"a7e8a91f-7786-4107-8c33-8592e49a15c8\") " pod="openshift-monitoring/telemeter-client-789f894974-2v8t6"
Jul 01 20:49:01.347906 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.347883    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2r9rx\" (UniqueName: \"kubernetes.io/projected/a7e8a91f-7786-4107-8c33-8592e49a15c8-kube-api-access-2r9rx\") pod \"telemeter-client-789f894974-2v8t6\" (UID: \"a7e8a91f-7786-4107-8c33-8592e49a15c8\") " pod="openshift-monitoring/telemeter-client-789f894974-2v8t6"
Jul 01 20:49:01.523857 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.523807    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-789f894974-2v8t6"
Jul 01 20:49:01.541841 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.541817    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 01 20:49:01.542327 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.542097    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b9fcac3d-2ec9-4947-ba90-5caabfa17576" containerName="prometheus" containerID="cri-o://f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce" gracePeriod=600
Jul 01 20:49:01.542327 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.542117    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b9fcac3d-2ec9-4947-ba90-5caabfa17576" containerName="kube-rbac-proxy" containerID="cri-o://ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2" gracePeriod=600
Jul 01 20:49:01.542327 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.542148    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b9fcac3d-2ec9-4947-ba90-5caabfa17576" containerName="kube-rbac-proxy-thanos" containerID="cri-o://e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11" gracePeriod=600
Jul 01 20:49:01.542327 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.542166    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b9fcac3d-2ec9-4947-ba90-5caabfa17576" containerName="thanos-sidecar" containerID="cri-o://ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6" gracePeriod=600
Jul 01 20:49:01.542327 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.542236    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b9fcac3d-2ec9-4947-ba90-5caabfa17576" containerName="config-reloader" containerID="cri-o://23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370" gracePeriod=600
Jul 01 20:49:01.542327 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.542312    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b9fcac3d-2ec9-4947-ba90-5caabfa17576" containerName="kube-rbac-proxy-web" containerID="cri-o://dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378" gracePeriod=600
Jul 01 20:49:01.663601 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.663574    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-789f894974-2v8t6"]
Jul 01 20:49:01.666682 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:49:01.666656    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7e8a91f_7786_4107_8c33_8592e49a15c8.slice/crio-ee5b870608209813f68963bf68fa1259f45720dbb27e1b3a7c3bc5dbb03d1223 WatchSource:0}: Error finding container ee5b870608209813f68963bf68fa1259f45720dbb27e1b3a7c3bc5dbb03d1223: Status 404 returned error can't find the container with id ee5b870608209813f68963bf68fa1259f45720dbb27e1b3a7c3bc5dbb03d1223
Jul 01 20:49:01.786323 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.786300    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:01.841687 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.841662    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-prometheus-k8s-rulefiles-2\") pod \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") "
Jul 01 20:49:01.841795 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.841707    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-prometheus-k8s-rulefiles-0\") pod \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") "
Jul 01 20:49:01.841795 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.841728    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7brxr\" (UniqueName: \"kubernetes.io/projected/b9fcac3d-2ec9-4947-ba90-5caabfa17576-kube-api-access-7brxr\") pod \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") "
Jul 01 20:49:01.841795 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.841748    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-configmap-kubelet-serving-ca-bundle\") pod \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") "
Jul 01 20:49:01.841938 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.841863    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-prometheus-k8s-rulefiles-1\") pod \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") "
Jul 01 20:49:01.841938 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.841899    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b9fcac3d-2ec9-4947-ba90-5caabfa17576-tls-assets\") pod \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") "
Jul 01 20:49:01.842029 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.841948    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-prometheus-trusted-ca-bundle\") pod \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") "
Jul 01 20:49:01.842029 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.841974    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-config\") pod \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") "
Jul 01 20:49:01.842029 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.842003    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-configmap-serving-certs-ca-bundle\") pod \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") "
Jul 01 20:49:01.842181 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.842026    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "b9fcac3d-2ec9-4947-ba90-5caabfa17576" (UID: "b9fcac3d-2ec9-4947-ba90-5caabfa17576"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 20:49:01.842181 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.842033    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-web-config\") pod \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") "
Jul 01 20:49:01.842181 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.842138    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-kube-rbac-proxy\") pod \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") "
Jul 01 20:49:01.842181 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.842149    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "b9fcac3d-2ec9-4947-ba90-5caabfa17576" (UID: "b9fcac3d-2ec9-4947-ba90-5caabfa17576"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 20:49:01.842366 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.842182    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-thanos-prometheus-http-client-file\") pod \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") "
Jul 01 20:49:01.842366 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.842210    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-prometheus-k8s-tls\") pod \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") "
Jul 01 20:49:01.842366 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.842254    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-grpc-tls\") pod \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") "
Jul 01 20:49:01.842366 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.842288    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b9fcac3d-2ec9-4947-ba90-5caabfa17576-config-out\") pod \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") "
Jul 01 20:49:01.842366 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.842314    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") "
Jul 01 20:49:01.842366 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.842350    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-metrics-client-certs\") pod \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") "
Jul 01 20:49:01.842681 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.842380    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") "
Jul 01 20:49:01.842681 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.842420    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b9fcac3d-2ec9-4947-ba90-5caabfa17576-prometheus-k8s-db\") pod \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") "
Jul 01 20:49:01.842681 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.842417    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "b9fcac3d-2ec9-4947-ba90-5caabfa17576" (UID: "b9fcac3d-2ec9-4947-ba90-5caabfa17576"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 20:49:01.842681 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.842456    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-configmap-metrics-client-ca\") pod \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\" (UID: \"b9fcac3d-2ec9-4947-ba90-5caabfa17576\") "
Jul 01 20:49:01.842922 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.842779    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "b9fcac3d-2ec9-4947-ba90-5caabfa17576" (UID: "b9fcac3d-2ec9-4947-ba90-5caabfa17576"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 20:49:01.842922 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.842902    2571 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 20:49:01.843014 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.842921    2571 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 20:49:01.843014 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.842936    2571 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 20:49:01.843014 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.842951    2571 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-prometheus-trusted-ca-bundle\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 20:49:01.843168 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.843142    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "b9fcac3d-2ec9-4947-ba90-5caabfa17576" (UID: "b9fcac3d-2ec9-4947-ba90-5caabfa17576"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 20:49:01.843310 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.843264    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "b9fcac3d-2ec9-4947-ba90-5caabfa17576" (UID: "b9fcac3d-2ec9-4947-ba90-5caabfa17576"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 20:49:01.845180 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.845148    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "b9fcac3d-2ec9-4947-ba90-5caabfa17576" (UID: "b9fcac3d-2ec9-4947-ba90-5caabfa17576"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 20:49:01.845416 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.845387    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "b9fcac3d-2ec9-4947-ba90-5caabfa17576" (UID: "b9fcac3d-2ec9-4947-ba90-5caabfa17576"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 20:49:01.845603 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.845569    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9fcac3d-2ec9-4947-ba90-5caabfa17576-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "b9fcac3d-2ec9-4947-ba90-5caabfa17576" (UID: "b9fcac3d-2ec9-4947-ba90-5caabfa17576"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 20:49:01.846295 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.846269    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "b9fcac3d-2ec9-4947-ba90-5caabfa17576" (UID: "b9fcac3d-2ec9-4947-ba90-5caabfa17576"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 20:49:01.846839 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.846815    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9fcac3d-2ec9-4947-ba90-5caabfa17576-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "b9fcac3d-2ec9-4947-ba90-5caabfa17576" (UID: "b9fcac3d-2ec9-4947-ba90-5caabfa17576"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 20:49:01.847459 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.847430    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "b9fcac3d-2ec9-4947-ba90-5caabfa17576" (UID: "b9fcac3d-2ec9-4947-ba90-5caabfa17576"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 20:49:01.847713 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.847687    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "b9fcac3d-2ec9-4947-ba90-5caabfa17576" (UID: "b9fcac3d-2ec9-4947-ba90-5caabfa17576"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 20:49:01.847800 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.847781    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "b9fcac3d-2ec9-4947-ba90-5caabfa17576" (UID: "b9fcac3d-2ec9-4947-ba90-5caabfa17576"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 20:49:01.847990 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.847955    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9fcac3d-2ec9-4947-ba90-5caabfa17576-config-out" (OuterVolumeSpecName: "config-out") pod "b9fcac3d-2ec9-4947-ba90-5caabfa17576" (UID: "b9fcac3d-2ec9-4947-ba90-5caabfa17576"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 20:49:01.848119 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.848099    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-config" (OuterVolumeSpecName: "config") pod "b9fcac3d-2ec9-4947-ba90-5caabfa17576" (UID: "b9fcac3d-2ec9-4947-ba90-5caabfa17576"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 20:49:01.848258 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.848236    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "b9fcac3d-2ec9-4947-ba90-5caabfa17576" (UID: "b9fcac3d-2ec9-4947-ba90-5caabfa17576"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 20:49:01.848498 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.848476    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9fcac3d-2ec9-4947-ba90-5caabfa17576-kube-api-access-7brxr" (OuterVolumeSpecName: "kube-api-access-7brxr") pod "b9fcac3d-2ec9-4947-ba90-5caabfa17576" (UID: "b9fcac3d-2ec9-4947-ba90-5caabfa17576"). InnerVolumeSpecName "kube-api-access-7brxr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 20:49:01.848982 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.848964    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "b9fcac3d-2ec9-4947-ba90-5caabfa17576" (UID: "b9fcac3d-2ec9-4947-ba90-5caabfa17576"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 20:49:01.856821 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.856798    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-web-config" (OuterVolumeSpecName: "web-config") pod "b9fcac3d-2ec9-4947-ba90-5caabfa17576" (UID: "b9fcac3d-2ec9-4947-ba90-5caabfa17576"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 20:49:01.944248 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.944224    2571 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-web-config\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 20:49:01.944248 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.944247    2571 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-kube-rbac-proxy\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 20:49:01.944400 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.944262    2571 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-thanos-prometheus-http-client-file\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 20:49:01.944400 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.944275    2571 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-prometheus-k8s-tls\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 20:49:01.944400 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.944288    2571 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-grpc-tls\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 20:49:01.944400 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.944301    2571 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b9fcac3d-2ec9-4947-ba90-5caabfa17576-config-out\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 20:49:01.944400 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.944314    2571 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 20:49:01.944400 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.944328    2571 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-metrics-client-certs\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 20:49:01.944400 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.944342    2571 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 20:49:01.944400 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.944357    2571 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b9fcac3d-2ec9-4947-ba90-5caabfa17576-prometheus-k8s-db\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 20:49:01.944400 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.944372    2571 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-configmap-metrics-client-ca\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 20:49:01.944400 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.944386    2571 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 20:49:01.944400 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.944402    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7brxr\" (UniqueName: \"kubernetes.io/projected/b9fcac3d-2ec9-4947-ba90-5caabfa17576-kube-api-access-7brxr\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 20:49:01.944760 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.944415    2571 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b9fcac3d-2ec9-4947-ba90-5caabfa17576-tls-assets\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 20:49:01.944760 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.944429    2571 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/b9fcac3d-2ec9-4947-ba90-5caabfa17576-config\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 20:49:01.944760 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:01.944442    2571 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9fcac3d-2ec9-4947-ba90-5caabfa17576-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 20:49:02.488593 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.488546    2571 generic.go:356] "Generic (PLEG): container finished" podID="b9fcac3d-2ec9-4947-ba90-5caabfa17576" containerID="e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11" exitCode=0
Jul 01 20:49:02.488593 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.488578    2571 generic.go:356] "Generic (PLEG): container finished" podID="b9fcac3d-2ec9-4947-ba90-5caabfa17576" containerID="ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2" exitCode=0
Jul 01 20:49:02.488593 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.488588    2571 generic.go:356] "Generic (PLEG): container finished" podID="b9fcac3d-2ec9-4947-ba90-5caabfa17576" containerID="dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378" exitCode=0
Jul 01 20:49:02.488593 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.488597    2571 generic.go:356] "Generic (PLEG): container finished" podID="b9fcac3d-2ec9-4947-ba90-5caabfa17576" containerID="ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6" exitCode=0
Jul 01 20:49:02.488916 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.488604    2571 generic.go:356] "Generic (PLEG): container finished" podID="b9fcac3d-2ec9-4947-ba90-5caabfa17576" containerID="23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370" exitCode=0
Jul 01 20:49:02.488916 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.488613    2571 generic.go:356] "Generic (PLEG): container finished" podID="b9fcac3d-2ec9-4947-ba90-5caabfa17576" containerID="f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce" exitCode=0
Jul 01 20:49:02.488916 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.488644    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b9fcac3d-2ec9-4947-ba90-5caabfa17576","Type":"ContainerDied","Data":"e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11"}
Jul 01 20:49:02.488916 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.488666    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.488916 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.488745    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b9fcac3d-2ec9-4947-ba90-5caabfa17576","Type":"ContainerDied","Data":"ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2"}
Jul 01 20:49:02.488916 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.488776    2571 scope.go:117] "RemoveContainer" containerID="e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11"
Jul 01 20:49:02.488916 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.488795    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b9fcac3d-2ec9-4947-ba90-5caabfa17576","Type":"ContainerDied","Data":"dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378"}
Jul 01 20:49:02.488916 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.488841    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b9fcac3d-2ec9-4947-ba90-5caabfa17576","Type":"ContainerDied","Data":"ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6"}
Jul 01 20:49:02.488916 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.488886    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b9fcac3d-2ec9-4947-ba90-5caabfa17576","Type":"ContainerDied","Data":"23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370"}
Jul 01 20:49:02.489304 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.488931    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b9fcac3d-2ec9-4947-ba90-5caabfa17576","Type":"ContainerDied","Data":"f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce"}
Jul 01 20:49:02.489304 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.488979    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b9fcac3d-2ec9-4947-ba90-5caabfa17576","Type":"ContainerDied","Data":"d80ec6190acfff37ed5ba75e0d5e0443d5087cc2ad5c04bda5be0dea7619a7f1"}
Jul 01 20:49:02.490332 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.490308    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-789f894974-2v8t6" event={"ID":"a7e8a91f-7786-4107-8c33-8592e49a15c8","Type":"ContainerStarted","Data":"ee5b870608209813f68963bf68fa1259f45720dbb27e1b3a7c3bc5dbb03d1223"}
Jul 01 20:49:02.499173 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.499151    2571 scope.go:117] "RemoveContainer" containerID="ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2"
Jul 01 20:49:02.509466 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.509446    2571 scope.go:117] "RemoveContainer" containerID="dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378"
Jul 01 20:49:02.516060 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.516039    2571 scope.go:117] "RemoveContainer" containerID="ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6"
Jul 01 20:49:02.519292 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.519267    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 01 20:49:02.523809 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.523787    2571 scope.go:117] "RemoveContainer" containerID="23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370"
Jul 01 20:49:02.524749 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.524646    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 01 20:49:02.530216 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.530081    2571 scope.go:117] "RemoveContainer" containerID="f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce"
Jul 01 20:49:02.536968 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.536952    2571 scope.go:117] "RemoveContainer" containerID="8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03"
Jul 01 20:49:02.543334 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.543320    2571 scope.go:117] "RemoveContainer" containerID="e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11"
Jul 01 20:49:02.543582 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:49:02.543565    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11\": container with ID starting with e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11 not found: ID does not exist" containerID="e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11"
Jul 01 20:49:02.543650 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.543592    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11"} err="failed to get container status \"e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11\": rpc error: code = NotFound desc = could not find container \"e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11\": container with ID starting with e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11 not found: ID does not exist"
Jul 01 20:49:02.543650 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.543608    2571 scope.go:117] "RemoveContainer" containerID="ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2"
Jul 01 20:49:02.543851 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:49:02.543833    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2\": container with ID starting with ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2 not found: ID does not exist" containerID="ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2"
Jul 01 20:49:02.543911 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.543860    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2"} err="failed to get container status \"ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2\": rpc error: code = NotFound desc = could not find container \"ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2\": container with ID starting with ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2 not found: ID does not exist"
Jul 01 20:49:02.543911 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.543881    2571 scope.go:117] "RemoveContainer" containerID="dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378"
Jul 01 20:49:02.544117 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:49:02.544101    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378\": container with ID starting with dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378 not found: ID does not exist" containerID="dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378"
Jul 01 20:49:02.544157 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.544124    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378"} err="failed to get container status \"dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378\": rpc error: code = NotFound desc = could not find container \"dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378\": container with ID starting with dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378 not found: ID does not exist"
Jul 01 20:49:02.544157 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.544143    2571 scope.go:117] "RemoveContainer" containerID="ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6"
Jul 01 20:49:02.544360 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:49:02.544343    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6\": container with ID starting with ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6 not found: ID does not exist" containerID="ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6"
Jul 01 20:49:02.544397 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.544372    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6"} err="failed to get container status \"ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6\": rpc error: code = NotFound desc = could not find container \"ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6\": container with ID starting with ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6 not found: ID does not exist"
Jul 01 20:49:02.544397 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.544388    2571 scope.go:117] "RemoveContainer" containerID="23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370"
Jul 01 20:49:02.544578 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:49:02.544562    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370\": container with ID starting with 23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370 not found: ID does not exist" containerID="23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370"
Jul 01 20:49:02.544621 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.544584    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370"} err="failed to get container status \"23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370\": rpc error: code = NotFound desc = could not find container \"23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370\": container with ID starting with 23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370 not found: ID does not exist"
Jul 01 20:49:02.544621 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.544602    2571 scope.go:117] "RemoveContainer" containerID="f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce"
Jul 01 20:49:02.544885 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:49:02.544870    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce\": container with ID starting with f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce not found: ID does not exist" containerID="f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce"
Jul 01 20:49:02.544940 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.544897    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce"} err="failed to get container status \"f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce\": rpc error: code = NotFound desc = could not find container \"f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce\": container with ID starting with f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce not found: ID does not exist"
Jul 01 20:49:02.544940 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.544909    2571 scope.go:117] "RemoveContainer" containerID="8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03"
Jul 01 20:49:02.545129 ip-10-0-140-104 kubenswrapper[2571]: E0701 20:49:02.545113    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03\": container with ID starting with 8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03 not found: ID does not exist" containerID="8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03"
Jul 01 20:49:02.545190 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.545136    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03"} err="failed to get container status \"8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03\": rpc error: code = NotFound desc = could not find container \"8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03\": container with ID starting with 8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03 not found: ID does not exist"
Jul 01 20:49:02.545190 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.545156    2571 scope.go:117] "RemoveContainer" containerID="e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11"
Jul 01 20:49:02.545392 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.545371    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11"} err="failed to get container status \"e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11\": rpc error: code = NotFound desc = could not find container \"e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11\": container with ID starting with e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11 not found: ID does not exist"
Jul 01 20:49:02.545444 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.545393    2571 scope.go:117] "RemoveContainer" containerID="ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2"
Jul 01 20:49:02.545652 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.545614    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2"} err="failed to get container status \"ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2\": rpc error: code = NotFound desc = could not find container \"ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2\": container with ID starting with ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2 not found: ID does not exist"
Jul 01 20:49:02.545707 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.545653    2571 scope.go:117] "RemoveContainer" containerID="dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378"
Jul 01 20:49:02.545884 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.545865    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378"} err="failed to get container status \"dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378\": rpc error: code = NotFound desc = could not find container \"dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378\": container with ID starting with dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378 not found: ID does not exist"
Jul 01 20:49:02.545926 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.545886    2571 scope.go:117] "RemoveContainer" containerID="ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6"
Jul 01 20:49:02.546119 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.546102    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6"} err="failed to get container status \"ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6\": rpc error: code = NotFound desc = could not find container \"ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6\": container with ID starting with ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6 not found: ID does not exist"
Jul 01 20:49:02.546181 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.546122    2571 scope.go:117] "RemoveContainer" containerID="23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370"
Jul 01 20:49:02.546343 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.546325    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370"} err="failed to get container status \"23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370\": rpc error: code = NotFound desc = could not find container \"23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370\": container with ID starting with 23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370 not found: ID does not exist"
Jul 01 20:49:02.546402 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.546344    2571 scope.go:117] "RemoveContainer" containerID="f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce"
Jul 01 20:49:02.546557 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.546537    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce"} err="failed to get container status \"f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce\": rpc error: code = NotFound desc = could not find container \"f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce\": container with ID starting with f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce not found: ID does not exist"
Jul 01 20:49:02.546557 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.546554    2571 scope.go:117] "RemoveContainer" containerID="8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03"
Jul 01 20:49:02.546795 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.546778    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03"} err="failed to get container status \"8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03\": rpc error: code = NotFound desc = could not find container \"8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03\": container with ID starting with 8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03 not found: ID does not exist"
Jul 01 20:49:02.546842 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.546796    2571 scope.go:117] "RemoveContainer" containerID="e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11"
Jul 01 20:49:02.547002 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.546983    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11"} err="failed to get container status \"e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11\": rpc error: code = NotFound desc = could not find container \"e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11\": container with ID starting with e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11 not found: ID does not exist"
Jul 01 20:49:02.547002 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.547000    2571 scope.go:117] "RemoveContainer" containerID="ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2"
Jul 01 20:49:02.547226 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.547208    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2"} err="failed to get container status \"ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2\": rpc error: code = NotFound desc = could not find container \"ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2\": container with ID starting with ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2 not found: ID does not exist"
Jul 01 20:49:02.547268 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.547235    2571 scope.go:117] "RemoveContainer" containerID="dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378"
Jul 01 20:49:02.547439 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.547421    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378"} err="failed to get container status \"dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378\": rpc error: code = NotFound desc = could not find container \"dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378\": container with ID starting with dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378 not found: ID does not exist"
Jul 01 20:49:02.547506 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.547441    2571 scope.go:117] "RemoveContainer" containerID="ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6"
Jul 01 20:49:02.547669 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.547652    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6"} err="failed to get container status \"ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6\": rpc error: code = NotFound desc = could not find container \"ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6\": container with ID starting with ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6 not found: ID does not exist"
Jul 01 20:49:02.547712 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.547670    2571 scope.go:117] "RemoveContainer" containerID="23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370"
Jul 01 20:49:02.547869 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.547852    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370"} err="failed to get container status \"23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370\": rpc error: code = NotFound desc = could not find container \"23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370\": container with ID starting with 23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370 not found: ID does not exist"
Jul 01 20:49:02.547929 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.547870    2571 scope.go:117] "RemoveContainer" containerID="f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce"
Jul 01 20:49:02.548075 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.548057    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce"} err="failed to get container status \"f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce\": rpc error: code = NotFound desc = could not find container \"f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce\": container with ID starting with f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce not found: ID does not exist"
Jul 01 20:49:02.548075 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.548071    2571 scope.go:117] "RemoveContainer" containerID="8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03"
Jul 01 20:49:02.548276 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.548258    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03"} err="failed to get container status \"8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03\": rpc error: code = NotFound desc = could not find container \"8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03\": container with ID starting with 8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03 not found: ID does not exist"
Jul 01 20:49:02.548326 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.548277    2571 scope.go:117] "RemoveContainer" containerID="e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11"
Jul 01 20:49:02.548494 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.548478    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11"} err="failed to get container status \"e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11\": rpc error: code = NotFound desc = could not find container \"e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11\": container with ID starting with e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11 not found: ID does not exist"
Jul 01 20:49:02.548533 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.548493    2571 scope.go:117] "RemoveContainer" containerID="ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2"
Jul 01 20:49:02.548721 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.548705    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2"} err="failed to get container status \"ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2\": rpc error: code = NotFound desc = could not find container \"ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2\": container with ID starting with ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2 not found: ID does not exist"
Jul 01 20:49:02.548721 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.548720    2571 scope.go:117] "RemoveContainer" containerID="dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378"
Jul 01 20:49:02.548929 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.548909    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378"} err="failed to get container status \"dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378\": rpc error: code = NotFound desc = could not find container \"dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378\": container with ID starting with dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378 not found: ID does not exist"
Jul 01 20:49:02.548929 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.548928    2571 scope.go:117] "RemoveContainer" containerID="ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6"
Jul 01 20:49:02.549124 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.549109    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6"} err="failed to get container status \"ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6\": rpc error: code = NotFound desc = could not find container \"ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6\": container with ID starting with ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6 not found: ID does not exist"
Jul 01 20:49:02.549124 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.549123    2571 scope.go:117] "RemoveContainer" containerID="23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370"
Jul 01 20:49:02.549301 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.549284    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370"} err="failed to get container status \"23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370\": rpc error: code = NotFound desc = could not find container \"23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370\": container with ID starting with 23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370 not found: ID does not exist"
Jul 01 20:49:02.549361 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.549302    2571 scope.go:117] "RemoveContainer" containerID="f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce"
Jul 01 20:49:02.549525 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.549509    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce"} err="failed to get container status \"f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce\": rpc error: code = NotFound desc = could not find container \"f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce\": container with ID starting with f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce not found: ID does not exist"
Jul 01 20:49:02.549572 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.549526    2571 scope.go:117] "RemoveContainer" containerID="8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03"
Jul 01 20:49:02.549743 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.549719    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03"} err="failed to get container status \"8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03\": rpc error: code = NotFound desc = could not find container \"8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03\": container with ID starting with 8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03 not found: ID does not exist"
Jul 01 20:49:02.549743 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.549740    2571 scope.go:117] "RemoveContainer" containerID="e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11"
Jul 01 20:49:02.549977 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.549925    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11"} err="failed to get container status \"e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11\": rpc error: code = NotFound desc = could not find container \"e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11\": container with ID starting with e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11 not found: ID does not exist"
Jul 01 20:49:02.549977 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.549945    2571 scope.go:117] "RemoveContainer" containerID="ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2"
Jul 01 20:49:02.550282 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.550248    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2"} err="failed to get container status \"ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2\": rpc error: code = NotFound desc = could not find container \"ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2\": container with ID starting with ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2 not found: ID does not exist"
Jul 01 20:49:02.550352 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.550271    2571 scope.go:117] "RemoveContainer" containerID="dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378"
Jul 01 20:49:02.550617 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.550592    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378"} err="failed to get container status \"dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378\": rpc error: code = NotFound desc = could not find container \"dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378\": container with ID starting with dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378 not found: ID does not exist"
Jul 01 20:49:02.550617 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.550617    2571 scope.go:117] "RemoveContainer" containerID="ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6"
Jul 01 20:49:02.550896 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.550872    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6"} err="failed to get container status \"ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6\": rpc error: code = NotFound desc = could not find container \"ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6\": container with ID starting with ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6 not found: ID does not exist"
Jul 01 20:49:02.550956 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.550899    2571 scope.go:117] "RemoveContainer" containerID="23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370"
Jul 01 20:49:02.551180 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.551142    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370"} err="failed to get container status \"23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370\": rpc error: code = NotFound desc = could not find container \"23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370\": container with ID starting with 23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370 not found: ID does not exist"
Jul 01 20:49:02.551180 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.551176    2571 scope.go:117] "RemoveContainer" containerID="f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce"
Jul 01 20:49:02.551466 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.551433    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce"} err="failed to get container status \"f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce\": rpc error: code = NotFound desc = could not find container \"f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce\": container with ID starting with f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce not found: ID does not exist"
Jul 01 20:49:02.551529 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.551470    2571 scope.go:117] "RemoveContainer" containerID="8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03"
Jul 01 20:49:02.551747 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.551724    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03"} err="failed to get container status \"8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03\": rpc error: code = NotFound desc = could not find container \"8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03\": container with ID starting with 8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03 not found: ID does not exist"
Jul 01 20:49:02.551828 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.551750    2571 scope.go:117] "RemoveContainer" containerID="e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11"
Jul 01 20:49:02.552031 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.552008    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11"} err="failed to get container status \"e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11\": rpc error: code = NotFound desc = could not find container \"e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11\": container with ID starting with e623fea7f62feb710ade21fc41049cd97af1bf001195424cf93522045344af11 not found: ID does not exist"
Jul 01 20:49:02.552077 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.552031    2571 scope.go:117] "RemoveContainer" containerID="ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2"
Jul 01 20:49:02.552114 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.552103    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 01 20:49:02.552300 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.552266    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2"} err="failed to get container status \"ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2\": rpc error: code = NotFound desc = could not find container \"ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2\": container with ID starting with ef562850dca24cc2557f2c4c57feac3ad6e395b2f6d29763cd618dcab5b1b5c2 not found: ID does not exist"
Jul 01 20:49:02.552300 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.552290    2571 scope.go:117] "RemoveContainer" containerID="dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378"
Jul 01 20:49:02.552412 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.552363    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9fcac3d-2ec9-4947-ba90-5caabfa17576" containerName="init-config-reloader"
Jul 01 20:49:02.552412 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.552374    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9fcac3d-2ec9-4947-ba90-5caabfa17576" containerName="init-config-reloader"
Jul 01 20:49:02.552412 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.552386    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9fcac3d-2ec9-4947-ba90-5caabfa17576" containerName="kube-rbac-proxy"
Jul 01 20:49:02.552412 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.552394    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9fcac3d-2ec9-4947-ba90-5caabfa17576" containerName="kube-rbac-proxy"
Jul 01 20:49:02.552412 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.552405    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9fcac3d-2ec9-4947-ba90-5caabfa17576" containerName="thanos-sidecar"
Jul 01 20:49:02.552412 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.552411    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9fcac3d-2ec9-4947-ba90-5caabfa17576" containerName="thanos-sidecar"
Jul 01 20:49:02.552606 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.552420    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9fcac3d-2ec9-4947-ba90-5caabfa17576" containerName="kube-rbac-proxy-thanos"
Jul 01 20:49:02.552606 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.552429    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9fcac3d-2ec9-4947-ba90-5caabfa17576" containerName="kube-rbac-proxy-thanos"
Jul 01 20:49:02.552606 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.552442    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9fcac3d-2ec9-4947-ba90-5caabfa17576" containerName="kube-rbac-proxy-web"
Jul 01 20:49:02.552606 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.552451    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9fcac3d-2ec9-4947-ba90-5caabfa17576" containerName="kube-rbac-proxy-web"
Jul 01 20:49:02.552606 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.552468    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9fcac3d-2ec9-4947-ba90-5caabfa17576" containerName="prometheus"
Jul 01 20:49:02.552606 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.552476    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9fcac3d-2ec9-4947-ba90-5caabfa17576" containerName="prometheus"
Jul 01 20:49:02.552606 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.552485    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9fcac3d-2ec9-4947-ba90-5caabfa17576" containerName="config-reloader"
Jul 01 20:49:02.552606 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.552493    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9fcac3d-2ec9-4947-ba90-5caabfa17576" containerName="config-reloader"
Jul 01 20:49:02.552606 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.552528    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378"} err="failed to get container status \"dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378\": rpc error: code = NotFound desc = could not find container \"dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378\": container with ID starting with dd10a43b9c21ce5bc9fa1e9090e5f39136659063b6135dd38254ec19af0ba378 not found: ID does not exist"
Jul 01 20:49:02.552606 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.552548    2571 scope.go:117] "RemoveContainer" containerID="ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6"
Jul 01 20:49:02.552606 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.552551    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9fcac3d-2ec9-4947-ba90-5caabfa17576" containerName="kube-rbac-proxy-web"
Jul 01 20:49:02.552606 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.552565    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9fcac3d-2ec9-4947-ba90-5caabfa17576" containerName="kube-rbac-proxy"
Jul 01 20:49:02.552606 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.552573    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9fcac3d-2ec9-4947-ba90-5caabfa17576" containerName="kube-rbac-proxy-thanos"
Jul 01 20:49:02.552606 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.552583    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9fcac3d-2ec9-4947-ba90-5caabfa17576" containerName="config-reloader"
Jul 01 20:49:02.552606 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.552591    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9fcac3d-2ec9-4947-ba90-5caabfa17576" containerName="thanos-sidecar"
Jul 01 20:49:02.552606 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.552599    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9fcac3d-2ec9-4947-ba90-5caabfa17576" containerName="prometheus"
Jul 01 20:49:02.553201 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.552814    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6"} err="failed to get container status \"ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6\": rpc error: code = NotFound desc = could not find container \"ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6\": container with ID starting with ec221f0a104ee3d68aa78494f664077499c7f3c00e0928eeb1cc6710a1bbc6b6 not found: ID does not exist"
Jul 01 20:49:02.553201 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.552835    2571 scope.go:117] "RemoveContainer" containerID="23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370"
Jul 01 20:49:02.553201 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.553072    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370"} err="failed to get container status \"23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370\": rpc error: code = NotFound desc = could not find container \"23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370\": container with ID starting with 23004022d729dc995d32bdb5c4c17ec9fe8c18e31c624c4b9efd25cbef89f370 not found: ID does not exist"
Jul 01 20:49:02.553201 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.553089    2571 scope.go:117] "RemoveContainer" containerID="f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce"
Jul 01 20:49:02.553385 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.553294    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce"} err="failed to get container status \"f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce\": rpc error: code = NotFound desc = could not find container \"f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce\": container with ID starting with f6d441eeddbae9032a24e178ec8b3f8c17667acb805034b7bb0238659ec791ce not found: ID does not exist"
Jul 01 20:49:02.553385 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.553315    2571 scope.go:117] "RemoveContainer" containerID="8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03"
Jul 01 20:49:02.553561 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.553543    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03"} err="failed to get container status \"8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03\": rpc error: code = NotFound desc = could not find container \"8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03\": container with ID starting with 8d7f76d0b58f6eef1b6eb396a7778a720cfbb992069f715319e182842d268b03 not found: ID does not exist"
Jul 01 20:49:02.557620 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.557606    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.561813 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.561790    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 01 20:49:02.561903 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.561887    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 01 20:49:02.561984 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.561897    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 01 20:49:02.561984 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.561903    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 01 20:49:02.562281 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.562237    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 01 20:49:02.562281 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.562263    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 01 20:49:02.562458 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.562280    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 01 20:49:02.562458 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.562299    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 01 20:49:02.562458 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.562309    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-v79nx\""
Jul 01 20:49:02.562640 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.562596    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 01 20:49:02.562712 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.562658    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 01 20:49:02.563104 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.563085    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 01 20:49:02.563104 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.563097    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 01 20:49:02.563300 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.563108    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-39a23vgovq5u0\""
Jul 01 20:49:02.563300 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.563162    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 01 20:49:02.565912 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.565891    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 01 20:49:02.567687 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.567672    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 01 20:49:02.573590 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.573569    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 01 20:49:02.649243 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.649196    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8a1b7312-62c8-47fe-a17d-2384295f4db6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.649243 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.649223    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a1b7312-62c8-47fe-a17d-2384295f4db6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.649243 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.649240    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8a1b7312-62c8-47fe-a17d-2384295f4db6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.649376 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.649261    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8a1b7312-62c8-47fe-a17d-2384295f4db6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.649376 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.649304    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a1b7312-62c8-47fe-a17d-2384295f4db6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.649376 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.649339    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8a1b7312-62c8-47fe-a17d-2384295f4db6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.649376 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.649357    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8a1b7312-62c8-47fe-a17d-2384295f4db6-web-config\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.649505 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.649377    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a1b7312-62c8-47fe-a17d-2384295f4db6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.649505 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.649397    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a1b7312-62c8-47fe-a17d-2384295f4db6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.649505 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.649447    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8a1b7312-62c8-47fe-a17d-2384295f4db6-config-out\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.649505 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.649463    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a1b7312-62c8-47fe-a17d-2384295f4db6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.649505 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.649486    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8a1b7312-62c8-47fe-a17d-2384295f4db6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.649649 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.649514    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a1b7312-62c8-47fe-a17d-2384295f4db6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.649649 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.649539    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8a1b7312-62c8-47fe-a17d-2384295f4db6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.649649 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.649577    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdd7g\" (UniqueName: \"kubernetes.io/projected/8a1b7312-62c8-47fe-a17d-2384295f4db6-kube-api-access-tdd7g\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.649649 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.649606    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8a1b7312-62c8-47fe-a17d-2384295f4db6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.649765 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.649648    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8a1b7312-62c8-47fe-a17d-2384295f4db6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.649765 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.649676    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8a1b7312-62c8-47fe-a17d-2384295f4db6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.649765 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.649693    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8a1b7312-62c8-47fe-a17d-2384295f4db6-config\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.649765 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.649706    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8a1b7312-62c8-47fe-a17d-2384295f4db6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.750650 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.750612    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8a1b7312-62c8-47fe-a17d-2384295f4db6-config-out\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.750750 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.750660    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a1b7312-62c8-47fe-a17d-2384295f4db6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.750750 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.750685    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8a1b7312-62c8-47fe-a17d-2384295f4db6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.750854 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.750793    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a1b7312-62c8-47fe-a17d-2384295f4db6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.750854 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.750821    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8a1b7312-62c8-47fe-a17d-2384295f4db6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.750854 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.750847    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdd7g\" (UniqueName: \"kubernetes.io/projected/8a1b7312-62c8-47fe-a17d-2384295f4db6-kube-api-access-tdd7g\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.752364 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.750883    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8a1b7312-62c8-47fe-a17d-2384295f4db6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.752364 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.750912    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8a1b7312-62c8-47fe-a17d-2384295f4db6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.752364 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.750961    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8a1b7312-62c8-47fe-a17d-2384295f4db6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.752364 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.751000    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8a1b7312-62c8-47fe-a17d-2384295f4db6-config\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.752364 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.751030    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8a1b7312-62c8-47fe-a17d-2384295f4db6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.752364 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.751082    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8a1b7312-62c8-47fe-a17d-2384295f4db6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.752364 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.751113    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a1b7312-62c8-47fe-a17d-2384295f4db6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.752364 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.751140    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8a1b7312-62c8-47fe-a17d-2384295f4db6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.752364 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.751175    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8a1b7312-62c8-47fe-a17d-2384295f4db6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.752364 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.751206    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a1b7312-62c8-47fe-a17d-2384295f4db6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.752364 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.751246    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8a1b7312-62c8-47fe-a17d-2384295f4db6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.752364 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.751276    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8a1b7312-62c8-47fe-a17d-2384295f4db6-web-config\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.752364 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.751311    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a1b7312-62c8-47fe-a17d-2384295f4db6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.752364 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.751342    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a1b7312-62c8-47fe-a17d-2384295f4db6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.752364 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.751442    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a1b7312-62c8-47fe-a17d-2384295f4db6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.752364 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.751862    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8a1b7312-62c8-47fe-a17d-2384295f4db6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.753456 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.752405    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a1b7312-62c8-47fe-a17d-2384295f4db6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.753456 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.753200    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a1b7312-62c8-47fe-a17d-2384295f4db6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.753615 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.753593    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8a1b7312-62c8-47fe-a17d-2384295f4db6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.755025 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.754996    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a1b7312-62c8-47fe-a17d-2384295f4db6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.755772 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.755274    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8a1b7312-62c8-47fe-a17d-2384295f4db6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.755772 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.755431    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8a1b7312-62c8-47fe-a17d-2384295f4db6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.755772 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.755609    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8a1b7312-62c8-47fe-a17d-2384295f4db6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.755772 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.755693    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8a1b7312-62c8-47fe-a17d-2384295f4db6-config-out\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.756020 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.755835    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8a1b7312-62c8-47fe-a17d-2384295f4db6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.756020 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.755999    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a1b7312-62c8-47fe-a17d-2384295f4db6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.756138 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.756090    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8a1b7312-62c8-47fe-a17d-2384295f4db6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.756456 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.756433    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8a1b7312-62c8-47fe-a17d-2384295f4db6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.756806 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.756729    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8a1b7312-62c8-47fe-a17d-2384295f4db6-web-config\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.757116 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.757094    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/8a1b7312-62c8-47fe-a17d-2384295f4db6-config\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.757207 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.757183    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8a1b7312-62c8-47fe-a17d-2384295f4db6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.757990 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.757968    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8a1b7312-62c8-47fe-a17d-2384295f4db6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.758479 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.758460    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a1b7312-62c8-47fe-a17d-2384295f4db6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.759933 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.759910    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdd7g\" (UniqueName: \"kubernetes.io/projected/8a1b7312-62c8-47fe-a17d-2384295f4db6-kube-api-access-tdd7g\") pod \"prometheus-k8s-0\" (UID: \"8a1b7312-62c8-47fe-a17d-2384295f4db6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:02.867151 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:02.867130    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:49:03.015930 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:03.015901    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 01 20:49:03.017793 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:49:03.017761    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a1b7312_62c8_47fe_a17d_2384295f4db6.slice/crio-c222134685e6c59384786cdbe37caa136274e2d3af27bdb04f5327dc53a5ed3c WatchSource:0}: Error finding container c222134685e6c59384786cdbe37caa136274e2d3af27bdb04f5327dc53a5ed3c: Status 404 returned error can't find the container with id c222134685e6c59384786cdbe37caa136274e2d3af27bdb04f5327dc53a5ed3c
Jul 01 20:49:03.497733 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:03.497694    2571 generic.go:356] "Generic (PLEG): container finished" podID="8a1b7312-62c8-47fe-a17d-2384295f4db6" containerID="c17787c857310c980ae14862a5c3fa1dc6af3806d5731952024cf4bae65ba9f4" exitCode=0
Jul 01 20:49:03.497949 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:03.497786    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a1b7312-62c8-47fe-a17d-2384295f4db6","Type":"ContainerDied","Data":"c17787c857310c980ae14862a5c3fa1dc6af3806d5731952024cf4bae65ba9f4"}
Jul 01 20:49:03.497949 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:03.497866    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a1b7312-62c8-47fe-a17d-2384295f4db6","Type":"ContainerStarted","Data":"c222134685e6c59384786cdbe37caa136274e2d3af27bdb04f5327dc53a5ed3c"}
Jul 01 20:49:03.839577 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:03.839500    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9fcac3d-2ec9-4947-ba90-5caabfa17576" path="/var/lib/kubelet/pods/b9fcac3d-2ec9-4947-ba90-5caabfa17576/volumes"
Jul 01 20:49:04.503432 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:04.503396    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-789f894974-2v8t6" event={"ID":"a7e8a91f-7786-4107-8c33-8592e49a15c8","Type":"ContainerStarted","Data":"4aeddeff6edc3864d28056b50d074808b2725e54bbf6dbdbba8fcd02e775d25a"}
Jul 01 20:49:04.503592 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:04.503460    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-789f894974-2v8t6" event={"ID":"a7e8a91f-7786-4107-8c33-8592e49a15c8","Type":"ContainerStarted","Data":"702804ff3dc994005ddfcff6be89ed5ac9f967ab20b147bda5936a5587619d5e"}
Jul 01 20:49:04.503592 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:04.503495    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-789f894974-2v8t6" event={"ID":"a7e8a91f-7786-4107-8c33-8592e49a15c8","Type":"ContainerStarted","Data":"77be40e46a052df4bbbd6d7ac48e4e61500442da3faf6eea966a5c56ffe8ee7a"}
Jul 01 20:49:04.506751 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:04.506729    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a1b7312-62c8-47fe-a17d-2384295f4db6","Type":"ContainerStarted","Data":"9beece9f369b91fc485ce0fd1c1cd5a3dae513eeec24cecdad744bec2e53d788"}
Jul 01 20:49:04.506863 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:04.506782    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a1b7312-62c8-47fe-a17d-2384295f4db6","Type":"ContainerStarted","Data":"485c65d5685549dfb1e3dda8d8d7ef7c88927c5708c23dd203d5627b196ee0eb"}
Jul 01 20:49:04.506863 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:04.506807    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a1b7312-62c8-47fe-a17d-2384295f4db6","Type":"ContainerStarted","Data":"45eab791545e2fa794f02a47297115263703a349b79e0232f595a336bb2dd8a5"}
Jul 01 20:49:04.506863 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:04.506832    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a1b7312-62c8-47fe-a17d-2384295f4db6","Type":"ContainerStarted","Data":"866d393846aff34fcf46ab87a6ae7f0292c4c47ccca2f547c5cef13600b39c14"}
Jul 01 20:49:04.506863 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:04.506856    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a1b7312-62c8-47fe-a17d-2384295f4db6","Type":"ContainerStarted","Data":"b015d8c01ff6af7e7357a76c588e6f2bcd077ef367ac0527a653c5466396d4d6"}
Jul 01 20:49:04.506978 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:04.506880    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a1b7312-62c8-47fe-a17d-2384295f4db6","Type":"ContainerStarted","Data":"f82038146426a8ba196c9f15976e9b33aef34d97fb05c1042134ece4f47b3d2d"}
Jul 01 20:49:04.531660 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:04.531592    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-789f894974-2v8t6" podStartSLOduration=1.078733333 podStartE2EDuration="3.531574422s" totalImagesPullingTime="2.452841089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:49:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:49:04.529197147 +0000 UTC m=+163.041672333" watchObservedRunningTime="2026-07-01 20:49:04.531574422 +0000 UTC m=+163.044049613"
Jul 01 20:49:04.560403 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:04.560361    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.538391467 podStartE2EDuration="2.56034674s" totalImagesPullingTime="0s" totalInitContainerRuntime="21.955273ms" isStatefulPod=false podCreationTimestamp="2026-07-01 20:49:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:49:04.558773489 +0000 UTC m=+163.071248679" watchObservedRunningTime="2026-07-01 20:49:04.56034674 +0000 UTC m=+163.072821930"
Jul 01 20:49:07.867282 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:49:07.867248    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:50:02.868291 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:50:02.868260    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:50:02.884754 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:50:02.884727    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:50:03.739758 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:50:03.739729    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 01 20:51:21.790101 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:51:21.790077    2571 kubelet.go:1648] "Image garbage collection succeeded"
Jul 01 20:53:38.124001 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:38.123960    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-5b66478c58-nbfvs"]
Jul 01 20:53:38.127322 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:38.127302    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-5b66478c58-nbfvs"
Jul 01 20:53:38.130563 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:38.130536    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 01 20:53:38.130732 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:38.130606    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 01 20:53:38.131177 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:38.131158    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-knxmj\""
Jul 01 20:53:38.157350 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:38.157319    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-5b66478c58-nbfvs"]
Jul 01 20:53:38.286570 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:38.286546    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa1a579e-8c01-4f02-8c43-757daf8ddb28-tmp\") pod \"cert-manager-operator-controller-manager-5b66478c58-nbfvs\" (UID: \"aa1a579e-8c01-4f02-8c43-757daf8ddb28\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-5b66478c58-nbfvs"
Jul 01 20:53:38.286689 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:38.286582    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnzt2\" (UniqueName: \"kubernetes.io/projected/aa1a579e-8c01-4f02-8c43-757daf8ddb28-kube-api-access-hnzt2\") pod \"cert-manager-operator-controller-manager-5b66478c58-nbfvs\" (UID: \"aa1a579e-8c01-4f02-8c43-757daf8ddb28\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-5b66478c58-nbfvs"
Jul 01 20:53:38.387898 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:38.387836    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa1a579e-8c01-4f02-8c43-757daf8ddb28-tmp\") pod \"cert-manager-operator-controller-manager-5b66478c58-nbfvs\" (UID: \"aa1a579e-8c01-4f02-8c43-757daf8ddb28\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-5b66478c58-nbfvs"
Jul 01 20:53:38.387898 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:38.387881    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnzt2\" (UniqueName: \"kubernetes.io/projected/aa1a579e-8c01-4f02-8c43-757daf8ddb28-kube-api-access-hnzt2\") pod \"cert-manager-operator-controller-manager-5b66478c58-nbfvs\" (UID: \"aa1a579e-8c01-4f02-8c43-757daf8ddb28\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-5b66478c58-nbfvs"
Jul 01 20:53:38.388181 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:38.388163    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa1a579e-8c01-4f02-8c43-757daf8ddb28-tmp\") pod \"cert-manager-operator-controller-manager-5b66478c58-nbfvs\" (UID: \"aa1a579e-8c01-4f02-8c43-757daf8ddb28\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-5b66478c58-nbfvs"
Jul 01 20:53:38.397201 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:38.397174    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnzt2\" (UniqueName: \"kubernetes.io/projected/aa1a579e-8c01-4f02-8c43-757daf8ddb28-kube-api-access-hnzt2\") pod \"cert-manager-operator-controller-manager-5b66478c58-nbfvs\" (UID: \"aa1a579e-8c01-4f02-8c43-757daf8ddb28\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-5b66478c58-nbfvs"
Jul 01 20:53:38.436699 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:38.436674    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-5b66478c58-nbfvs"
Jul 01 20:53:38.574313 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:38.574287    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-5b66478c58-nbfvs"]
Jul 01 20:53:38.577151 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:53:38.577126    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa1a579e_8c01_4f02_8c43_757daf8ddb28.slice/crio-45f666bf7c905e4e51c3c4f0613a5442bcc939c3809e4370d0558dfa38b90e41 WatchSource:0}: Error finding container 45f666bf7c905e4e51c3c4f0613a5442bcc939c3809e4370d0558dfa38b90e41: Status 404 returned error can't find the container with id 45f666bf7c905e4e51c3c4f0613a5442bcc939c3809e4370d0558dfa38b90e41
Jul 01 20:53:38.579810 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:38.579794    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 01 20:53:39.503173 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:39.503137    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-5b66478c58-nbfvs" event={"ID":"aa1a579e-8c01-4f02-8c43-757daf8ddb28","Type":"ContainerStarted","Data":"45f666bf7c905e4e51c3c4f0613a5442bcc939c3809e4370d0558dfa38b90e41"}
Jul 01 20:53:42.515876 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:42.515790    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-5b66478c58-nbfvs" event={"ID":"aa1a579e-8c01-4f02-8c43-757daf8ddb28","Type":"ContainerStarted","Data":"06b61e4407ab8162a0a968ed8f6ffe4bcef846851b2db0c15ceb6aedd0577554"}
Jul 01 20:53:42.538869 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:42.538817    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-5b66478c58-nbfvs" podStartSLOduration=1.611806943 podStartE2EDuration="4.538800153s" totalImagesPullingTime="2.92699321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:53:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:53:42.537362905 +0000 UTC m=+441.049838092" watchObservedRunningTime="2026-07-01 20:53:42.538800153 +0000 UTC m=+441.051275343"
Jul 01 20:53:45.030149 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:45.030097    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-g47v7"]
Jul 01 20:53:45.033399 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:45.033366    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-g47v7"
Jul 01 20:53:45.036518 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:45.036498    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 01 20:53:45.036607 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:45.036542    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 01 20:53:45.036607 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:45.036587    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-bnttp\""
Jul 01 20:53:45.043404 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:45.043382    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-g47v7"]
Jul 01 20:53:45.133732 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:45.133703    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtzgd\" (UniqueName: \"kubernetes.io/projected/12f3eedf-37e5-4fa0-ad7c-3066d8b504c4-kube-api-access-mtzgd\") pod \"cert-manager-webhook-7dc76cbb94-g47v7\" (UID: \"12f3eedf-37e5-4fa0-ad7c-3066d8b504c4\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-g47v7"
Jul 01 20:53:45.133829 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:45.133742    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12f3eedf-37e5-4fa0-ad7c-3066d8b504c4-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-g47v7\" (UID: \"12f3eedf-37e5-4fa0-ad7c-3066d8b504c4\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-g47v7"
Jul 01 20:53:45.234892 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:45.234871    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtzgd\" (UniqueName: \"kubernetes.io/projected/12f3eedf-37e5-4fa0-ad7c-3066d8b504c4-kube-api-access-mtzgd\") pod \"cert-manager-webhook-7dc76cbb94-g47v7\" (UID: \"12f3eedf-37e5-4fa0-ad7c-3066d8b504c4\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-g47v7"
Jul 01 20:53:45.234978 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:45.234900    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12f3eedf-37e5-4fa0-ad7c-3066d8b504c4-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-g47v7\" (UID: \"12f3eedf-37e5-4fa0-ad7c-3066d8b504c4\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-g47v7"
Jul 01 20:53:45.246076 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:45.246048    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12f3eedf-37e5-4fa0-ad7c-3066d8b504c4-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-g47v7\" (UID: \"12f3eedf-37e5-4fa0-ad7c-3066d8b504c4\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-g47v7"
Jul 01 20:53:45.246219 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:45.246198    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtzgd\" (UniqueName: \"kubernetes.io/projected/12f3eedf-37e5-4fa0-ad7c-3066d8b504c4-kube-api-access-mtzgd\") pod \"cert-manager-webhook-7dc76cbb94-g47v7\" (UID: \"12f3eedf-37e5-4fa0-ad7c-3066d8b504c4\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-g47v7"
Jul 01 20:53:45.353458 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:45.353395    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-g47v7"
Jul 01 20:53:45.520426 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:45.520401    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-g47v7"]
Jul 01 20:53:45.522954 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:53:45.522918    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod12f3eedf_37e5_4fa0_ad7c_3066d8b504c4.slice/crio-f0c6016b95918a05588b549a3eeb521726b15357037911da7beefbfcd5a2d170 WatchSource:0}: Error finding container f0c6016b95918a05588b549a3eeb521726b15357037911da7beefbfcd5a2d170: Status 404 returned error can't find the container with id f0c6016b95918a05588b549a3eeb521726b15357037911da7beefbfcd5a2d170
Jul 01 20:53:46.530877 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:46.530844    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-g47v7" event={"ID":"12f3eedf-37e5-4fa0-ad7c-3066d8b504c4","Type":"ContainerStarted","Data":"f0c6016b95918a05588b549a3eeb521726b15357037911da7beefbfcd5a2d170"}
Jul 01 20:53:49.543391 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:49.543351    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-g47v7" event={"ID":"12f3eedf-37e5-4fa0-ad7c-3066d8b504c4","Type":"ContainerStarted","Data":"cd553a145dcd168594022792102cfa928de4cbb4aff6ca7501544994c34ace6c"}
Jul 01 20:53:49.543752 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:49.543413    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-g47v7"
Jul 01 20:53:49.560353 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:49.560314    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-7dc76cbb94-g47v7" podStartSLOduration=1.175930164 podStartE2EDuration="4.560299312s" totalImagesPullingTime="3.384369148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:53:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:53:49.559863175 +0000 UTC m=+448.072338366" watchObservedRunningTime="2026-07-01 20:53:49.560299312 +0000 UTC m=+448.072774501"
Jul 01 20:53:49.821983 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:49.821913    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5d99449ffd-g7sjg"]
Jul 01 20:53:49.824690 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:49.824674    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5d99449ffd-g7sjg"
Jul 01 20:53:49.827497 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:49.827479    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-clbpv\""
Jul 01 20:53:49.838781 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:49.838759    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5d99449ffd-g7sjg"]
Jul 01 20:53:49.969424 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:49.969399    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1d315c69-6fca-40ac-9159-d126891e65f4-bound-sa-token\") pod \"cert-manager-cainjector-5d99449ffd-g7sjg\" (UID: \"1d315c69-6fca-40ac-9159-d126891e65f4\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-g7sjg"
Jul 01 20:53:49.969544 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:49.969452    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6vsz\" (UniqueName: \"kubernetes.io/projected/1d315c69-6fca-40ac-9159-d126891e65f4-kube-api-access-m6vsz\") pod \"cert-manager-cainjector-5d99449ffd-g7sjg\" (UID: \"1d315c69-6fca-40ac-9159-d126891e65f4\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-g7sjg"
Jul 01 20:53:50.070044 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:50.070016    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1d315c69-6fca-40ac-9159-d126891e65f4-bound-sa-token\") pod \"cert-manager-cainjector-5d99449ffd-g7sjg\" (UID: \"1d315c69-6fca-40ac-9159-d126891e65f4\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-g7sjg"
Jul 01 20:53:50.070160 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:50.070076    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6vsz\" (UniqueName: \"kubernetes.io/projected/1d315c69-6fca-40ac-9159-d126891e65f4-kube-api-access-m6vsz\") pod \"cert-manager-cainjector-5d99449ffd-g7sjg\" (UID: \"1d315c69-6fca-40ac-9159-d126891e65f4\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-g7sjg"
Jul 01 20:53:50.083970 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:50.083909    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1d315c69-6fca-40ac-9159-d126891e65f4-bound-sa-token\") pod \"cert-manager-cainjector-5d99449ffd-g7sjg\" (UID: \"1d315c69-6fca-40ac-9159-d126891e65f4\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-g7sjg"
Jul 01 20:53:50.084158 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:50.084140    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6vsz\" (UniqueName: \"kubernetes.io/projected/1d315c69-6fca-40ac-9159-d126891e65f4-kube-api-access-m6vsz\") pod \"cert-manager-cainjector-5d99449ffd-g7sjg\" (UID: \"1d315c69-6fca-40ac-9159-d126891e65f4\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-g7sjg"
Jul 01 20:53:50.134553 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:50.134522    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5d99449ffd-g7sjg"
Jul 01 20:53:50.256352 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:50.256262    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5d99449ffd-g7sjg"]
Jul 01 20:53:50.258567 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:53:50.258541    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1d315c69_6fca_40ac_9159_d126891e65f4.slice/crio-441a2ee2c8fcf960aacbd5b50eefc4dc488f6645750c2a34b283673585b03d6b WatchSource:0}: Error finding container 441a2ee2c8fcf960aacbd5b50eefc4dc488f6645750c2a34b283673585b03d6b: Status 404 returned error can't find the container with id 441a2ee2c8fcf960aacbd5b50eefc4dc488f6645750c2a34b283673585b03d6b
Jul 01 20:53:50.548210 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:50.548176    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5d99449ffd-g7sjg" event={"ID":"1d315c69-6fca-40ac-9159-d126891e65f4","Type":"ContainerStarted","Data":"a2990eac97f71ec22ff479f382eb0b9557f1964dd9f71e71bad2df3e4991cf09"}
Jul 01 20:53:50.548676 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:50.548221    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5d99449ffd-g7sjg" event={"ID":"1d315c69-6fca-40ac-9159-d126891e65f4","Type":"ContainerStarted","Data":"441a2ee2c8fcf960aacbd5b50eefc4dc488f6645750c2a34b283673585b03d6b"}
Jul 01 20:53:50.566923 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:50.566880    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5d99449ffd-g7sjg" podStartSLOduration=1.566862552 podStartE2EDuration="1.566862552s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:53:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:53:50.565490265 +0000 UTC m=+449.077965457" watchObservedRunningTime="2026-07-01 20:53:50.566862552 +0000 UTC m=+449.079337883"
Jul 01 20:53:55.550948 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:53:55.550907    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-g47v7"
Jul 01 20:54:00.575356 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:00.575310    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ktmkv"]
Jul 01 20:54:00.582297 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:00.582273    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ktmkv"
Jul 01 20:54:00.585254 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:00.585229    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 01 20:54:00.585417 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:00.585298    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 01 20:54:00.586544 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:00.586525    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-529nn\""
Jul 01 20:54:00.590187 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:00.590159    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ktmkv"]
Jul 01 20:54:00.644754 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:00.644732    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mh25k\" (UniqueName: \"kubernetes.io/projected/f3e9046b-d3fa-41ab-8a22-d3ea9adbdc4e-kube-api-access-mh25k\") pod \"openshift-lws-operator-fd8ccff4c-ktmkv\" (UID: \"f3e9046b-d3fa-41ab-8a22-d3ea9adbdc4e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ktmkv"
Jul 01 20:54:00.644866 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:00.644780    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f3e9046b-d3fa-41ab-8a22-d3ea9adbdc4e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-ktmkv\" (UID: \"f3e9046b-d3fa-41ab-8a22-d3ea9adbdc4e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ktmkv"
Jul 01 20:54:00.745612 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:00.745588    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mh25k\" (UniqueName: \"kubernetes.io/projected/f3e9046b-d3fa-41ab-8a22-d3ea9adbdc4e-kube-api-access-mh25k\") pod \"openshift-lws-operator-fd8ccff4c-ktmkv\" (UID: \"f3e9046b-d3fa-41ab-8a22-d3ea9adbdc4e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ktmkv"
Jul 01 20:54:00.745728 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:00.745652    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f3e9046b-d3fa-41ab-8a22-d3ea9adbdc4e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-ktmkv\" (UID: \"f3e9046b-d3fa-41ab-8a22-d3ea9adbdc4e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ktmkv"
Jul 01 20:54:00.745945 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:00.745929    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f3e9046b-d3fa-41ab-8a22-d3ea9adbdc4e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-ktmkv\" (UID: \"f3e9046b-d3fa-41ab-8a22-d3ea9adbdc4e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ktmkv"
Jul 01 20:54:00.755516 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:00.755484    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh25k\" (UniqueName: \"kubernetes.io/projected/f3e9046b-d3fa-41ab-8a22-d3ea9adbdc4e-kube-api-access-mh25k\") pod \"openshift-lws-operator-fd8ccff4c-ktmkv\" (UID: \"f3e9046b-d3fa-41ab-8a22-d3ea9adbdc4e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ktmkv"
Jul 01 20:54:00.893818 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:00.893765    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ktmkv"
Jul 01 20:54:01.024118 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:01.024096    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ktmkv"]
Jul 01 20:54:01.026150 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:54:01.026124    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3e9046b_d3fa_41ab_8a22_d3ea9adbdc4e.slice/crio-5a7a38b5300391fd8ceab74d656f626d7e55565dc6e6a0b4225a6e2bdf68b056 WatchSource:0}: Error finding container 5a7a38b5300391fd8ceab74d656f626d7e55565dc6e6a0b4225a6e2bdf68b056: Status 404 returned error can't find the container with id 5a7a38b5300391fd8ceab74d656f626d7e55565dc6e6a0b4225a6e2bdf68b056
Jul 01 20:54:01.594105 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:01.594035    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ktmkv" event={"ID":"f3e9046b-d3fa-41ab-8a22-d3ea9adbdc4e","Type":"ContainerStarted","Data":"5a7a38b5300391fd8ceab74d656f626d7e55565dc6e6a0b4225a6e2bdf68b056"}
Jul 01 20:54:03.995284 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:03.995253    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-59c77684cc-m4zj8"]
Jul 01 20:54:03.998995 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:03.998967    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-59c77684cc-m4zj8"
Jul 01 20:54:04.001733 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:04.001711    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-cs98d\""
Jul 01 20:54:04.007812 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:04.007776    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-59c77684cc-m4zj8"]
Jul 01 20:54:04.069834 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:04.069804    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2c4b037e-3004-4cfd-971d-896b92edfdd5-bound-sa-token\") pod \"cert-manager-59c77684cc-m4zj8\" (UID: \"2c4b037e-3004-4cfd-971d-896b92edfdd5\") " pod="cert-manager/cert-manager-59c77684cc-m4zj8"
Jul 01 20:54:04.070002 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:04.069847    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xphx\" (UniqueName: \"kubernetes.io/projected/2c4b037e-3004-4cfd-971d-896b92edfdd5-kube-api-access-7xphx\") pod \"cert-manager-59c77684cc-m4zj8\" (UID: \"2c4b037e-3004-4cfd-971d-896b92edfdd5\") " pod="cert-manager/cert-manager-59c77684cc-m4zj8"
Jul 01 20:54:04.170765 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:04.170740    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2c4b037e-3004-4cfd-971d-896b92edfdd5-bound-sa-token\") pod \"cert-manager-59c77684cc-m4zj8\" (UID: \"2c4b037e-3004-4cfd-971d-896b92edfdd5\") " pod="cert-manager/cert-manager-59c77684cc-m4zj8"
Jul 01 20:54:04.170923 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:04.170772    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xphx\" (UniqueName: \"kubernetes.io/projected/2c4b037e-3004-4cfd-971d-896b92edfdd5-kube-api-access-7xphx\") pod \"cert-manager-59c77684cc-m4zj8\" (UID: \"2c4b037e-3004-4cfd-971d-896b92edfdd5\") " pod="cert-manager/cert-manager-59c77684cc-m4zj8"
Jul 01 20:54:04.179350 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:04.179315    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2c4b037e-3004-4cfd-971d-896b92edfdd5-bound-sa-token\") pod \"cert-manager-59c77684cc-m4zj8\" (UID: \"2c4b037e-3004-4cfd-971d-896b92edfdd5\") " pod="cert-manager/cert-manager-59c77684cc-m4zj8"
Jul 01 20:54:04.179455 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:04.179440    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xphx\" (UniqueName: \"kubernetes.io/projected/2c4b037e-3004-4cfd-971d-896b92edfdd5-kube-api-access-7xphx\") pod \"cert-manager-59c77684cc-m4zj8\" (UID: \"2c4b037e-3004-4cfd-971d-896b92edfdd5\") " pod="cert-manager/cert-manager-59c77684cc-m4zj8"
Jul 01 20:54:04.310642 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:04.310597    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-59c77684cc-m4zj8"
Jul 01 20:54:04.452750 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:04.452724    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-59c77684cc-m4zj8"]
Jul 01 20:54:04.455127 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:54:04.455096    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c4b037e_3004_4cfd_971d_896b92edfdd5.slice/crio-01b37f4f8e2bce5bac714683671a2fea7676607bcd50544da6ae8e3ecf61b498 WatchSource:0}: Error finding container 01b37f4f8e2bce5bac714683671a2fea7676607bcd50544da6ae8e3ecf61b498: Status 404 returned error can't find the container with id 01b37f4f8e2bce5bac714683671a2fea7676607bcd50544da6ae8e3ecf61b498
Jul 01 20:54:04.610580 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:04.610549    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ktmkv" event={"ID":"f3e9046b-d3fa-41ab-8a22-d3ea9adbdc4e","Type":"ContainerStarted","Data":"3889c070c5005cbb2645ca0369bb57497a89f9b5f575c7135e1527a2c47abcab"}
Jul 01 20:54:04.612073 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:04.612047    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-59c77684cc-m4zj8" event={"ID":"2c4b037e-3004-4cfd-971d-896b92edfdd5","Type":"ContainerStarted","Data":"3ecd82b14ff2858fcbe08403a077ef04d0af96717db8db885c58fa58495bd959"}
Jul 01 20:54:04.612200 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:04.612081    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-59c77684cc-m4zj8" event={"ID":"2c4b037e-3004-4cfd-971d-896b92edfdd5","Type":"ContainerStarted","Data":"01b37f4f8e2bce5bac714683671a2fea7676607bcd50544da6ae8e3ecf61b498"}
Jul 01 20:54:04.629097 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:04.629015    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ktmkv" podStartSLOduration=1.446301354 podStartE2EDuration="4.628998676s" totalImagesPullingTime="3.182697322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:54:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:54:04.628417687 +0000 UTC m=+463.140892877" watchObservedRunningTime="2026-07-01 20:54:04.628998676 +0000 UTC m=+463.141473868"
Jul 01 20:54:04.645387 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:04.645337    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-59c77684cc-m4zj8" podStartSLOduration=1.645323691 podStartE2EDuration="1.645323691s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:54:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:54:04.643657504 +0000 UTC m=+463.156132696" watchObservedRunningTime="2026-07-01 20:54:04.645323691 +0000 UTC m=+463.157798881"
Jul 01 20:54:21.405940 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.405899    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"]
Jul 01 20:54:21.409888 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.409860    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:54:21.412723 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.412694    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 01 20:54:21.412879 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.412755    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 01 20:54:21.413010 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.412988    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 01 20:54:21.413108 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.413046    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-default-dockercfg-hzvdm\""
Jul 01 20:54:21.422009 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.421983    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"]
Jul 01 20:54:21.490243 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.490218    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbn5j\" (UniqueName: \"kubernetes.io/projected/f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13-kube-api-access-hbn5j\") pod \"openshift-ai-inference-openshift-default-655f7c58b4-57g8x\" (UID: \"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:54:21.490344 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.490246    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13-workload-certs\") pod \"openshift-ai-inference-openshift-default-655f7c58b4-57g8x\" (UID: \"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:54:21.490344 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.490271    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13-workload-socket\") pod \"openshift-ai-inference-openshift-default-655f7c58b4-57g8x\" (UID: \"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:54:21.490344 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.490288    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-655f7c58b4-57g8x\" (UID: \"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:54:21.490463 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.490357    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13-istio-data\") pod \"openshift-ai-inference-openshift-default-655f7c58b4-57g8x\" (UID: \"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:54:21.490463 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.490398    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13-istio-envoy\") pod \"openshift-ai-inference-openshift-default-655f7c58b4-57g8x\" (UID: \"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:54:21.490463 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.490422    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13-credential-socket\") pod \"openshift-ai-inference-openshift-default-655f7c58b4-57g8x\" (UID: \"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:54:21.490463 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.490440    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-655f7c58b4-57g8x\" (UID: \"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:54:21.490603 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.490463    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13-istio-token\") pod \"openshift-ai-inference-openshift-default-655f7c58b4-57g8x\" (UID: \"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:54:21.591042 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.591020    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13-istio-data\") pod \"openshift-ai-inference-openshift-default-655f7c58b4-57g8x\" (UID: \"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:54:21.591134 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.591053    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13-istio-envoy\") pod \"openshift-ai-inference-openshift-default-655f7c58b4-57g8x\" (UID: \"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:54:21.591134 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.591074    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13-credential-socket\") pod \"openshift-ai-inference-openshift-default-655f7c58b4-57g8x\" (UID: \"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:54:21.591134 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.591092    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-655f7c58b4-57g8x\" (UID: \"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:54:21.591294 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.591238    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13-istio-token\") pod \"openshift-ai-inference-openshift-default-655f7c58b4-57g8x\" (UID: \"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:54:21.591349 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.591310    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbn5j\" (UniqueName: \"kubernetes.io/projected/f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13-kube-api-access-hbn5j\") pod \"openshift-ai-inference-openshift-default-655f7c58b4-57g8x\" (UID: \"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:54:21.591399 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.591348    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13-workload-certs\") pod \"openshift-ai-inference-openshift-default-655f7c58b4-57g8x\" (UID: \"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:54:21.591399 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.591379    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13-istio-data\") pod \"openshift-ai-inference-openshift-default-655f7c58b4-57g8x\" (UID: \"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:54:21.591501 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.591425    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13-credential-socket\") pod \"openshift-ai-inference-openshift-default-655f7c58b4-57g8x\" (UID: \"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:54:21.591556 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.591509    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13-workload-socket\") pod \"openshift-ai-inference-openshift-default-655f7c58b4-57g8x\" (UID: \"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:54:21.591609 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.591560    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-655f7c58b4-57g8x\" (UID: \"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:54:21.591703 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.591675    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13-workload-certs\") pod \"openshift-ai-inference-openshift-default-655f7c58b4-57g8x\" (UID: \"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:54:21.591835 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.591815    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13-workload-socket\") pod \"openshift-ai-inference-openshift-default-655f7c58b4-57g8x\" (UID: \"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:54:21.592134 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.592117    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-655f7c58b4-57g8x\" (UID: \"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:54:21.594036 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.594012    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13-istio-envoy\") pod \"openshift-ai-inference-openshift-default-655f7c58b4-57g8x\" (UID: \"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:54:21.594193 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.594177    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-655f7c58b4-57g8x\" (UID: \"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:54:21.600898 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.600875    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbn5j\" (UniqueName: \"kubernetes.io/projected/f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13-kube-api-access-hbn5j\") pod \"openshift-ai-inference-openshift-default-655f7c58b4-57g8x\" (UID: \"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:54:21.601031 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.601014    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13-istio-token\") pod \"openshift-ai-inference-openshift-default-655f7c58b4-57g8x\" (UID: \"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:54:21.722452 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.722387    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:54:21.855302 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:21.855280    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"]
Jul 01 20:54:21.857314 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:54:21.857290    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1d4abbb_fa80_4dff_8760_4c5e5f0d7c13.slice/crio-46ae75471bde1ec46ba77746580a19b97159ef188c7444f165c576969a13ec9a WatchSource:0}: Error finding container 46ae75471bde1ec46ba77746580a19b97159ef188c7444f165c576969a13ec9a: Status 404 returned error can't find the container with id 46ae75471bde1ec46ba77746580a19b97159ef188c7444f165c576969a13ec9a
Jul 01 20:54:22.693856 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:22.693820    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" event={"ID":"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13","Type":"ContainerStarted","Data":"46ae75471bde1ec46ba77746580a19b97159ef188c7444f165c576969a13ec9a"}
Jul 01 20:54:26.882876 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:26.882836    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 01 20:54:26.883113 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:26.882940    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 01 20:54:26.883113 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:26.882973    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 01 20:54:27.720953 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:27.720914    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" event={"ID":"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13","Type":"ContainerStarted","Data":"8b32b47e6634253200d19f5c9f1d0c62cfe74575698b0f0bc4576d9719693d6d"}
Jul 01 20:54:27.723307 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:27.723276    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:54:27.727873 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:27.727851    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:54:27.756298 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:27.756252    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" podStartSLOduration=1.732848355 podStartE2EDuration="6.756239943s" totalImagesPullingTime="5.023391588s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:54:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:54:27.753736557 +0000 UTC m=+486.266211746" watchObservedRunningTime="2026-07-01 20:54:27.756239943 +0000 UTC m=+486.268715132"
Jul 01 20:54:28.724765 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:28.724742    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:54:28.725747 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:28.725728    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:54:30.035961 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:30.035934    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-55f5fcc78b-5zh2r"]
Jul 01 20:54:30.039390 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:30.039366    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-55f5fcc78b-5zh2r"
Jul 01 20:54:30.043873 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:30.043853    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-dkqv8\""
Jul 01 20:54:30.043873 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:30.043863    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 01 20:54:30.044035 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:30.043875    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 01 20:54:30.044035 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:30.043884    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 01 20:54:30.056358 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:30.056330    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-55f5fcc78b-5zh2r"]
Jul 01 20:54:30.162386 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:30.162366    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/91ab49c8-06a7-44a4-9c12-25f87767dca3-manager-config\") pod \"lws-controller-manager-55f5fcc78b-5zh2r\" (UID: \"91ab49c8-06a7-44a4-9c12-25f87767dca3\") " pod="openshift-lws-operator/lws-controller-manager-55f5fcc78b-5zh2r"
Jul 01 20:54:30.162487 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:30.162404    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mvpt\" (UniqueName: \"kubernetes.io/projected/91ab49c8-06a7-44a4-9c12-25f87767dca3-kube-api-access-6mvpt\") pod \"lws-controller-manager-55f5fcc78b-5zh2r\" (UID: \"91ab49c8-06a7-44a4-9c12-25f87767dca3\") " pod="openshift-lws-operator/lws-controller-manager-55f5fcc78b-5zh2r"
Jul 01 20:54:30.162487 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:30.162423    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/91ab49c8-06a7-44a4-9c12-25f87767dca3-cert\") pod \"lws-controller-manager-55f5fcc78b-5zh2r\" (UID: \"91ab49c8-06a7-44a4-9c12-25f87767dca3\") " pod="openshift-lws-operator/lws-controller-manager-55f5fcc78b-5zh2r"
Jul 01 20:54:30.162579 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:30.162480    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/91ab49c8-06a7-44a4-9c12-25f87767dca3-metrics-cert\") pod \"lws-controller-manager-55f5fcc78b-5zh2r\" (UID: \"91ab49c8-06a7-44a4-9c12-25f87767dca3\") " pod="openshift-lws-operator/lws-controller-manager-55f5fcc78b-5zh2r"
Jul 01 20:54:30.263604 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:30.263573    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/91ab49c8-06a7-44a4-9c12-25f87767dca3-manager-config\") pod \"lws-controller-manager-55f5fcc78b-5zh2r\" (UID: \"91ab49c8-06a7-44a4-9c12-25f87767dca3\") " pod="openshift-lws-operator/lws-controller-manager-55f5fcc78b-5zh2r"
Jul 01 20:54:30.263768 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:30.263657    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mvpt\" (UniqueName: \"kubernetes.io/projected/91ab49c8-06a7-44a4-9c12-25f87767dca3-kube-api-access-6mvpt\") pod \"lws-controller-manager-55f5fcc78b-5zh2r\" (UID: \"91ab49c8-06a7-44a4-9c12-25f87767dca3\") " pod="openshift-lws-operator/lws-controller-manager-55f5fcc78b-5zh2r"
Jul 01 20:54:30.263768 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:30.263685    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/91ab49c8-06a7-44a4-9c12-25f87767dca3-cert\") pod \"lws-controller-manager-55f5fcc78b-5zh2r\" (UID: \"91ab49c8-06a7-44a4-9c12-25f87767dca3\") " pod="openshift-lws-operator/lws-controller-manager-55f5fcc78b-5zh2r"
Jul 01 20:54:30.263768 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:30.263723    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/91ab49c8-06a7-44a4-9c12-25f87767dca3-metrics-cert\") pod \"lws-controller-manager-55f5fcc78b-5zh2r\" (UID: \"91ab49c8-06a7-44a4-9c12-25f87767dca3\") " pod="openshift-lws-operator/lws-controller-manager-55f5fcc78b-5zh2r"
Jul 01 20:54:30.264327 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:30.264260    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/91ab49c8-06a7-44a4-9c12-25f87767dca3-manager-config\") pod \"lws-controller-manager-55f5fcc78b-5zh2r\" (UID: \"91ab49c8-06a7-44a4-9c12-25f87767dca3\") " pod="openshift-lws-operator/lws-controller-manager-55f5fcc78b-5zh2r"
Jul 01 20:54:30.266227 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:30.266207    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/91ab49c8-06a7-44a4-9c12-25f87767dca3-metrics-cert\") pod \"lws-controller-manager-55f5fcc78b-5zh2r\" (UID: \"91ab49c8-06a7-44a4-9c12-25f87767dca3\") " pod="openshift-lws-operator/lws-controller-manager-55f5fcc78b-5zh2r"
Jul 01 20:54:30.266291 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:30.266268    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/91ab49c8-06a7-44a4-9c12-25f87767dca3-cert\") pod \"lws-controller-manager-55f5fcc78b-5zh2r\" (UID: \"91ab49c8-06a7-44a4-9c12-25f87767dca3\") " pod="openshift-lws-operator/lws-controller-manager-55f5fcc78b-5zh2r"
Jul 01 20:54:30.274043 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:30.274015    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mvpt\" (UniqueName: \"kubernetes.io/projected/91ab49c8-06a7-44a4-9c12-25f87767dca3-kube-api-access-6mvpt\") pod \"lws-controller-manager-55f5fcc78b-5zh2r\" (UID: \"91ab49c8-06a7-44a4-9c12-25f87767dca3\") " pod="openshift-lws-operator/lws-controller-manager-55f5fcc78b-5zh2r"
Jul 01 20:54:30.349067 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:30.349021    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-55f5fcc78b-5zh2r"
Jul 01 20:54:30.485130 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:30.485104    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-55f5fcc78b-5zh2r"]
Jul 01 20:54:30.487507 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:54:30.487483    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91ab49c8_06a7_44a4_9c12_25f87767dca3.slice/crio-50982fe7957d297ce57859a32e2e08c43f643e46f2062d523cb476a97e9def05 WatchSource:0}: Error finding container 50982fe7957d297ce57859a32e2e08c43f643e46f2062d523cb476a97e9def05: Status 404 returned error can't find the container with id 50982fe7957d297ce57859a32e2e08c43f643e46f2062d523cb476a97e9def05
Jul 01 20:54:30.735292 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:30.735261    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-55f5fcc78b-5zh2r" event={"ID":"91ab49c8-06a7-44a4-9c12-25f87767dca3","Type":"ContainerStarted","Data":"50982fe7957d297ce57859a32e2e08c43f643e46f2062d523cb476a97e9def05"}
Jul 01 20:54:32.745936 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:32.745902    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-55f5fcc78b-5zh2r" event={"ID":"91ab49c8-06a7-44a4-9c12-25f87767dca3","Type":"ContainerStarted","Data":"3f173538c74b6b99969b62cf2c58cf35264373a397f32cd4df00281d4f71c01d"}
Jul 01 20:54:32.746292 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:32.746014    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-55f5fcc78b-5zh2r"
Jul 01 20:54:32.763585 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:32.763541    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-55f5fcc78b-5zh2r" podStartSLOduration=0.938933281 podStartE2EDuration="2.763525589s" totalImagesPullingTime="1.824592308s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:54:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:54:32.762187099 +0000 UTC m=+491.274662290" watchObservedRunningTime="2026-07-01 20:54:32.763525589 +0000 UTC m=+491.276000779"
Jul 01 20:54:43.752157 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:43.752128    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-55f5fcc78b-5zh2r"
Jul 01 20:54:51.530218 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:51.530169    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65b49595d7-c59jh"]
Jul 01 20:54:51.533573 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:51.533555    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-c59jh"
Jul 01 20:54:51.536283 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:51.536256    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 01 20:54:51.536283 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:51.536271    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-r422g\""
Jul 01 20:54:51.536490 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:51.536401    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 01 20:54:51.537400 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:51.537380    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 01 20:54:51.551262 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:51.551238    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65b49595d7-c59jh"]
Jul 01 20:54:51.626312 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:51.626288    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nz4tc\" (UniqueName: \"kubernetes.io/projected/a2454155-5137-4ea9-9b3a-6656ed62603c-kube-api-access-nz4tc\") pod \"dns-operator-controller-manager-65b49595d7-c59jh\" (UID: \"a2454155-5137-4ea9-9b3a-6656ed62603c\") " pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-c59jh"
Jul 01 20:54:51.727264 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:51.727228    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz4tc\" (UniqueName: \"kubernetes.io/projected/a2454155-5137-4ea9-9b3a-6656ed62603c-kube-api-access-nz4tc\") pod \"dns-operator-controller-manager-65b49595d7-c59jh\" (UID: \"a2454155-5137-4ea9-9b3a-6656ed62603c\") " pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-c59jh"
Jul 01 20:54:51.739253 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:51.739223    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nz4tc\" (UniqueName: \"kubernetes.io/projected/a2454155-5137-4ea9-9b3a-6656ed62603c-kube-api-access-nz4tc\") pod \"dns-operator-controller-manager-65b49595d7-c59jh\" (UID: \"a2454155-5137-4ea9-9b3a-6656ed62603c\") " pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-c59jh"
Jul 01 20:54:51.844534 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:51.844481    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-c59jh"
Jul 01 20:54:51.980128 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:51.980108    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65b49595d7-c59jh"]
Jul 01 20:54:51.982138 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:54:51.982112    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2454155_5137_4ea9_9b3a_6656ed62603c.slice/crio-d8bb2894f3ae0c060256e98d0f61091c9c8dd7ee068dc1561a00417f76539cac WatchSource:0}: Error finding container d8bb2894f3ae0c060256e98d0f61091c9c8dd7ee068dc1561a00417f76539cac: Status 404 returned error can't find the container with id d8bb2894f3ae0c060256e98d0f61091c9c8dd7ee068dc1561a00417f76539cac
Jul 01 20:54:52.840135 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:52.840098    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-c59jh" event={"ID":"a2454155-5137-4ea9-9b3a-6656ed62603c","Type":"ContainerStarted","Data":"d8bb2894f3ae0c060256e98d0f61091c9c8dd7ee068dc1561a00417f76539cac"}
Jul 01 20:54:54.098483 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:54.098438    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-pgv8h"]
Jul 01 20:54:54.102189 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:54.102077    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-pgv8h"
Jul 01 20:54:54.104918 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:54.104766    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 01 20:54:54.104918 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:54.104822    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-xp5xs\""
Jul 01 20:54:54.104918 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:54.104766    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 01 20:54:54.112856 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:54.112808    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-pgv8h"]
Jul 01 20:54:54.246215 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:54.246183    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b7720a2b-19f8-4d1a-b0f6-a481c4dc0f62-nginx-conf\") pod \"kuadrant-console-plugin-85fd4b8b9d-pgv8h\" (UID: \"b7720a2b-19f8-4d1a-b0f6-a481c4dc0f62\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-pgv8h"
Jul 01 20:54:54.246402 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:54.246322    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zt7p\" (UniqueName: \"kubernetes.io/projected/b7720a2b-19f8-4d1a-b0f6-a481c4dc0f62-kube-api-access-2zt7p\") pod \"kuadrant-console-plugin-85fd4b8b9d-pgv8h\" (UID: \"b7720a2b-19f8-4d1a-b0f6-a481c4dc0f62\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-pgv8h"
Jul 01 20:54:54.246402 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:54.246359    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b7720a2b-19f8-4d1a-b0f6-a481c4dc0f62-plugin-serving-cert\") pod \"kuadrant-console-plugin-85fd4b8b9d-pgv8h\" (UID: \"b7720a2b-19f8-4d1a-b0f6-a481c4dc0f62\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-pgv8h"
Jul 01 20:54:54.347688 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:54.347652    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b7720a2b-19f8-4d1a-b0f6-a481c4dc0f62-nginx-conf\") pod \"kuadrant-console-plugin-85fd4b8b9d-pgv8h\" (UID: \"b7720a2b-19f8-4d1a-b0f6-a481c4dc0f62\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-pgv8h"
Jul 01 20:54:54.347900 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:54.347780    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zt7p\" (UniqueName: \"kubernetes.io/projected/b7720a2b-19f8-4d1a-b0f6-a481c4dc0f62-kube-api-access-2zt7p\") pod \"kuadrant-console-plugin-85fd4b8b9d-pgv8h\" (UID: \"b7720a2b-19f8-4d1a-b0f6-a481c4dc0f62\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-pgv8h"
Jul 01 20:54:54.347900 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:54.347815    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b7720a2b-19f8-4d1a-b0f6-a481c4dc0f62-plugin-serving-cert\") pod \"kuadrant-console-plugin-85fd4b8b9d-pgv8h\" (UID: \"b7720a2b-19f8-4d1a-b0f6-a481c4dc0f62\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-pgv8h"
Jul 01 20:54:54.349214 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:54.349128    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b7720a2b-19f8-4d1a-b0f6-a481c4dc0f62-nginx-conf\") pod \"kuadrant-console-plugin-85fd4b8b9d-pgv8h\" (UID: \"b7720a2b-19f8-4d1a-b0f6-a481c4dc0f62\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-pgv8h"
Jul 01 20:54:54.351004 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:54.350976    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b7720a2b-19f8-4d1a-b0f6-a481c4dc0f62-plugin-serving-cert\") pod \"kuadrant-console-plugin-85fd4b8b9d-pgv8h\" (UID: \"b7720a2b-19f8-4d1a-b0f6-a481c4dc0f62\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-pgv8h"
Jul 01 20:54:54.357393 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:54.357360    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zt7p\" (UniqueName: \"kubernetes.io/projected/b7720a2b-19f8-4d1a-b0f6-a481c4dc0f62-kube-api-access-2zt7p\") pod \"kuadrant-console-plugin-85fd4b8b9d-pgv8h\" (UID: \"b7720a2b-19f8-4d1a-b0f6-a481c4dc0f62\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-pgv8h"
Jul 01 20:54:54.416492 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:54.416252    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-pgv8h"
Jul 01 20:54:54.582003 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:54.581973    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-pgv8h"]
Jul 01 20:54:54.583948 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:54:54.583920    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb7720a2b_19f8_4d1a_b0f6_a481c4dc0f62.slice/crio-79f32c0138979493d51ca5c481d99d18425ab285bc53d5c9de7ce2926f8be8d5 WatchSource:0}: Error finding container 79f32c0138979493d51ca5c481d99d18425ab285bc53d5c9de7ce2926f8be8d5: Status 404 returned error can't find the container with id 79f32c0138979493d51ca5c481d99d18425ab285bc53d5c9de7ce2926f8be8d5
Jul 01 20:54:54.851811 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:54.851774    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-pgv8h" event={"ID":"b7720a2b-19f8-4d1a-b0f6-a481c4dc0f62","Type":"ContainerStarted","Data":"79f32c0138979493d51ca5c481d99d18425ab285bc53d5c9de7ce2926f8be8d5"}
Jul 01 20:54:56.864950 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:56.864909    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-c59jh" event={"ID":"a2454155-5137-4ea9-9b3a-6656ed62603c","Type":"ContainerStarted","Data":"8f6a0c44c52f38f6b6ffdc9fdca68781f86581041328d9828f424c8cd36a87f9"}
Jul 01 20:54:56.865340 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:56.864982    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-c59jh"
Jul 01 20:54:56.887855 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:56.887802    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-c59jh" podStartSLOduration=2.028708307 podStartE2EDuration="5.887788122s" totalImagesPullingTime="3.859079815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:54:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:54:56.886038587 +0000 UTC m=+515.398513918" watchObservedRunningTime="2026-07-01 20:54:56.887788122 +0000 UTC m=+515.400263311"
Jul 01 20:54:59.879725 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:59.879684    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-pgv8h" event={"ID":"b7720a2b-19f8-4d1a-b0f6-a481c4dc0f62","Type":"ContainerStarted","Data":"6df4363f5c9d67ddb8250e358162bfb270a427df9407bf42c52424831712c637"}
Jul 01 20:54:59.905655 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:54:59.905592    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-pgv8h" podStartSLOduration=0.814436913 podStartE2EDuration="5.905577351s" totalImagesPullingTime="5.091140438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:54:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:54:59.90325435 +0000 UTC m=+518.415729543" watchObservedRunningTime="2026-07-01 20:54:59.905577351 +0000 UTC m=+518.418052566"
Jul 01 20:55:07.871670 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:55:07.871621    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-c59jh"
Jul 01 20:55:36.614222 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:55:36.614188    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-69574b596d-8mmn4"]
Jul 01 20:55:36.636719 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:55:36.636691    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-69574b596d-8mmn4"]
Jul 01 20:55:36.636845 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:55:36.636777    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-69574b596d-8mmn4"
Jul 01 20:55:36.639568 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:55:36.639538    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 01 20:55:36.645680 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:55:36.645601    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-69574b596d-8mmn4"]
Jul 01 20:55:36.772363 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:55:36.772329    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f0cb5c83-be3d-4f3d-8bd4-ea2170710461-config-file\") pod \"limitador-limitador-69574b596d-8mmn4\" (UID: \"f0cb5c83-be3d-4f3d-8bd4-ea2170710461\") " pod="kuadrant-system/limitador-limitador-69574b596d-8mmn4"
Jul 01 20:55:36.772469 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:55:36.772381    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp7jd\" (UniqueName: \"kubernetes.io/projected/f0cb5c83-be3d-4f3d-8bd4-ea2170710461-kube-api-access-gp7jd\") pod \"limitador-limitador-69574b596d-8mmn4\" (UID: \"f0cb5c83-be3d-4f3d-8bd4-ea2170710461\") " pod="kuadrant-system/limitador-limitador-69574b596d-8mmn4"
Jul 01 20:55:36.873340 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:55:36.873277    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f0cb5c83-be3d-4f3d-8bd4-ea2170710461-config-file\") pod \"limitador-limitador-69574b596d-8mmn4\" (UID: \"f0cb5c83-be3d-4f3d-8bd4-ea2170710461\") " pod="kuadrant-system/limitador-limitador-69574b596d-8mmn4"
Jul 01 20:55:36.873340 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:55:36.873319    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp7jd\" (UniqueName: \"kubernetes.io/projected/f0cb5c83-be3d-4f3d-8bd4-ea2170710461-kube-api-access-gp7jd\") pod \"limitador-limitador-69574b596d-8mmn4\" (UID: \"f0cb5c83-be3d-4f3d-8bd4-ea2170710461\") " pod="kuadrant-system/limitador-limitador-69574b596d-8mmn4"
Jul 01 20:55:36.873899 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:55:36.873879    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f0cb5c83-be3d-4f3d-8bd4-ea2170710461-config-file\") pod \"limitador-limitador-69574b596d-8mmn4\" (UID: \"f0cb5c83-be3d-4f3d-8bd4-ea2170710461\") " pod="kuadrant-system/limitador-limitador-69574b596d-8mmn4"
Jul 01 20:55:36.881858 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:55:36.881835    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp7jd\" (UniqueName: \"kubernetes.io/projected/f0cb5c83-be3d-4f3d-8bd4-ea2170710461-kube-api-access-gp7jd\") pod \"limitador-limitador-69574b596d-8mmn4\" (UID: \"f0cb5c83-be3d-4f3d-8bd4-ea2170710461\") " pod="kuadrant-system/limitador-limitador-69574b596d-8mmn4"
Jul 01 20:55:36.949266 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:55:36.949244    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-69574b596d-8mmn4"
Jul 01 20:55:37.079263 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:55:37.079242    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-69574b596d-8mmn4"]
Jul 01 20:55:37.081161 ip-10-0-140-104 kubenswrapper[2571]: W0701 20:55:37.081139    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0cb5c83_be3d_4f3d_8bd4_ea2170710461.slice/crio-457f02511cd08ce5281429bd1e1cf55eca4a1f3cd6396da40af1a4bea9510b9d WatchSource:0}: Error finding container 457f02511cd08ce5281429bd1e1cf55eca4a1f3cd6396da40af1a4bea9510b9d: Status 404 returned error can't find the container with id 457f02511cd08ce5281429bd1e1cf55eca4a1f3cd6396da40af1a4bea9510b9d
Jul 01 20:55:38.050041 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:55:38.050004    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-69574b596d-8mmn4" event={"ID":"f0cb5c83-be3d-4f3d-8bd4-ea2170710461","Type":"ContainerStarted","Data":"457f02511cd08ce5281429bd1e1cf55eca4a1f3cd6396da40af1a4bea9510b9d"}
Jul 01 20:55:42.072232 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:55:42.072182    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-69574b596d-8mmn4" event={"ID":"f0cb5c83-be3d-4f3d-8bd4-ea2170710461","Type":"ContainerStarted","Data":"bf43e44a39d791d8fe53fad5d2d91deaf91167b85773ace724b0bcc4acb58e27"}
Jul 01 20:55:42.072746 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:55:42.072254    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-69574b596d-8mmn4"
Jul 01 20:55:42.090505 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:55:42.090453    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-69574b596d-8mmn4" podStartSLOduration=2.06716753 podStartE2EDuration="6.090436492s" totalImagesPullingTime="4.023268962s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:55:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:55:42.088009365 +0000 UTC m=+560.600484555" watchObservedRunningTime="2026-07-01 20:55:42.090436492 +0000 UTC m=+560.602911750"
Jul 01 20:55:53.077304 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:55:53.077271    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-69574b596d-8mmn4"
Jul 01 20:58:53.937995 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:58:53.937958    2571 generic.go:356] "Generic (PLEG): container finished" podID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13" containerID="8b32b47e6634253200d19f5c9f1d0c62cfe74575698b0f0bc4576d9719693d6d" exitCode=0
Jul 01 20:58:53.938393 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:58:53.938037    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" event={"ID":"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13","Type":"ContainerDied","Data":"8b32b47e6634253200d19f5c9f1d0c62cfe74575698b0f0bc4576d9719693d6d"}
Jul 01 20:58:53.938393 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:58:53.938381    2571 scope.go:117] "RemoveContainer" containerID="8b32b47e6634253200d19f5c9f1d0c62cfe74575698b0f0bc4576d9719693d6d"
Jul 01 20:58:53.939336 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:58:53.939304    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 01 20:58:53.939450 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:58:53.939391    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 01 20:58:53.939450 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:58:53.939443    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 01 20:58:54.944994 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:58:54.944962    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" event={"ID":"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13","Type":"ContainerStarted","Data":"4a59d2bd5cd10bb3cefa7b8eeb96217d17136090dccd2ed2e939a4d007c654e3"}
Jul 01 20:58:55.722853 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:58:55.722819    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:58:55.723927 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:58:55.723890    2571 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-655f7c58b4-57g8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body=
Jul 01 20:58:55.724057 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:58:55.723950    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" podUID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused"
Jul 01 20:58:56.723379 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:58:56.723343    2571 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-655f7c58b4-57g8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body=
Jul 01 20:58:56.723786 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:58:56.723409    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" podUID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused"
Jul 01 20:58:57.723578 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:58:57.723549    2571 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-655f7c58b4-57g8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body=
Jul 01 20:58:57.723950 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:58:57.723607    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" podUID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused"
Jul 01 20:58:58.723566 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:58:58.723530    2571 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-655f7c58b4-57g8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body=
Jul 01 20:58:58.723750 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:58:58.723599    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" podUID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused"
Jul 01 20:58:58.724694 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:58:58.724674    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:58:59.722829 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:58:59.722793    2571 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-655f7c58b4-57g8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body=
Jul 01 20:58:59.722992 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:58:59.722846    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" podUID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused"
Jul 01 20:59:00.727771 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:59:00.727738    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 20:59:00.728722 ip-10-0-140-104 kubenswrapper[2571]: I0701 20:59:00.728700    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 21:00:31.469310 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:00:31.469277    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m"]
Jul 01 21:00:31.472673 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:00:31.472657    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m"
Jul 01 21:00:31.476762 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:00:31.476736    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-76tz7\""
Jul 01 21:00:31.476762 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:00:31.476760    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv5c7e67b6c51568d1d6d13829a9337f2a-kserve-self-signed-certs\""
Jul 01 21:00:31.477009 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:00:31.476996    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 01 21:00:31.477055 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:00:31.477034    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 01 21:00:31.480970 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:00:31.480949    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m"]
Jul 01 21:00:31.650617 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:00:31.650589    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-dshm\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m\" (UID: \"e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m"
Jul 01 21:00:31.650760 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:00:31.650657    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-model-cache\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m\" (UID: \"e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m"
Jul 01 21:00:31.650760 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:00:31.650679    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-tmp-dir\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m\" (UID: \"e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m"
Jul 01 21:00:31.650760 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:00:31.650713    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-tls-certs\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m\" (UID: \"e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m"
Jul 01 21:00:31.650876 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:00:31.650800    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnr4n\" (UniqueName: \"kubernetes.io/projected/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-kube-api-access-rnr4n\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m\" (UID: \"e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m"
Jul 01 21:00:31.650876 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:00:31.650831    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-home\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m\" (UID: \"e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m"
Jul 01 21:00:31.752256 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:00:31.752182    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-dshm\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m\" (UID: \"e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m"
Jul 01 21:00:31.752256 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:00:31.752223    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-model-cache\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m\" (UID: \"e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m"
Jul 01 21:00:31.752256 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:00:31.752246    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-tmp-dir\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m\" (UID: \"e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m"
Jul 01 21:00:31.752471 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:00:31.752292    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-tls-certs\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m\" (UID: \"e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m"
Jul 01 21:00:31.752471 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:00:31.752339    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnr4n\" (UniqueName: \"kubernetes.io/projected/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-kube-api-access-rnr4n\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m\" (UID: \"e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m"
Jul 01 21:00:31.752471 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:00:31.752365    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-home\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m\" (UID: \"e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m"
Jul 01 21:00:31.752722 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:00:31.752699    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-model-cache\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m\" (UID: \"e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m"
Jul 01 21:00:31.752790 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:00:31.752744    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-tmp-dir\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m\" (UID: \"e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m"
Jul 01 21:00:31.752790 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:00:31.752768    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-home\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m\" (UID: \"e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m"
Jul 01 21:00:31.754508 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:00:31.754485    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-dshm\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m\" (UID: \"e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m"
Jul 01 21:00:31.754848 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:00:31.754833    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-tls-certs\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m\" (UID: \"e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m"
Jul 01 21:00:31.767487 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:00:31.767465    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnr4n\" (UniqueName: \"kubernetes.io/projected/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-kube-api-access-rnr4n\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m\" (UID: \"e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m"
Jul 01 21:00:31.783359 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:00:31.783338    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m"
Jul 01 21:00:31.916075 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:00:31.916055    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m"]
Jul 01 21:00:31.918157 ip-10-0-140-104 kubenswrapper[2571]: W0701 21:00:31.918135    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode72945c5_fe24_4b1b_8a16_a7b8afb2cf5e.slice/crio-bad972ee3f970fc5618f6b2d2655e8930009cd731820ed1d25f4476dbb593528 WatchSource:0}: Error finding container bad972ee3f970fc5618f6b2d2655e8930009cd731820ed1d25f4476dbb593528: Status 404 returned error can't find the container with id bad972ee3f970fc5618f6b2d2655e8930009cd731820ed1d25f4476dbb593528
Jul 01 21:00:31.920015 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:00:31.920000    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 01 21:00:32.405977 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:00:32.405942    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m" event={"ID":"e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e","Type":"ContainerStarted","Data":"bad972ee3f970fc5618f6b2d2655e8930009cd731820ed1d25f4476dbb593528"}
Jul 01 21:01:02.555717 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:02.555676    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m" event={"ID":"e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e","Type":"ContainerStarted","Data":"b50091e0527028228cbfe877bcb23d2fa71d81c8ffa95b4bbdbb89c17452cea6"}
Jul 01 21:01:02.574338 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:02.574285    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m" podStartSLOduration=1.9265798680000001 podStartE2EDuration="31.574268764s" totalImagesPullingTime="29.647688896s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 21:00:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 21:01:02.572350177 +0000 UTC m=+881.084825367" watchObservedRunningTime="2026-07-01 21:01:02.574268764 +0000 UTC m=+881.086743954"
Jul 01 21:01:02.783904 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:02.783876    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m"
Jul 01 21:01:11.784269 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:11.784227    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m"
Jul 01 21:01:11.796961 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:11.796936    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m"
Jul 01 21:01:12.617643 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:12.617562    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m"
Jul 01 21:01:25.578053 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:25.578020    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m"]
Jul 01 21:01:40.598575 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:40.598518    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m" podUID="e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e" containerName="main" containerID="cri-o://b50091e0527028228cbfe877bcb23d2fa71d81c8ffa95b4bbdbb89c17452cea6" gracePeriod=45
Jul 01 21:01:40.607309 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:40.607274    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m" podUID="e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.25:8000/health\": dial tcp 10.133.0.25:8000: connect: connection refused"
Jul 01 21:01:40.757146 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:40.757115    2571 generic.go:356] "Generic (PLEG): container finished" podID="e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e" containerID="b50091e0527028228cbfe877bcb23d2fa71d81c8ffa95b4bbdbb89c17452cea6" exitCode=0
Jul 01 21:01:40.757316 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:40.757191    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m" event={"ID":"e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e","Type":"ContainerDied","Data":"b50091e0527028228cbfe877bcb23d2fa71d81c8ffa95b4bbdbb89c17452cea6"}
Jul 01 21:01:40.840444 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:40.840422    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m"
Jul 01 21:01:40.995126 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:40.995103    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-tmp-dir\") pod \"e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e\" (UID: \"e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e\") "
Jul 01 21:01:40.995243 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:40.995140    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-model-cache\") pod \"e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e\" (UID: \"e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e\") "
Jul 01 21:01:40.995243 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:40.995159    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rnr4n\" (UniqueName: \"kubernetes.io/projected/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-kube-api-access-rnr4n\") pod \"e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e\" (UID: \"e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e\") "
Jul 01 21:01:40.995243 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:40.995202    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-home\") pod \"e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e\" (UID: \"e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e\") "
Jul 01 21:01:40.995407 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:40.995260    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-dshm\") pod \"e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e\" (UID: \"e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e\") "
Jul 01 21:01:40.995407 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:40.995310    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-tls-certs\") pod \"e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e\" (UID: \"e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e\") "
Jul 01 21:01:40.995407 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:40.995366    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e" (UID: "e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 21:01:40.995550 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:40.995447    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-model-cache" (OuterVolumeSpecName: "model-cache") pod "e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e" (UID: "e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 21:01:40.995550 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:40.995459    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-home" (OuterVolumeSpecName: "home") pod "e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e" (UID: "e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 21:01:40.995737 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:40.995582    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-tmp-dir\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:01:40.995737 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:40.995603    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-model-cache\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:01:40.995737 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:40.995617    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-home\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:01:40.997507 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:40.997480    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e" (UID: "e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 21:01:40.997744 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:40.997719    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-dshm" (OuterVolumeSpecName: "dshm") pod "e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e" (UID: "e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 21:01:40.997839 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:40.997793    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-kube-api-access-rnr4n" (OuterVolumeSpecName: "kube-api-access-rnr4n") pod "e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e" (UID: "e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e"). InnerVolumeSpecName "kube-api-access-rnr4n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 21:01:41.096134 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:41.096113    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-dshm\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:01:41.096134 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:41.096134    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-tls-certs\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:01:41.096258 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:41.096145    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rnr4n\" (UniqueName: \"kubernetes.io/projected/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e-kube-api-access-rnr4n\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:01:41.762883 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:41.762853    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m"
Jul 01 21:01:41.763243 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:41.762859    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m" event={"ID":"e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e","Type":"ContainerDied","Data":"bad972ee3f970fc5618f6b2d2655e8930009cd731820ed1d25f4476dbb593528"}
Jul 01 21:01:41.763243 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:41.763011    2571 scope.go:117] "RemoveContainer" containerID="b50091e0527028228cbfe877bcb23d2fa71d81c8ffa95b4bbdbb89c17452cea6"
Jul 01 21:01:41.785423 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:41.785399    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m"]
Jul 01 21:01:41.791500 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:41.791480    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879h867m"]
Jul 01 21:01:41.839178 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:41.839155    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e" path="/var/lib/kubelet/pods/e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e/volumes"
Jul 01 21:01:48.372354 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:48.372249    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7"]
Jul 01 21:01:48.373118 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:48.373059    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e" containerName="main"
Jul 01 21:01:48.373118 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:48.373089    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e" containerName="main"
Jul 01 21:01:48.373354 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:48.373305    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e72945c5-fe24-4b1b-8a16-a7b8afb2cf5e" containerName="main"
Jul 01 21:01:48.377469 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:48.377278    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7"
Jul 01 21:01:48.380692 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:48.380668    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-76tz7\""
Jul 01 21:01:48.380831 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:48.380734    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 01 21:01:48.381860 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:48.381837    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 01 21:01:48.382126 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:48.382110    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisve55ae740357a3a31a27cdb8b66ffe20f-kserve-self-signed-certs\""
Jul 01 21:01:48.384378 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:48.384353    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7"]
Jul 01 21:01:48.550347 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:48.550318    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf16d772-1783-4c92-af1f-87f43149e7ec-tls-certs\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7\" (UID: \"bf16d772-1783-4c92-af1f-87f43149e7ec\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7"
Jul 01 21:01:48.550347 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:48.550348    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf16d772-1783-4c92-af1f-87f43149e7ec-dshm\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7\" (UID: \"bf16d772-1783-4c92-af1f-87f43149e7ec\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7"
Jul 01 21:01:48.550543 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:48.550392    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf16d772-1783-4c92-af1f-87f43149e7ec-model-cache\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7\" (UID: \"bf16d772-1783-4c92-af1f-87f43149e7ec\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7"
Jul 01 21:01:48.550543 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:48.550472    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7lxj\" (UniqueName: \"kubernetes.io/projected/bf16d772-1783-4c92-af1f-87f43149e7ec-kube-api-access-z7lxj\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7\" (UID: \"bf16d772-1783-4c92-af1f-87f43149e7ec\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7"
Jul 01 21:01:48.550543 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:48.550512    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf16d772-1783-4c92-af1f-87f43149e7ec-home\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7\" (UID: \"bf16d772-1783-4c92-af1f-87f43149e7ec\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7"
Jul 01 21:01:48.550543 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:48.550541    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf16d772-1783-4c92-af1f-87f43149e7ec-tmp-dir\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7\" (UID: \"bf16d772-1783-4c92-af1f-87f43149e7ec\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7"
Jul 01 21:01:48.651848 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:48.651774    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf16d772-1783-4c92-af1f-87f43149e7ec-model-cache\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7\" (UID: \"bf16d772-1783-4c92-af1f-87f43149e7ec\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7"
Jul 01 21:01:48.651848 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:48.651817    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7lxj\" (UniqueName: \"kubernetes.io/projected/bf16d772-1783-4c92-af1f-87f43149e7ec-kube-api-access-z7lxj\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7\" (UID: \"bf16d772-1783-4c92-af1f-87f43149e7ec\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7"
Jul 01 21:01:48.651848 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:48.651837    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf16d772-1783-4c92-af1f-87f43149e7ec-home\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7\" (UID: \"bf16d772-1783-4c92-af1f-87f43149e7ec\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7"
Jul 01 21:01:48.652051 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:48.651876    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf16d772-1783-4c92-af1f-87f43149e7ec-tmp-dir\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7\" (UID: \"bf16d772-1783-4c92-af1f-87f43149e7ec\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7"
Jul 01 21:01:48.652051 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:48.651903    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf16d772-1783-4c92-af1f-87f43149e7ec-tls-certs\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7\" (UID: \"bf16d772-1783-4c92-af1f-87f43149e7ec\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7"
Jul 01 21:01:48.652051 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:48.651923    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf16d772-1783-4c92-af1f-87f43149e7ec-dshm\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7\" (UID: \"bf16d772-1783-4c92-af1f-87f43149e7ec\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7"
Jul 01 21:01:48.652235 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:48.652216    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf16d772-1783-4c92-af1f-87f43149e7ec-model-cache\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7\" (UID: \"bf16d772-1783-4c92-af1f-87f43149e7ec\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7"
Jul 01 21:01:48.652319 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:48.652300    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf16d772-1783-4c92-af1f-87f43149e7ec-tmp-dir\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7\" (UID: \"bf16d772-1783-4c92-af1f-87f43149e7ec\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7"
Jul 01 21:01:48.652373 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:48.652321    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf16d772-1783-4c92-af1f-87f43149e7ec-home\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7\" (UID: \"bf16d772-1783-4c92-af1f-87f43149e7ec\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7"
Jul 01 21:01:48.654343 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:48.654322    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf16d772-1783-4c92-af1f-87f43149e7ec-dshm\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7\" (UID: \"bf16d772-1783-4c92-af1f-87f43149e7ec\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7"
Jul 01 21:01:48.654581 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:48.654563    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf16d772-1783-4c92-af1f-87f43149e7ec-tls-certs\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7\" (UID: \"bf16d772-1783-4c92-af1f-87f43149e7ec\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7"
Jul 01 21:01:48.660616 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:48.660596    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7lxj\" (UniqueName: \"kubernetes.io/projected/bf16d772-1783-4c92-af1f-87f43149e7ec-kube-api-access-z7lxj\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7\" (UID: \"bf16d772-1783-4c92-af1f-87f43149e7ec\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7"
Jul 01 21:01:48.689644 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:48.689605    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7"
Jul 01 21:01:48.823375 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:48.823351    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7"]
Jul 01 21:01:48.825186 ip-10-0-140-104 kubenswrapper[2571]: W0701 21:01:48.825157    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf16d772_1783_4c92_af1f_87f43149e7ec.slice/crio-acd3aef63a40adc2081c60fa27b6c6c337ae3885c0ea8dbdddea6588901372be WatchSource:0}: Error finding container acd3aef63a40adc2081c60fa27b6c6c337ae3885c0ea8dbdddea6588901372be: Status 404 returned error can't find the container with id acd3aef63a40adc2081c60fa27b6c6c337ae3885c0ea8dbdddea6588901372be
Jul 01 21:01:49.805780 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:49.805738    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7" event={"ID":"bf16d772-1783-4c92-af1f-87f43149e7ec","Type":"ContainerStarted","Data":"074bfda63699b477253c0e660954baa0a09f5c658555b50971c800fd66fbc5a6"}
Jul 01 21:01:49.806131 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:49.805790    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7" event={"ID":"bf16d772-1783-4c92-af1f-87f43149e7ec","Type":"ContainerStarted","Data":"acd3aef63a40adc2081c60fa27b6c6c337ae3885c0ea8dbdddea6588901372be"}
Jul 01 21:01:49.826734 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:49.826690    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7" podStartSLOduration=1.826676132 podStartE2EDuration="1.826676132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 21:01:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 21:01:49.825136996 +0000 UTC m=+928.337612187" watchObservedRunningTime="2026-07-01 21:01:49.826676132 +0000 UTC m=+928.339151321"
Jul 01 21:01:50.689721 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:50.689683    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7"
Jul 01 21:01:58.690542 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:58.690499    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7"
Jul 01 21:01:58.703469 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:58.703444    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7"
Jul 01 21:01:58.859412 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:01:58.859385    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7"
Jul 01 21:03:49.629444 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:03:49.629411    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7"]
Jul 01 21:04:04.649674 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:04.649639    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7" podUID="bf16d772-1783-4c92-af1f-87f43149e7ec" containerName="main" containerID="cri-o://074bfda63699b477253c0e660954baa0a09f5c658555b50971c800fd66fbc5a6" gracePeriod=45
Jul 01 21:04:04.887794 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:04.887767    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7"
Jul 01 21:04:04.970186 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:04.970161    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7lxj\" (UniqueName: \"kubernetes.io/projected/bf16d772-1783-4c92-af1f-87f43149e7ec-kube-api-access-z7lxj\") pod \"bf16d772-1783-4c92-af1f-87f43149e7ec\" (UID: \"bf16d772-1783-4c92-af1f-87f43149e7ec\") "
Jul 01 21:04:04.970300 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:04.970193    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf16d772-1783-4c92-af1f-87f43149e7ec-tls-certs\") pod \"bf16d772-1783-4c92-af1f-87f43149e7ec\" (UID: \"bf16d772-1783-4c92-af1f-87f43149e7ec\") "
Jul 01 21:04:04.970300 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:04.970230    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf16d772-1783-4c92-af1f-87f43149e7ec-home\") pod \"bf16d772-1783-4c92-af1f-87f43149e7ec\" (UID: \"bf16d772-1783-4c92-af1f-87f43149e7ec\") "
Jul 01 21:04:04.970300 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:04.970279    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf16d772-1783-4c92-af1f-87f43149e7ec-dshm\") pod \"bf16d772-1783-4c92-af1f-87f43149e7ec\" (UID: \"bf16d772-1783-4c92-af1f-87f43149e7ec\") "
Jul 01 21:04:04.970457 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:04.970305    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf16d772-1783-4c92-af1f-87f43149e7ec-model-cache\") pod \"bf16d772-1783-4c92-af1f-87f43149e7ec\" (UID: \"bf16d772-1783-4c92-af1f-87f43149e7ec\") "
Jul 01 21:04:04.970457 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:04.970351    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf16d772-1783-4c92-af1f-87f43149e7ec-tmp-dir\") pod \"bf16d772-1783-4c92-af1f-87f43149e7ec\" (UID: \"bf16d772-1783-4c92-af1f-87f43149e7ec\") "
Jul 01 21:04:04.970577 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:04.970555    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf16d772-1783-4c92-af1f-87f43149e7ec-home" (OuterVolumeSpecName: "home") pod "bf16d772-1783-4c92-af1f-87f43149e7ec" (UID: "bf16d772-1783-4c92-af1f-87f43149e7ec"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 21:04:04.970674 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:04.970575    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf16d772-1783-4c92-af1f-87f43149e7ec-model-cache" (OuterVolumeSpecName: "model-cache") pod "bf16d772-1783-4c92-af1f-87f43149e7ec" (UID: "bf16d772-1783-4c92-af1f-87f43149e7ec"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 21:04:04.970736 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:04.970670    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf16d772-1783-4c92-af1f-87f43149e7ec-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bf16d772-1783-4c92-af1f-87f43149e7ec" (UID: "bf16d772-1783-4c92-af1f-87f43149e7ec"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 21:04:04.971049 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:04.970895    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf16d772-1783-4c92-af1f-87f43149e7ec-home\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:04:04.972362 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:04.972334    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf16d772-1783-4c92-af1f-87f43149e7ec-dshm" (OuterVolumeSpecName: "dshm") pod "bf16d772-1783-4c92-af1f-87f43149e7ec" (UID: "bf16d772-1783-4c92-af1f-87f43149e7ec"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 21:04:04.972446 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:04.972379    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf16d772-1783-4c92-af1f-87f43149e7ec-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bf16d772-1783-4c92-af1f-87f43149e7ec" (UID: "bf16d772-1783-4c92-af1f-87f43149e7ec"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 21:04:04.972446 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:04.972424    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf16d772-1783-4c92-af1f-87f43149e7ec-kube-api-access-z7lxj" (OuterVolumeSpecName: "kube-api-access-z7lxj") pod "bf16d772-1783-4c92-af1f-87f43149e7ec" (UID: "bf16d772-1783-4c92-af1f-87f43149e7ec"). InnerVolumeSpecName "kube-api-access-z7lxj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 21:04:05.071983 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:05.071965    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf16d772-1783-4c92-af1f-87f43149e7ec-dshm\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:04:05.071983 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:05.071983    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf16d772-1783-4c92-af1f-87f43149e7ec-model-cache\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:04:05.072094 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:05.071991    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf16d772-1783-4c92-af1f-87f43149e7ec-tmp-dir\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:04:05.072094 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:05.072000    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7lxj\" (UniqueName: \"kubernetes.io/projected/bf16d772-1783-4c92-af1f-87f43149e7ec-kube-api-access-z7lxj\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:04:05.072094 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:05.072007    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf16d772-1783-4c92-af1f-87f43149e7ec-tls-certs\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:04:05.413381 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:05.413359    2571 generic.go:356] "Generic (PLEG): container finished" podID="bf16d772-1783-4c92-af1f-87f43149e7ec" containerID="074bfda63699b477253c0e660954baa0a09f5c658555b50971c800fd66fbc5a6" exitCode=0
Jul 01 21:04:05.413473 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:05.413420    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7"
Jul 01 21:04:05.413473 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:05.413440    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7" event={"ID":"bf16d772-1783-4c92-af1f-87f43149e7ec","Type":"ContainerDied","Data":"074bfda63699b477253c0e660954baa0a09f5c658555b50971c800fd66fbc5a6"}
Jul 01 21:04:05.413540 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:05.413520    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7" event={"ID":"bf16d772-1783-4c92-af1f-87f43149e7ec","Type":"ContainerDied","Data":"acd3aef63a40adc2081c60fa27b6c6c337ae3885c0ea8dbdddea6588901372be"}
Jul 01 21:04:05.413574 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:05.413548    2571 scope.go:117] "RemoveContainer" containerID="074bfda63699b477253c0e660954baa0a09f5c658555b50971c800fd66fbc5a6"
Jul 01 21:04:05.422196 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:05.422178    2571 scope.go:117] "RemoveContainer" containerID="074bfda63699b477253c0e660954baa0a09f5c658555b50971c800fd66fbc5a6"
Jul 01 21:04:05.422431 ip-10-0-140-104 kubenswrapper[2571]: E0701 21:04:05.422414    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"074bfda63699b477253c0e660954baa0a09f5c658555b50971c800fd66fbc5a6\": container with ID starting with 074bfda63699b477253c0e660954baa0a09f5c658555b50971c800fd66fbc5a6 not found: ID does not exist" containerID="074bfda63699b477253c0e660954baa0a09f5c658555b50971c800fd66fbc5a6"
Jul 01 21:04:05.422475 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:05.422438    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"074bfda63699b477253c0e660954baa0a09f5c658555b50971c800fd66fbc5a6"} err="failed to get container status \"074bfda63699b477253c0e660954baa0a09f5c658555b50971c800fd66fbc5a6\": rpc error: code = NotFound desc = could not find container \"074bfda63699b477253c0e660954baa0a09f5c658555b50971c800fd66fbc5a6\": container with ID starting with 074bfda63699b477253c0e660954baa0a09f5c658555b50971c800fd66fbc5a6 not found: ID does not exist"
Jul 01 21:04:05.434047 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:05.434023    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7"]
Jul 01 21:04:05.436263 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:05.436242    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7"]
Jul 01 21:04:05.839456 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:05.839382    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf16d772-1783-4c92-af1f-87f43149e7ec" path="/var/lib/kubelet/pods/bf16d772-1783-4c92-af1f-87f43149e7ec/volumes"
Jul 01 21:04:05.848551 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:05.848525    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4xwdp7" podUID="bf16d772-1783-4c92-af1f-87f43149e7ec" containerName="main" probeResult="failure" output="Get \"https://10.133.0.26:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 01 21:04:32.472488 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:32.472456    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z"]
Jul 01 21:04:32.472949 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:32.472934    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf16d772-1783-4c92-af1f-87f43149e7ec" containerName="main"
Jul 01 21:04:32.472992 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:32.472953    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf16d772-1783-4c92-af1f-87f43149e7ec" containerName="main"
Jul 01 21:04:32.473080 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:32.473069    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf16d772-1783-4c92-af1f-87f43149e7ec" containerName="main"
Jul 01 21:04:32.477464 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:32.477444    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z"
Jul 01 21:04:32.480528 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:32.480505    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 01 21:04:32.480661 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:32.480569    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 01 21:04:32.481800 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:32.481781    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-76tz7\""
Jul 01 21:04:32.481910 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:32.481781    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvca2d2d7d499abb359505529ebe02c136-kserve-self-signed-certs\""
Jul 01 21:04:32.486316 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:32.486296    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z"]
Jul 01 21:04:32.569012 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:32.568991    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30806680-6984-4127-ad3d-9994ba2909ff-dshm\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z\" (UID: \"30806680-6984-4127-ad3d-9994ba2909ff\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z"
Jul 01 21:04:32.569135 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:32.569020    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30806680-6984-4127-ad3d-9994ba2909ff-model-cache\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z\" (UID: \"30806680-6984-4127-ad3d-9994ba2909ff\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z"
Jul 01 21:04:32.569135 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:32.569049    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62srw\" (UniqueName: \"kubernetes.io/projected/30806680-6984-4127-ad3d-9994ba2909ff-kube-api-access-62srw\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z\" (UID: \"30806680-6984-4127-ad3d-9994ba2909ff\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z"
Jul 01 21:04:32.569135 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:32.569113    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30806680-6984-4127-ad3d-9994ba2909ff-tls-certs\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z\" (UID: \"30806680-6984-4127-ad3d-9994ba2909ff\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z"
Jul 01 21:04:32.569280 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:32.569164    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30806680-6984-4127-ad3d-9994ba2909ff-tmp-dir\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z\" (UID: \"30806680-6984-4127-ad3d-9994ba2909ff\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z"
Jul 01 21:04:32.569280 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:32.569211    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30806680-6984-4127-ad3d-9994ba2909ff-home\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z\" (UID: \"30806680-6984-4127-ad3d-9994ba2909ff\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z"
Jul 01 21:04:32.670079 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:32.670054    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30806680-6984-4127-ad3d-9994ba2909ff-home\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z\" (UID: \"30806680-6984-4127-ad3d-9994ba2909ff\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z"
Jul 01 21:04:32.670172 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:32.670085    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30806680-6984-4127-ad3d-9994ba2909ff-dshm\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z\" (UID: \"30806680-6984-4127-ad3d-9994ba2909ff\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z"
Jul 01 21:04:32.670172 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:32.670105    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30806680-6984-4127-ad3d-9994ba2909ff-model-cache\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z\" (UID: \"30806680-6984-4127-ad3d-9994ba2909ff\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z"
Jul 01 21:04:32.670172 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:32.670130    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62srw\" (UniqueName: \"kubernetes.io/projected/30806680-6984-4127-ad3d-9994ba2909ff-kube-api-access-62srw\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z\" (UID: \"30806680-6984-4127-ad3d-9994ba2909ff\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z"
Jul 01 21:04:32.670172 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:32.670151    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30806680-6984-4127-ad3d-9994ba2909ff-tls-certs\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z\" (UID: \"30806680-6984-4127-ad3d-9994ba2909ff\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z"
Jul 01 21:04:32.670352 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:32.670193    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30806680-6984-4127-ad3d-9994ba2909ff-tmp-dir\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z\" (UID: \"30806680-6984-4127-ad3d-9994ba2909ff\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z"
Jul 01 21:04:32.670470 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:32.670447    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30806680-6984-4127-ad3d-9994ba2909ff-home\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z\" (UID: \"30806680-6984-4127-ad3d-9994ba2909ff\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z"
Jul 01 21:04:32.670527 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:32.670456    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30806680-6984-4127-ad3d-9994ba2909ff-model-cache\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z\" (UID: \"30806680-6984-4127-ad3d-9994ba2909ff\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z"
Jul 01 21:04:32.670612 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:32.670597    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30806680-6984-4127-ad3d-9994ba2909ff-tmp-dir\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z\" (UID: \"30806680-6984-4127-ad3d-9994ba2909ff\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z"
Jul 01 21:04:32.672463 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:32.672438    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30806680-6984-4127-ad3d-9994ba2909ff-dshm\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z\" (UID: \"30806680-6984-4127-ad3d-9994ba2909ff\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z"
Jul 01 21:04:32.672744 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:32.672730    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30806680-6984-4127-ad3d-9994ba2909ff-tls-certs\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z\" (UID: \"30806680-6984-4127-ad3d-9994ba2909ff\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z"
Jul 01 21:04:32.679794 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:32.679772    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62srw\" (UniqueName: \"kubernetes.io/projected/30806680-6984-4127-ad3d-9994ba2909ff-kube-api-access-62srw\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z\" (UID: \"30806680-6984-4127-ad3d-9994ba2909ff\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z"
Jul 01 21:04:32.788576 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:32.788523    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z"
Jul 01 21:04:32.920093 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:32.920057    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z"]
Jul 01 21:04:32.926947 ip-10-0-140-104 kubenswrapper[2571]: W0701 21:04:32.926920    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30806680_6984_4127_ad3d_9994ba2909ff.slice/crio-f99e64afc7637ac8131dc0d3d0b4df952500112306022629f26d8aa702afbd48 WatchSource:0}: Error finding container f99e64afc7637ac8131dc0d3d0b4df952500112306022629f26d8aa702afbd48: Status 404 returned error can't find the container with id f99e64afc7637ac8131dc0d3d0b4df952500112306022629f26d8aa702afbd48
Jul 01 21:04:33.532143 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:33.532105    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z" event={"ID":"30806680-6984-4127-ad3d-9994ba2909ff","Type":"ContainerStarted","Data":"32ccd362ec939303491b5c602cefc4217b517bedaa66a6aeba9bed0a6daf4cd5"}
Jul 01 21:04:33.532599 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:33.532156    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z" event={"ID":"30806680-6984-4127-ad3d-9994ba2909ff","Type":"ContainerStarted","Data":"f99e64afc7637ac8131dc0d3d0b4df952500112306022629f26d8aa702afbd48"}
Jul 01 21:04:33.554055 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:33.554010    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z" podStartSLOduration=1.553995754 podStartE2EDuration="1.553995754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 21:04:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 21:04:33.552953353 +0000 UTC m=+1092.065428544" watchObservedRunningTime="2026-07-01 21:04:33.553995754 +0000 UTC m=+1092.066470973"
Jul 01 21:04:33.789440 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:33.789369    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z"
Jul 01 21:04:42.789661 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:42.789613    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z"
Jul 01 21:04:42.801357 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:42.801329    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z"
Jul 01 21:04:43.588331 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:04:43.588302    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z"
Jul 01 21:06:19.983449 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:19.983412    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr"]
Jul 01 21:06:19.986956 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:19.986933    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr"
Jul 01 21:06:19.991305 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:19.991277    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-epp-sa-dockercfg-bwdxf\""
Jul 01 21:06:19.991305 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:19.991289    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv08544b88a8d963ffd553cc1f3ed82d16-kserve-self-signed-certs\""
Jul 01 21:06:19.997002 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:19.996977    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr"]
Jul 01 21:06:20.018042 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:20.018021    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/3c077208-6deb-4a69-ab5c-4853033a1429-tokenizer-uds\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr\" (UID: \"3c077208-6deb-4a69-ab5c-4853033a1429\") " pod="kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr"
Jul 01 21:06:20.018137 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:20.018073    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtt5m\" (UniqueName: \"kubernetes.io/projected/3c077208-6deb-4a69-ab5c-4853033a1429-kube-api-access-gtt5m\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr\" (UID: \"3c077208-6deb-4a69-ab5c-4853033a1429\") " pod="kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr"
Jul 01 21:06:20.018137 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:20.018100    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c077208-6deb-4a69-ab5c-4853033a1429-tls-certs\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr\" (UID: \"3c077208-6deb-4a69-ab5c-4853033a1429\") " pod="kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr"
Jul 01 21:06:20.119067 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:20.119044    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/3c077208-6deb-4a69-ab5c-4853033a1429-tokenizer-uds\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr\" (UID: \"3c077208-6deb-4a69-ab5c-4853033a1429\") " pod="kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr"
Jul 01 21:06:20.119162 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:20.119099    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtt5m\" (UniqueName: \"kubernetes.io/projected/3c077208-6deb-4a69-ab5c-4853033a1429-kube-api-access-gtt5m\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr\" (UID: \"3c077208-6deb-4a69-ab5c-4853033a1429\") " pod="kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr"
Jul 01 21:06:20.119162 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:20.119124    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c077208-6deb-4a69-ab5c-4853033a1429-tls-certs\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr\" (UID: \"3c077208-6deb-4a69-ab5c-4853033a1429\") " pod="kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr"
Jul 01 21:06:20.119420 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:20.119401    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/3c077208-6deb-4a69-ab5c-4853033a1429-tokenizer-uds\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr\" (UID: \"3c077208-6deb-4a69-ab5c-4853033a1429\") " pod="kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr"
Jul 01 21:06:20.121649 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:20.121614    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c077208-6deb-4a69-ab5c-4853033a1429-tls-certs\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr\" (UID: \"3c077208-6deb-4a69-ab5c-4853033a1429\") " pod="kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr"
Jul 01 21:06:20.130579 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:20.130560    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtt5m\" (UniqueName: \"kubernetes.io/projected/3c077208-6deb-4a69-ab5c-4853033a1429-kube-api-access-gtt5m\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr\" (UID: \"3c077208-6deb-4a69-ab5c-4853033a1429\") " pod="kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr"
Jul 01 21:06:20.297926 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:20.297870    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr"
Jul 01 21:06:20.427004 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:20.426981    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr"]
Jul 01 21:06:20.429189 ip-10-0-140-104 kubenswrapper[2571]: W0701 21:06:20.429168    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c077208_6deb_4a69_ab5c_4853033a1429.slice/crio-c74a7145b21b7d63ce70333356212b3a484a8df4c89de49cb0704d1a34907611 WatchSource:0}: Error finding container c74a7145b21b7d63ce70333356212b3a484a8df4c89de49cb0704d1a34907611: Status 404 returned error can't find the container with id c74a7145b21b7d63ce70333356212b3a484a8df4c89de49cb0704d1a34907611
Jul 01 21:06:20.431544 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:20.431528    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 01 21:06:21.004200 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:21.004170    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr" event={"ID":"3c077208-6deb-4a69-ab5c-4853033a1429","Type":"ContainerStarted","Data":"c74a7145b21b7d63ce70333356212b3a484a8df4c89de49cb0704d1a34907611"}
Jul 01 21:06:23.017003 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:23.016970    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr" event={"ID":"3c077208-6deb-4a69-ab5c-4853033a1429","Type":"ContainerStarted","Data":"9c82ea245a356b8344723144530986dca1977a75eac56b7013d611db9dff8340"}
Jul 01 21:06:23.017375 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:23.017160    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr"
Jul 01 21:06:23.037444 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:23.037385    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr" podStartSLOduration=2.514175055 podStartE2EDuration="4.037367599s" totalImagesPullingTime="1.523192544s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 21:06:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 21:06:23.036758136 +0000 UTC m=+1201.549233326" watchObservedRunningTime="2026-07-01 21:06:23.037367599 +0000 UTC m=+1201.549842770"
Jul 01 21:06:54.023608 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:54.023583    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr"
Jul 01 21:06:55.516308 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:55.516275    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr"]
Jul 01 21:06:55.527412 ip-10-0-140-104 kubenswrapper[2571]: E0701 21:06:55.527383    2571 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 01 21:06:55.527412 ip-10-0-140-104 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 01 21:06:55.527412 ip-10-0-140-104 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 01 21:06:55.527412 ip-10-0-140-104 kubenswrapper[2571]:  > containerID="9c82ea245a356b8344723144530986dca1977a75eac56b7013d611db9dff8340" cmd=["/bin/sleep","15"]
Jul 01 21:06:55.527731 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:55.527430    2571 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr" message="" err=<
Jul 01 21:06:55.527731 ip-10-0-140-104 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 01 21:06:55.527731 ip-10-0-140-104 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 01 21:06:55.527731 ip-10-0-140-104 kubenswrapper[2571]:  >
Jul 01 21:06:55.527731 ip-10-0-140-104 kubenswrapper[2571]: E0701 21:06:55.527442    2571 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 01 21:06:55.527731 ip-10-0-140-104 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 01 21:06:55.527731 ip-10-0-140-104 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 01 21:06:55.527731 ip-10-0-140-104 kubenswrapper[2571]:  > pod="kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr" podUID="3c077208-6deb-4a69-ab5c-4853033a1429" containerName="main" containerID="cri-o://9c82ea245a356b8344723144530986dca1977a75eac56b7013d611db9dff8340"
Jul 01 21:06:55.527731 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:55.527480    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr" podUID="3c077208-6deb-4a69-ab5c-4853033a1429" containerName="main" containerID="cri-o://9c82ea245a356b8344723144530986dca1977a75eac56b7013d611db9dff8340" gracePeriod=60
Jul 01 21:06:55.773179 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:55.773128    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr"
Jul 01 21:06:55.893644 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:55.893600    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtt5m\" (UniqueName: \"kubernetes.io/projected/3c077208-6deb-4a69-ab5c-4853033a1429-kube-api-access-gtt5m\") pod \"3c077208-6deb-4a69-ab5c-4853033a1429\" (UID: \"3c077208-6deb-4a69-ab5c-4853033a1429\") "
Jul 01 21:06:55.893755 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:55.893699    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c077208-6deb-4a69-ab5c-4853033a1429-tls-certs\") pod \"3c077208-6deb-4a69-ab5c-4853033a1429\" (UID: \"3c077208-6deb-4a69-ab5c-4853033a1429\") "
Jul 01 21:06:55.893806 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:55.893766    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/3c077208-6deb-4a69-ab5c-4853033a1429-tokenizer-uds\") pod \"3c077208-6deb-4a69-ab5c-4853033a1429\" (UID: \"3c077208-6deb-4a69-ab5c-4853033a1429\") "
Jul 01 21:06:55.894072 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:55.894041    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c077208-6deb-4a69-ab5c-4853033a1429-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "3c077208-6deb-4a69-ab5c-4853033a1429" (UID: "3c077208-6deb-4a69-ab5c-4853033a1429"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 21:06:55.894211 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:55.894189    2571 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/3c077208-6deb-4a69-ab5c-4853033a1429-tokenizer-uds\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:06:55.896005 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:55.895981    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c077208-6deb-4a69-ab5c-4853033a1429-kube-api-access-gtt5m" (OuterVolumeSpecName: "kube-api-access-gtt5m") pod "3c077208-6deb-4a69-ab5c-4853033a1429" (UID: "3c077208-6deb-4a69-ab5c-4853033a1429"). InnerVolumeSpecName "kube-api-access-gtt5m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 21:06:55.896005 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:55.895994    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c077208-6deb-4a69-ab5c-4853033a1429-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3c077208-6deb-4a69-ab5c-4853033a1429" (UID: "3c077208-6deb-4a69-ab5c-4853033a1429"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 21:06:55.995336 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:55.995294    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtt5m\" (UniqueName: \"kubernetes.io/projected/3c077208-6deb-4a69-ab5c-4853033a1429-kube-api-access-gtt5m\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:06:55.995336 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:55.995326    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c077208-6deb-4a69-ab5c-4853033a1429-tls-certs\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:06:56.177103 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:56.177072    2571 generic.go:356] "Generic (PLEG): container finished" podID="3c077208-6deb-4a69-ab5c-4853033a1429" containerID="9c82ea245a356b8344723144530986dca1977a75eac56b7013d611db9dff8340" exitCode=0
Jul 01 21:06:56.177265 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:56.177136    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr"
Jul 01 21:06:56.177265 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:56.177142    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr" event={"ID":"3c077208-6deb-4a69-ab5c-4853033a1429","Type":"ContainerDied","Data":"9c82ea245a356b8344723144530986dca1977a75eac56b7013d611db9dff8340"}
Jul 01 21:06:56.177391 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:56.177266    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr" event={"ID":"3c077208-6deb-4a69-ab5c-4853033a1429","Type":"ContainerDied","Data":"c74a7145b21b7d63ce70333356212b3a484a8df4c89de49cb0704d1a34907611"}
Jul 01 21:06:56.177391 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:56.177286    2571 scope.go:117] "RemoveContainer" containerID="9c82ea245a356b8344723144530986dca1977a75eac56b7013d611db9dff8340"
Jul 01 21:06:56.186287 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:56.186271    2571 scope.go:117] "RemoveContainer" containerID="9c82ea245a356b8344723144530986dca1977a75eac56b7013d611db9dff8340"
Jul 01 21:06:56.186570 ip-10-0-140-104 kubenswrapper[2571]: E0701 21:06:56.186553    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c82ea245a356b8344723144530986dca1977a75eac56b7013d611db9dff8340\": container with ID starting with 9c82ea245a356b8344723144530986dca1977a75eac56b7013d611db9dff8340 not found: ID does not exist" containerID="9c82ea245a356b8344723144530986dca1977a75eac56b7013d611db9dff8340"
Jul 01 21:06:56.186653 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:56.186578    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c82ea245a356b8344723144530986dca1977a75eac56b7013d611db9dff8340"} err="failed to get container status \"9c82ea245a356b8344723144530986dca1977a75eac56b7013d611db9dff8340\": rpc error: code = NotFound desc = could not find container \"9c82ea245a356b8344723144530986dca1977a75eac56b7013d611db9dff8340\": container with ID starting with 9c82ea245a356b8344723144530986dca1977a75eac56b7013d611db9dff8340 not found: ID does not exist"
Jul 01 21:06:56.198615 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:56.198595    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr"]
Jul 01 21:06:56.205206 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:56.205184    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schednpfr"]
Jul 01 21:06:57.838859 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:06:57.838818    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c077208-6deb-4a69-ab5c-4853033a1429" path="/var/lib/kubelet/pods/3c077208-6deb-4a69-ab5c-4853033a1429/volumes"
Jul 01 21:07:15.861061 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:07:15.860971    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z"]
Jul 01 21:07:30.882116 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:07:30.882061    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z" podUID="30806680-6984-4127-ad3d-9994ba2909ff" containerName="main" containerID="cri-o://32ccd362ec939303491b5c602cefc4217b517bedaa66a6aeba9bed0a6daf4cd5" gracePeriod=45
Jul 01 21:07:31.125160 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:07:31.125139    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z"
Jul 01 21:07:31.253038 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:07:31.253016    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30806680-6984-4127-ad3d-9994ba2909ff-model-cache\") pod \"30806680-6984-4127-ad3d-9994ba2909ff\" (UID: \"30806680-6984-4127-ad3d-9994ba2909ff\") "
Jul 01 21:07:31.253184 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:07:31.253047    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30806680-6984-4127-ad3d-9994ba2909ff-tls-certs\") pod \"30806680-6984-4127-ad3d-9994ba2909ff\" (UID: \"30806680-6984-4127-ad3d-9994ba2909ff\") "
Jul 01 21:07:31.253184 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:07:31.253096    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30806680-6984-4127-ad3d-9994ba2909ff-tmp-dir\") pod \"30806680-6984-4127-ad3d-9994ba2909ff\" (UID: \"30806680-6984-4127-ad3d-9994ba2909ff\") "
Jul 01 21:07:31.253184 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:07:31.253114    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30806680-6984-4127-ad3d-9994ba2909ff-dshm\") pod \"30806680-6984-4127-ad3d-9994ba2909ff\" (UID: \"30806680-6984-4127-ad3d-9994ba2909ff\") "
Jul 01 21:07:31.253184 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:07:31.253130    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-62srw\" (UniqueName: \"kubernetes.io/projected/30806680-6984-4127-ad3d-9994ba2909ff-kube-api-access-62srw\") pod \"30806680-6984-4127-ad3d-9994ba2909ff\" (UID: \"30806680-6984-4127-ad3d-9994ba2909ff\") "
Jul 01 21:07:31.253184 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:07:31.253172    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30806680-6984-4127-ad3d-9994ba2909ff-home\") pod \"30806680-6984-4127-ad3d-9994ba2909ff\" (UID: \"30806680-6984-4127-ad3d-9994ba2909ff\") "
Jul 01 21:07:31.253438 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:07:31.253289    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30806680-6984-4127-ad3d-9994ba2909ff-model-cache" (OuterVolumeSpecName: "model-cache") pod "30806680-6984-4127-ad3d-9994ba2909ff" (UID: "30806680-6984-4127-ad3d-9994ba2909ff"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 21:07:31.253487 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:07:31.253398    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30806680-6984-4127-ad3d-9994ba2909ff-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "30806680-6984-4127-ad3d-9994ba2909ff" (UID: "30806680-6984-4127-ad3d-9994ba2909ff"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 21:07:31.253540 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:07:31.253496    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30806680-6984-4127-ad3d-9994ba2909ff-model-cache\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:07:31.253540 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:07:31.253503    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30806680-6984-4127-ad3d-9994ba2909ff-home" (OuterVolumeSpecName: "home") pod "30806680-6984-4127-ad3d-9994ba2909ff" (UID: "30806680-6984-4127-ad3d-9994ba2909ff"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 21:07:31.255271 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:07:31.255242    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30806680-6984-4127-ad3d-9994ba2909ff-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "30806680-6984-4127-ad3d-9994ba2909ff" (UID: "30806680-6984-4127-ad3d-9994ba2909ff"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 21:07:31.255358 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:07:31.255313    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30806680-6984-4127-ad3d-9994ba2909ff-dshm" (OuterVolumeSpecName: "dshm") pod "30806680-6984-4127-ad3d-9994ba2909ff" (UID: "30806680-6984-4127-ad3d-9994ba2909ff"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 21:07:31.255396 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:07:31.255377    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30806680-6984-4127-ad3d-9994ba2909ff-kube-api-access-62srw" (OuterVolumeSpecName: "kube-api-access-62srw") pod "30806680-6984-4127-ad3d-9994ba2909ff" (UID: "30806680-6984-4127-ad3d-9994ba2909ff"). InnerVolumeSpecName "kube-api-access-62srw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 21:07:31.326468 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:07:31.326440    2571 generic.go:356] "Generic (PLEG): container finished" podID="30806680-6984-4127-ad3d-9994ba2909ff" containerID="32ccd362ec939303491b5c602cefc4217b517bedaa66a6aeba9bed0a6daf4cd5" exitCode=0
Jul 01 21:07:31.326566 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:07:31.326481    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z"
Jul 01 21:07:31.326566 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:07:31.326527    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z" event={"ID":"30806680-6984-4127-ad3d-9994ba2909ff","Type":"ContainerDied","Data":"32ccd362ec939303491b5c602cefc4217b517bedaa66a6aeba9bed0a6daf4cd5"}
Jul 01 21:07:31.326667 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:07:31.326576    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z" event={"ID":"30806680-6984-4127-ad3d-9994ba2909ff","Type":"ContainerDied","Data":"f99e64afc7637ac8131dc0d3d0b4df952500112306022629f26d8aa702afbd48"}
Jul 01 21:07:31.326667 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:07:31.326598    2571 scope.go:117] "RemoveContainer" containerID="32ccd362ec939303491b5c602cefc4217b517bedaa66a6aeba9bed0a6daf4cd5"
Jul 01 21:07:31.334994 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:07:31.334980    2571 scope.go:117] "RemoveContainer" containerID="32ccd362ec939303491b5c602cefc4217b517bedaa66a6aeba9bed0a6daf4cd5"
Jul 01 21:07:31.335264 ip-10-0-140-104 kubenswrapper[2571]: E0701 21:07:31.335246    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32ccd362ec939303491b5c602cefc4217b517bedaa66a6aeba9bed0a6daf4cd5\": container with ID starting with 32ccd362ec939303491b5c602cefc4217b517bedaa66a6aeba9bed0a6daf4cd5 not found: ID does not exist" containerID="32ccd362ec939303491b5c602cefc4217b517bedaa66a6aeba9bed0a6daf4cd5"
Jul 01 21:07:31.335318 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:07:31.335272    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32ccd362ec939303491b5c602cefc4217b517bedaa66a6aeba9bed0a6daf4cd5"} err="failed to get container status \"32ccd362ec939303491b5c602cefc4217b517bedaa66a6aeba9bed0a6daf4cd5\": rpc error: code = NotFound desc = could not find container \"32ccd362ec939303491b5c602cefc4217b517bedaa66a6aeba9bed0a6daf4cd5\": container with ID starting with 32ccd362ec939303491b5c602cefc4217b517bedaa66a6aeba9bed0a6daf4cd5 not found: ID does not exist"
Jul 01 21:07:31.348297 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:07:31.348277    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z"]
Jul 01 21:07:31.354306 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:07:31.354285    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30806680-6984-4127-ad3d-9994ba2909ff-tls-certs\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:07:31.354394 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:07:31.354335    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30806680-6984-4127-ad3d-9994ba2909ff-tmp-dir\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:07:31.354394 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:07:31.354349    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30806680-6984-4127-ad3d-9994ba2909ff-dshm\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:07:31.354394 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:07:31.354362    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62srw\" (UniqueName: \"kubernetes.io/projected/30806680-6984-4127-ad3d-9994ba2909ff-kube-api-access-62srw\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:07:31.354394 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:07:31.354376    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30806680-6984-4127-ad3d-9994ba2909ff-home\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:07:31.354803 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:07:31.354788    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-kcm8z"]
Jul 01 21:07:31.842866 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:07:31.839488    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30806680-6984-4127-ad3d-9994ba2909ff" path="/var/lib/kubelet/pods/30806680-6984-4127-ad3d-9994ba2909ff/volumes"
Jul 01 21:08:26.673157 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:26.673107    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6"]
Jul 01 21:08:26.673651 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:26.673616    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c077208-6deb-4a69-ab5c-4853033a1429" containerName="main"
Jul 01 21:08:26.673704 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:26.673654    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c077208-6deb-4a69-ab5c-4853033a1429" containerName="main"
Jul 01 21:08:26.673704 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:26.673683    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30806680-6984-4127-ad3d-9994ba2909ff" containerName="main"
Jul 01 21:08:26.673704 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:26.673692    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="30806680-6984-4127-ad3d-9994ba2909ff" containerName="main"
Jul 01 21:08:26.673807 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:26.673781    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c077208-6deb-4a69-ab5c-4853033a1429" containerName="main"
Jul 01 21:08:26.673807 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:26.673796    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="30806680-6984-4127-ad3d-9994ba2909ff" containerName="main"
Jul 01 21:08:26.677287 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:26.677268    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6"
Jul 01 21:08:26.681613 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:26.681588    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv122f03714c5bdf915a2917fdf1262b98-kserve-self-signed-certs\""
Jul 01 21:08:26.681760 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:26.681722    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 01 21:08:26.681833 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:26.681590    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 01 21:08:26.682041 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:26.682012    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-76tz7\""
Jul 01 21:08:26.687231 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:26.687210    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6"]
Jul 01 21:08:26.829250 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:26.829208    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bab7dbe-5089-4d7c-9922-6be2f612a917-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6\" (UID: \"4bab7dbe-5089-4d7c-9922-6be2f612a917\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6"
Jul 01 21:08:26.829250 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:26.829254    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bab7dbe-5089-4d7c-9922-6be2f612a917-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6\" (UID: \"4bab7dbe-5089-4d7c-9922-6be2f612a917\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6"
Jul 01 21:08:26.829468 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:26.829295    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzmx7\" (UniqueName: \"kubernetes.io/projected/4bab7dbe-5089-4d7c-9922-6be2f612a917-kube-api-access-pzmx7\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6\" (UID: \"4bab7dbe-5089-4d7c-9922-6be2f612a917\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6"
Jul 01 21:08:26.829468 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:26.829319    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bab7dbe-5089-4d7c-9922-6be2f612a917-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6\" (UID: \"4bab7dbe-5089-4d7c-9922-6be2f612a917\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6"
Jul 01 21:08:26.829468 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:26.829363    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bab7dbe-5089-4d7c-9922-6be2f612a917-dshm\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6\" (UID: \"4bab7dbe-5089-4d7c-9922-6be2f612a917\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6"
Jul 01 21:08:26.829468 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:26.829399    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4bab7dbe-5089-4d7c-9922-6be2f612a917-home\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6\" (UID: \"4bab7dbe-5089-4d7c-9922-6be2f612a917\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6"
Jul 01 21:08:26.930929 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:26.930822    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bab7dbe-5089-4d7c-9922-6be2f612a917-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6\" (UID: \"4bab7dbe-5089-4d7c-9922-6be2f612a917\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6"
Jul 01 21:08:26.930929 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:26.930868    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bab7dbe-5089-4d7c-9922-6be2f612a917-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6\" (UID: \"4bab7dbe-5089-4d7c-9922-6be2f612a917\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6"
Jul 01 21:08:26.930929 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:26.930892    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzmx7\" (UniqueName: \"kubernetes.io/projected/4bab7dbe-5089-4d7c-9922-6be2f612a917-kube-api-access-pzmx7\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6\" (UID: \"4bab7dbe-5089-4d7c-9922-6be2f612a917\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6"
Jul 01 21:08:26.930929 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:26.930915    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bab7dbe-5089-4d7c-9922-6be2f612a917-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6\" (UID: \"4bab7dbe-5089-4d7c-9922-6be2f612a917\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6"
Jul 01 21:08:26.931263 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:26.930984    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bab7dbe-5089-4d7c-9922-6be2f612a917-dshm\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6\" (UID: \"4bab7dbe-5089-4d7c-9922-6be2f612a917\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6"
Jul 01 21:08:26.931263 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:26.931032    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4bab7dbe-5089-4d7c-9922-6be2f612a917-home\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6\" (UID: \"4bab7dbe-5089-4d7c-9922-6be2f612a917\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6"
Jul 01 21:08:26.931405 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:26.931376    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bab7dbe-5089-4d7c-9922-6be2f612a917-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6\" (UID: \"4bab7dbe-5089-4d7c-9922-6be2f612a917\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6"
Jul 01 21:08:26.931738 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:26.931707    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4bab7dbe-5089-4d7c-9922-6be2f612a917-home\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6\" (UID: \"4bab7dbe-5089-4d7c-9922-6be2f612a917\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6"
Jul 01 21:08:26.931876 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:26.931798    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bab7dbe-5089-4d7c-9922-6be2f612a917-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6\" (UID: \"4bab7dbe-5089-4d7c-9922-6be2f612a917\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6"
Jul 01 21:08:26.933680 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:26.933648    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bab7dbe-5089-4d7c-9922-6be2f612a917-dshm\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6\" (UID: \"4bab7dbe-5089-4d7c-9922-6be2f612a917\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6"
Jul 01 21:08:26.934287 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:26.934265    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bab7dbe-5089-4d7c-9922-6be2f612a917-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6\" (UID: \"4bab7dbe-5089-4d7c-9922-6be2f612a917\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6"
Jul 01 21:08:26.940166 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:26.940130    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzmx7\" (UniqueName: \"kubernetes.io/projected/4bab7dbe-5089-4d7c-9922-6be2f612a917-kube-api-access-pzmx7\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6\" (UID: \"4bab7dbe-5089-4d7c-9922-6be2f612a917\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6"
Jul 01 21:08:26.990058 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:26.990009    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6"
Jul 01 21:08:27.130488 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:27.130462    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6"]
Jul 01 21:08:27.132573 ip-10-0-140-104 kubenswrapper[2571]: W0701 21:08:27.132538    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bab7dbe_5089_4d7c_9922_6be2f612a917.slice/crio-e4d0ebc2a6ed6d0d45903aeb329fed57d248463d2030e0c05c74d3af739db2e9 WatchSource:0}: Error finding container e4d0ebc2a6ed6d0d45903aeb329fed57d248463d2030e0c05c74d3af739db2e9: Status 404 returned error can't find the container with id e4d0ebc2a6ed6d0d45903aeb329fed57d248463d2030e0c05c74d3af739db2e9
Jul 01 21:08:27.581785 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:27.581693    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6" event={"ID":"4bab7dbe-5089-4d7c-9922-6be2f612a917","Type":"ContainerStarted","Data":"19d03e1d57b312bcd9efdd9e972c9dad8f29ba26f71fe40c9b6a225f0d2daed8"}
Jul 01 21:08:27.581785 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:27.581745    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6" event={"ID":"4bab7dbe-5089-4d7c-9922-6be2f612a917","Type":"ContainerStarted","Data":"e4d0ebc2a6ed6d0d45903aeb329fed57d248463d2030e0c05c74d3af739db2e9"}
Jul 01 21:08:27.602845 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:27.602789    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6" podStartSLOduration=1.602772541 podStartE2EDuration="1.602772541s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 21:08:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 21:08:27.601112416 +0000 UTC m=+1326.113587598" watchObservedRunningTime="2026-07-01 21:08:27.602772541 +0000 UTC m=+1326.115247731"
Jul 01 21:08:27.990552 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:27.990503    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6"
Jul 01 21:08:36.990872 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:36.990838    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6"
Jul 01 21:08:37.003141 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:37.003117    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6"
Jul 01 21:08:37.639161 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:37.639135    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6"
Jul 01 21:08:39.196804 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:39.196768    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6"]
Jul 01 21:08:48.683217 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:48.683187    2571 generic.go:356] "Generic (PLEG): container finished" podID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13" containerID="4a59d2bd5cd10bb3cefa7b8eeb96217d17136090dccd2ed2e939a4d007c654e3" exitCode=137
Jul 01 21:08:48.683667 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:48.683263    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" event={"ID":"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13","Type":"ContainerDied","Data":"4a59d2bd5cd10bb3cefa7b8eeb96217d17136090dccd2ed2e939a4d007c654e3"}
Jul 01 21:08:48.683667 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:48.683324    2571 scope.go:117] "RemoveContainer" containerID="8b32b47e6634253200d19f5c9f1d0c62cfe74575698b0f0bc4576d9719693d6d"
Jul 01 21:08:48.683769 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:48.683751    2571 scope.go:117] "RemoveContainer" containerID="4a59d2bd5cd10bb3cefa7b8eeb96217d17136090dccd2ed2e939a4d007c654e3"
Jul 01 21:08:48.683959 ip-10-0-140-104 kubenswrapper[2571]: E0701 21:08:48.683922    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"istio-proxy\" with CrashLoopBackOff: \"back-off 10s restarting failed container=istio-proxy pod=openshift-ai-inference-openshift-default-655f7c58b4-57g8x_openshift-ingress(f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13)\"" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" podUID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13"
Jul 01 21:08:54.658109 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:54.658048    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6" podUID="4bab7dbe-5089-4d7c-9922-6be2f612a917" containerName="main" containerID="cri-o://19d03e1d57b312bcd9efdd9e972c9dad8f29ba26f71fe40c9b6a225f0d2daed8" gracePeriod=45
Jul 01 21:08:54.904802 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:54.904778    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6"
Jul 01 21:08:54.979293 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:54.979201    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bab7dbe-5089-4d7c-9922-6be2f612a917-tls-certs\") pod \"4bab7dbe-5089-4d7c-9922-6be2f612a917\" (UID: \"4bab7dbe-5089-4d7c-9922-6be2f612a917\") "
Jul 01 21:08:54.979437 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:54.979302    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bab7dbe-5089-4d7c-9922-6be2f612a917-dshm\") pod \"4bab7dbe-5089-4d7c-9922-6be2f612a917\" (UID: \"4bab7dbe-5089-4d7c-9922-6be2f612a917\") "
Jul 01 21:08:54.979437 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:54.979332    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4bab7dbe-5089-4d7c-9922-6be2f612a917-home\") pod \"4bab7dbe-5089-4d7c-9922-6be2f612a917\" (UID: \"4bab7dbe-5089-4d7c-9922-6be2f612a917\") "
Jul 01 21:08:54.979437 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:54.979370    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bab7dbe-5089-4d7c-9922-6be2f612a917-model-cache\") pod \"4bab7dbe-5089-4d7c-9922-6be2f612a917\" (UID: \"4bab7dbe-5089-4d7c-9922-6be2f612a917\") "
Jul 01 21:08:54.979437 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:54.979420    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pzmx7\" (UniqueName: \"kubernetes.io/projected/4bab7dbe-5089-4d7c-9922-6be2f612a917-kube-api-access-pzmx7\") pod \"4bab7dbe-5089-4d7c-9922-6be2f612a917\" (UID: \"4bab7dbe-5089-4d7c-9922-6be2f612a917\") "
Jul 01 21:08:54.979621 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:54.979454    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bab7dbe-5089-4d7c-9922-6be2f612a917-tmp-dir\") pod \"4bab7dbe-5089-4d7c-9922-6be2f612a917\" (UID: \"4bab7dbe-5089-4d7c-9922-6be2f612a917\") "
Jul 01 21:08:54.979714 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:54.979658    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bab7dbe-5089-4d7c-9922-6be2f612a917-home" (OuterVolumeSpecName: "home") pod "4bab7dbe-5089-4d7c-9922-6be2f612a917" (UID: "4bab7dbe-5089-4d7c-9922-6be2f612a917"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 21:08:54.979714 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:54.979697    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bab7dbe-5089-4d7c-9922-6be2f612a917-model-cache" (OuterVolumeSpecName: "model-cache") pod "4bab7dbe-5089-4d7c-9922-6be2f612a917" (UID: "4bab7dbe-5089-4d7c-9922-6be2f612a917"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 21:08:54.979835 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:54.979815    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4bab7dbe-5089-4d7c-9922-6be2f612a917-home\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:08:54.979898 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:54.979842    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bab7dbe-5089-4d7c-9922-6be2f612a917-model-cache\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:08:54.979898 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:54.979815    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bab7dbe-5089-4d7c-9922-6be2f612a917-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4bab7dbe-5089-4d7c-9922-6be2f612a917" (UID: "4bab7dbe-5089-4d7c-9922-6be2f612a917"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 21:08:54.981663 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:54.981617    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bab7dbe-5089-4d7c-9922-6be2f612a917-kube-api-access-pzmx7" (OuterVolumeSpecName: "kube-api-access-pzmx7") pod "4bab7dbe-5089-4d7c-9922-6be2f612a917" (UID: "4bab7dbe-5089-4d7c-9922-6be2f612a917"). InnerVolumeSpecName "kube-api-access-pzmx7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 21:08:54.981663 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:54.981652    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bab7dbe-5089-4d7c-9922-6be2f612a917-dshm" (OuterVolumeSpecName: "dshm") pod "4bab7dbe-5089-4d7c-9922-6be2f612a917" (UID: "4bab7dbe-5089-4d7c-9922-6be2f612a917"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 21:08:54.981810 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:54.981743    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4bab7dbe-5089-4d7c-9922-6be2f612a917-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4bab7dbe-5089-4d7c-9922-6be2f612a917" (UID: "4bab7dbe-5089-4d7c-9922-6be2f612a917"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 21:08:55.080907 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:55.080879    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bab7dbe-5089-4d7c-9922-6be2f612a917-tls-certs\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:08:55.080907 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:55.080903    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bab7dbe-5089-4d7c-9922-6be2f612a917-dshm\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:08:55.080907 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:55.080912    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzmx7\" (UniqueName: \"kubernetes.io/projected/4bab7dbe-5089-4d7c-9922-6be2f612a917-kube-api-access-pzmx7\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:08:55.081122 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:55.080921    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bab7dbe-5089-4d7c-9922-6be2f612a917-tmp-dir\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:08:55.720272 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:55.720227    2571 generic.go:356] "Generic (PLEG): container finished" podID="4bab7dbe-5089-4d7c-9922-6be2f612a917" containerID="19d03e1d57b312bcd9efdd9e972c9dad8f29ba26f71fe40c9b6a225f0d2daed8" exitCode=0
Jul 01 21:08:55.720776 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:55.720282    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6"
Jul 01 21:08:55.720776 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:55.720286    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6" event={"ID":"4bab7dbe-5089-4d7c-9922-6be2f612a917","Type":"ContainerDied","Data":"19d03e1d57b312bcd9efdd9e972c9dad8f29ba26f71fe40c9b6a225f0d2daed8"}
Jul 01 21:08:55.720776 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:55.720343    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6" event={"ID":"4bab7dbe-5089-4d7c-9922-6be2f612a917","Type":"ContainerDied","Data":"e4d0ebc2a6ed6d0d45903aeb329fed57d248463d2030e0c05c74d3af739db2e9"}
Jul 01 21:08:55.720776 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:55.720369    2571 scope.go:117] "RemoveContainer" containerID="19d03e1d57b312bcd9efdd9e972c9dad8f29ba26f71fe40c9b6a225f0d2daed8"
Jul 01 21:08:55.731007 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:55.730989    2571 scope.go:117] "RemoveContainer" containerID="19d03e1d57b312bcd9efdd9e972c9dad8f29ba26f71fe40c9b6a225f0d2daed8"
Jul 01 21:08:55.731342 ip-10-0-140-104 kubenswrapper[2571]: E0701 21:08:55.731321    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19d03e1d57b312bcd9efdd9e972c9dad8f29ba26f71fe40c9b6a225f0d2daed8\": container with ID starting with 19d03e1d57b312bcd9efdd9e972c9dad8f29ba26f71fe40c9b6a225f0d2daed8 not found: ID does not exist" containerID="19d03e1d57b312bcd9efdd9e972c9dad8f29ba26f71fe40c9b6a225f0d2daed8"
Jul 01 21:08:55.731380 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:55.731352    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19d03e1d57b312bcd9efdd9e972c9dad8f29ba26f71fe40c9b6a225f0d2daed8"} err="failed to get container status \"19d03e1d57b312bcd9efdd9e972c9dad8f29ba26f71fe40c9b6a225f0d2daed8\": rpc error: code = NotFound desc = could not find container \"19d03e1d57b312bcd9efdd9e972c9dad8f29ba26f71fe40c9b6a225f0d2daed8\": container with ID starting with 19d03e1d57b312bcd9efdd9e972c9dad8f29ba26f71fe40c9b6a225f0d2daed8 not found: ID does not exist"
Jul 01 21:08:55.743969 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:55.743931    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6"]
Jul 01 21:08:55.747575 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:55.747554    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d48hqx6"]
Jul 01 21:08:55.839187 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:08:55.839161    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4bab7dbe-5089-4d7c-9922-6be2f612a917" path="/var/lib/kubelet/pods/4bab7dbe-5089-4d7c-9922-6be2f612a917/volumes"
Jul 01 21:09:00.728467 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:09:00.728433    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 21:09:00.728956 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:09:00.728931    2571 scope.go:117] "RemoveContainer" containerID="4a59d2bd5cd10bb3cefa7b8eeb96217d17136090dccd2ed2e939a4d007c654e3"
Jul 01 21:09:00.730022 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:09:00.729993    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 01 21:09:00.730083 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:09:00.730062    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 01 21:09:00.730117 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:09:00.730094    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 01 21:09:01.750850 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:09:01.750818    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" event={"ID":"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13","Type":"ContainerStarted","Data":"f95c0ecea7ac8cd5ed4021680c8cb1a480340d51a90e4ac57593e89b55fb3421"}
Jul 01 21:09:02.722954 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:09:02.722921    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 21:09:02.723875 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:09:02.723853    2571 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-655f7c58b4-57g8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body=
Jul 01 21:09:02.723981 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:09:02.723913    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" podUID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused"
Jul 01 21:09:03.723325 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:09:03.723290    2571 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-655f7c58b4-57g8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body=
Jul 01 21:09:03.723798 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:09:03.723346    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" podUID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused"
Jul 01 21:09:04.722866 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:09:04.722827    2571 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-655f7c58b4-57g8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body=
Jul 01 21:09:04.723070 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:09:04.722888    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" podUID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused"
Jul 01 21:09:05.723217 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:09:05.723182    2571 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-655f7c58b4-57g8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body=
Jul 01 21:09:05.723610 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:09:05.723241    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" podUID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused"
Jul 01 21:09:06.723349 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:09:06.723295    2571 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-655f7c58b4-57g8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body=
Jul 01 21:09:06.723865 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:09:06.723355    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" podUID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused"
Jul 01 21:09:08.538584 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:09:08.538543    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 21:09:08.539082 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:09:08.538950    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 21:09:08.539773 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:09:08.539753    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 21:11:25.395474 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:11:25.395447    2571 generic.go:356] "Generic (PLEG): container finished" podID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13" containerID="f95c0ecea7ac8cd5ed4021680c8cb1a480340d51a90e4ac57593e89b55fb3421" exitCode=137
Jul 01 21:11:25.395876 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:11:25.395518    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" event={"ID":"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13","Type":"ContainerDied","Data":"f95c0ecea7ac8cd5ed4021680c8cb1a480340d51a90e4ac57593e89b55fb3421"}
Jul 01 21:11:25.395876 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:11:25.395578    2571 scope.go:117] "RemoveContainer" containerID="4a59d2bd5cd10bb3cefa7b8eeb96217d17136090dccd2ed2e939a4d007c654e3"
Jul 01 21:11:25.395966 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:11:25.395938    2571 scope.go:117] "RemoveContainer" containerID="f95c0ecea7ac8cd5ed4021680c8cb1a480340d51a90e4ac57593e89b55fb3421"
Jul 01 21:11:25.396189 ip-10-0-140-104 kubenswrapper[2571]: E0701 21:11:25.396163    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"istio-proxy\" with CrashLoopBackOff: \"back-off 10s restarting failed container=istio-proxy pod=openshift-ai-inference-openshift-default-655f7c58b4-57g8x_openshift-ingress(f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13)\"" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" podUID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13"
Jul 01 21:11:25.401193 ip-10-0-140-104 kubenswrapper[2571]: E0701 21:11:25.401170    2571 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1d4abbb_fa80_4dff_8760_4c5e5f0d7c13.slice/crio-conmon-f95c0ecea7ac8cd5ed4021680c8cb1a480340d51a90e4ac57593e89b55fb3421.scope\": RecentStats: unable to find data in memory cache]"
Jul 01 21:11:38.539460 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:11:38.539418    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 21:11:38.539922 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:11:38.539911    2571 scope.go:117] "RemoveContainer" containerID="f95c0ecea7ac8cd5ed4021680c8cb1a480340d51a90e4ac57593e89b55fb3421"
Jul 01 21:11:38.541080 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:11:38.541047    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 01 21:11:38.541176 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:11:38.541138    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 01 21:11:38.541235 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:11:38.541186    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 01 21:11:39.464281 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:11:39.464243    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" event={"ID":"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13","Type":"ContainerStarted","Data":"c71445b5ddf9bcfc1081317b4de334650e5c94af28db02a35fd0d6cea7cd6317"}
Jul 01 21:11:39.723090 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:11:39.723000    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 21:11:39.723854 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:11:39.723831    2571 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-655f7c58b4-57g8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body=
Jul 01 21:11:39.723913 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:11:39.723883    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" podUID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused"
Jul 01 21:11:40.723209 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:11:40.723173    2571 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-655f7c58b4-57g8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body=
Jul 01 21:11:40.723584 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:11:40.723227    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" podUID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused"
Jul 01 21:11:41.723409 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:11:41.723380    2571 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-655f7c58b4-57g8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body=
Jul 01 21:11:41.723766 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:11:41.723430    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" podUID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused"
Jul 01 21:11:42.722966 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:11:42.722888    2571 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-655f7c58b4-57g8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body=
Jul 01 21:11:42.722966 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:11:42.722945    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" podUID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused"
Jul 01 21:11:43.723473 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:11:43.723434    2571 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-655f7c58b4-57g8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body=
Jul 01 21:11:43.723900 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:11:43.723487    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" podUID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused"
Jul 01 21:11:44.723004 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:11:44.722966    2571 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-655f7c58b4-57g8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body=
Jul 01 21:11:44.723187 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:11:44.723023    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" podUID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused"
Jul 01 21:11:46.448430 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:11:46.448400    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 21:11:46.448953 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:11:46.448753    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 21:11:46.449529 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:11:46.449508    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 21:12:24.690398 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:12:24.690361    2571 generic.go:356] "Generic (PLEG): container finished" podID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13" containerID="c71445b5ddf9bcfc1081317b4de334650e5c94af28db02a35fd0d6cea7cd6317" exitCode=137
Jul 01 21:12:24.690892 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:12:24.690412    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" event={"ID":"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13","Type":"ContainerDied","Data":"c71445b5ddf9bcfc1081317b4de334650e5c94af28db02a35fd0d6cea7cd6317"}
Jul 01 21:12:24.690892 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:12:24.690478    2571 scope.go:117] "RemoveContainer" containerID="f95c0ecea7ac8cd5ed4021680c8cb1a480340d51a90e4ac57593e89b55fb3421"
Jul 01 21:12:24.690892 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:12:24.690866    2571 scope.go:117] "RemoveContainer" containerID="c71445b5ddf9bcfc1081317b4de334650e5c94af28db02a35fd0d6cea7cd6317"
Jul 01 21:12:24.691094 ip-10-0-140-104 kubenswrapper[2571]: E0701 21:12:24.691069    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"istio-proxy\" with CrashLoopBackOff: \"back-off 20s restarting failed container=istio-proxy pod=openshift-ai-inference-openshift-default-655f7c58b4-57g8x_openshift-ingress(f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13)\"" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" podUID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13"
Jul 01 21:12:31.449299 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:12:31.449260    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 21:12:31.449714 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:12:31.449667    2571 scope.go:117] "RemoveContainer" containerID="c71445b5ddf9bcfc1081317b4de334650e5c94af28db02a35fd0d6cea7cd6317"
Jul 01 21:12:31.449881 ip-10-0-140-104 kubenswrapper[2571]: E0701 21:12:31.449858    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"istio-proxy\" with CrashLoopBackOff: \"back-off 20s restarting failed container=istio-proxy pod=openshift-ai-inference-openshift-default-655f7c58b4-57g8x_openshift-ingress(f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13)\"" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" podUID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13"
Jul 01 21:12:42.834202 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:12:42.834168    2571 scope.go:117] "RemoveContainer" containerID="c71445b5ddf9bcfc1081317b4de334650e5c94af28db02a35fd0d6cea7cd6317"
Jul 01 21:12:42.834599 ip-10-0-140-104 kubenswrapper[2571]: E0701 21:12:42.834352    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"istio-proxy\" with CrashLoopBackOff: \"back-off 20s restarting failed container=istio-proxy pod=openshift-ai-inference-openshift-default-655f7c58b4-57g8x_openshift-ingress(f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13)\"" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" podUID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13"
Jul 01 21:12:57.834149 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:12:57.834118    2571 scope.go:117] "RemoveContainer" containerID="c71445b5ddf9bcfc1081317b4de334650e5c94af28db02a35fd0d6cea7cd6317"
Jul 01 21:12:57.835390 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:12:57.835347    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 01 21:12:57.835473 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:12:57.835453    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 01 21:12:57.835538 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:12:57.835514    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 01 21:12:58.849949 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:12:58.849915    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" event={"ID":"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13","Type":"ContainerStarted","Data":"dfb41cc2a4c264ca1199e6658fb2d4b84e508515ab2af334913b5184552d4de3"}
Jul 01 21:12:59.722931 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:12:59.722898    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 21:12:59.724016 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:12:59.723993    2571 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-655f7c58b4-57g8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body=
Jul 01 21:12:59.724123 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:12:59.724039    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" podUID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused"
Jul 01 21:13:00.723000 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:13:00.722963    2571 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-655f7c58b4-57g8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body=
Jul 01 21:13:00.723373 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:13:00.723015    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" podUID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused"
Jul 01 21:13:01.449179 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:13:01.449145    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 21:13:01.722967 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:13:01.722937    2571 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-655f7c58b4-57g8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body=
Jul 01 21:13:01.723131 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:13:01.722991    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" podUID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused"
Jul 01 21:13:02.723231 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:13:02.723198    2571 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-655f7c58b4-57g8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body=
Jul 01 21:13:02.723589 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:13:02.723257    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" podUID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused"
Jul 01 21:13:03.723431 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:13:03.723396    2571 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-655f7c58b4-57g8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body=
Jul 01 21:13:03.723826 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:13:03.723451    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" podUID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused"
Jul 01 21:13:05.375148 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:13:05.375113    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 21:13:05.376523 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:13:05.376500    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 21:14:52.530839 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:14:52.530729    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz"]
Jul 01 21:14:52.533145 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:14:52.531091    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bab7dbe-5089-4d7c-9922-6be2f612a917" containerName="main"
Jul 01 21:14:52.533145 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:14:52.531101    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bab7dbe-5089-4d7c-9922-6be2f612a917" containerName="main"
Jul 01 21:14:52.533145 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:14:52.531154    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bab7dbe-5089-4d7c-9922-6be2f612a917" containerName="main"
Jul 01 21:14:52.534052 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:14:52.534035    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz"
Jul 01 21:14:52.538187 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:14:52.538156    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 01 21:14:52.538894 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:14:52.538862    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 01 21:14:52.538894 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:14:52.538880    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv8bf079eb6eda4debfb4ef5bb7817824c-kserve-self-signed-certs\""
Jul 01 21:14:52.539250 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:14:52.538927    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-pvc-router-manage-59b9d263-epp-sa-dockercfg-22bzr\""
Jul 01 21:14:52.539250 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:14:52.539017    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-76tz7\""
Jul 01 21:14:52.555395 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:14:52.555359    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz"]
Jul 01 21:14:52.717990 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:14:52.717940    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3bdf6e10-6de1-4906-9f43-a4d77900f38a-tls-certs\") pod \"llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz\" (UID: \"3bdf6e10-6de1-4906-9f43-a4d77900f38a\") " pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz"
Jul 01 21:14:52.718200 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:14:52.718012    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/3bdf6e10-6de1-4906-9f43-a4d77900f38a-tokenizer-uds\") pod \"llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz\" (UID: \"3bdf6e10-6de1-4906-9f43-a4d77900f38a\") " pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz"
Jul 01 21:14:52.718200 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:14:52.718071    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6rvs\" (UniqueName: \"kubernetes.io/projected/3bdf6e10-6de1-4906-9f43-a4d77900f38a-kube-api-access-n6rvs\") pod \"llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz\" (UID: \"3bdf6e10-6de1-4906-9f43-a4d77900f38a\") " pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz"
Jul 01 21:14:52.819026 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:14:52.818915    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3bdf6e10-6de1-4906-9f43-a4d77900f38a-tls-certs\") pod \"llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz\" (UID: \"3bdf6e10-6de1-4906-9f43-a4d77900f38a\") " pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz"
Jul 01 21:14:52.819026 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:14:52.818978    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/3bdf6e10-6de1-4906-9f43-a4d77900f38a-tokenizer-uds\") pod \"llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz\" (UID: \"3bdf6e10-6de1-4906-9f43-a4d77900f38a\") " pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz"
Jul 01 21:14:52.819286 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:14:52.819031    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6rvs\" (UniqueName: \"kubernetes.io/projected/3bdf6e10-6de1-4906-9f43-a4d77900f38a-kube-api-access-n6rvs\") pod \"llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz\" (UID: \"3bdf6e10-6de1-4906-9f43-a4d77900f38a\") " pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz"
Jul 01 21:14:52.819361 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:14:52.819339    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/3bdf6e10-6de1-4906-9f43-a4d77900f38a-tokenizer-uds\") pod \"llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz\" (UID: \"3bdf6e10-6de1-4906-9f43-a4d77900f38a\") " pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz"
Jul 01 21:14:52.821663 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:14:52.821642    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3bdf6e10-6de1-4906-9f43-a4d77900f38a-tls-certs\") pod \"llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz\" (UID: \"3bdf6e10-6de1-4906-9f43-a4d77900f38a\") " pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz"
Jul 01 21:14:52.836155 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:14:52.836117    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6rvs\" (UniqueName: \"kubernetes.io/projected/3bdf6e10-6de1-4906-9f43-a4d77900f38a-kube-api-access-n6rvs\") pod \"llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz\" (UID: \"3bdf6e10-6de1-4906-9f43-a4d77900f38a\") " pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz"
Jul 01 21:14:52.843983 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:14:52.843955    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz"
Jul 01 21:14:52.993830 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:14:52.993699    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz"]
Jul 01 21:14:52.996660 ip-10-0-140-104 kubenswrapper[2571]: W0701 21:14:52.996613    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bdf6e10_6de1_4906_9f43_a4d77900f38a.slice/crio-f9fd4bcd492d7d5dc66ee8398bdc7e9509e253e4a475918edf5d496e39d80f9a WatchSource:0}: Error finding container f9fd4bcd492d7d5dc66ee8398bdc7e9509e253e4a475918edf5d496e39d80f9a: Status 404 returned error can't find the container with id f9fd4bcd492d7d5dc66ee8398bdc7e9509e253e4a475918edf5d496e39d80f9a
Jul 01 21:14:53.367584 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:14:53.367550    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz" event={"ID":"3bdf6e10-6de1-4906-9f43-a4d77900f38a","Type":"ContainerStarted","Data":"bffa1bc981e250a5866da89994ed3aebf8a47e7edebf7a89e0525befc5043dc7"}
Jul 01 21:14:53.367855 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:14:53.367609    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz" event={"ID":"3bdf6e10-6de1-4906-9f43-a4d77900f38a","Type":"ContainerStarted","Data":"f9fd4bcd492d7d5dc66ee8398bdc7e9509e253e4a475918edf5d496e39d80f9a"}
Jul 01 21:14:53.367855 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:14:53.367812    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz"
Jul 01 21:14:53.394838 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:14:53.392458    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz" podStartSLOduration=1.392432877 podStartE2EDuration="1.392432877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 21:14:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 21:14:53.392089583 +0000 UTC m=+1711.904564771" watchObservedRunningTime="2026-07-01 21:14:53.392432877 +0000 UTC m=+1711.904908067"
Jul 01 21:15:24.374912 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:15:24.374884    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz"
Jul 01 21:17:24.461764 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:17:24.461730    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz"]
Jul 01 21:17:24.470927 ip-10-0-140-104 kubenswrapper[2571]: E0701 21:17:24.470893    2571 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 01 21:17:24.470927 ip-10-0-140-104 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 01 21:17:24.470927 ip-10-0-140-104 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 01 21:17:24.470927 ip-10-0-140-104 kubenswrapper[2571]:  > containerID="bffa1bc981e250a5866da89994ed3aebf8a47e7edebf7a89e0525befc5043dc7" cmd=["/bin/sleep","15"]
Jul 01 21:17:24.471181 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:17:24.470937    2571 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz" message="" err=<
Jul 01 21:17:24.471181 ip-10-0-140-104 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 01 21:17:24.471181 ip-10-0-140-104 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 01 21:17:24.471181 ip-10-0-140-104 kubenswrapper[2571]:  >
Jul 01 21:17:24.471181 ip-10-0-140-104 kubenswrapper[2571]: E0701 21:17:24.470950    2571 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 01 21:17:24.471181 ip-10-0-140-104 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 01 21:17:24.471181 ip-10-0-140-104 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 01 21:17:24.471181 ip-10-0-140-104 kubenswrapper[2571]:  > pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz" podUID="3bdf6e10-6de1-4906-9f43-a4d77900f38a" containerName="main" containerID="cri-o://bffa1bc981e250a5866da89994ed3aebf8a47e7edebf7a89e0525befc5043dc7"
Jul 01 21:17:24.471181 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:17:24.470991    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz" podUID="3bdf6e10-6de1-4906-9f43-a4d77900f38a" containerName="main" containerID="cri-o://bffa1bc981e250a5866da89994ed3aebf8a47e7edebf7a89e0525befc5043dc7" gracePeriod=60
Jul 01 21:17:24.719991 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:17:24.719939    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz"
Jul 01 21:17:24.822505 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:17:24.822481    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/3bdf6e10-6de1-4906-9f43-a4d77900f38a-tokenizer-uds\") pod \"3bdf6e10-6de1-4906-9f43-a4d77900f38a\" (UID: \"3bdf6e10-6de1-4906-9f43-a4d77900f38a\") "
Jul 01 21:17:24.822652 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:17:24.822511    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6rvs\" (UniqueName: \"kubernetes.io/projected/3bdf6e10-6de1-4906-9f43-a4d77900f38a-kube-api-access-n6rvs\") pod \"3bdf6e10-6de1-4906-9f43-a4d77900f38a\" (UID: \"3bdf6e10-6de1-4906-9f43-a4d77900f38a\") "
Jul 01 21:17:24.822652 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:17:24.822560    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3bdf6e10-6de1-4906-9f43-a4d77900f38a-tls-certs\") pod \"3bdf6e10-6de1-4906-9f43-a4d77900f38a\" (UID: \"3bdf6e10-6de1-4906-9f43-a4d77900f38a\") "
Jul 01 21:17:24.822797 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:17:24.822774    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bdf6e10-6de1-4906-9f43-a4d77900f38a-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "3bdf6e10-6de1-4906-9f43-a4d77900f38a" (UID: "3bdf6e10-6de1-4906-9f43-a4d77900f38a"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 21:17:24.822875 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:17:24.822854    2571 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/3bdf6e10-6de1-4906-9f43-a4d77900f38a-tokenizer-uds\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:17:24.824698 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:17:24.824678    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bdf6e10-6de1-4906-9f43-a4d77900f38a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3bdf6e10-6de1-4906-9f43-a4d77900f38a" (UID: "3bdf6e10-6de1-4906-9f43-a4d77900f38a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 21:17:24.824821 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:17:24.824806    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bdf6e10-6de1-4906-9f43-a4d77900f38a-kube-api-access-n6rvs" (OuterVolumeSpecName: "kube-api-access-n6rvs") pod "3bdf6e10-6de1-4906-9f43-a4d77900f38a" (UID: "3bdf6e10-6de1-4906-9f43-a4d77900f38a"). InnerVolumeSpecName "kube-api-access-n6rvs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 21:17:24.923333 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:17:24.923308    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6rvs\" (UniqueName: \"kubernetes.io/projected/3bdf6e10-6de1-4906-9f43-a4d77900f38a-kube-api-access-n6rvs\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:17:24.923417 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:17:24.923333    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3bdf6e10-6de1-4906-9f43-a4d77900f38a-tls-certs\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:17:25.016560 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:17:25.016508    2571 generic.go:356] "Generic (PLEG): container finished" podID="3bdf6e10-6de1-4906-9f43-a4d77900f38a" containerID="bffa1bc981e250a5866da89994ed3aebf8a47e7edebf7a89e0525befc5043dc7" exitCode=0
Jul 01 21:17:25.016560 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:17:25.016543    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz"
Jul 01 21:17:25.016690 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:17:25.016586    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz" event={"ID":"3bdf6e10-6de1-4906-9f43-a4d77900f38a","Type":"ContainerDied","Data":"bffa1bc981e250a5866da89994ed3aebf8a47e7edebf7a89e0525befc5043dc7"}
Jul 01 21:17:25.016690 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:17:25.016619    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz" event={"ID":"3bdf6e10-6de1-4906-9f43-a4d77900f38a","Type":"ContainerDied","Data":"f9fd4bcd492d7d5dc66ee8398bdc7e9509e253e4a475918edf5d496e39d80f9a"}
Jul 01 21:17:25.016690 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:17:25.016662    2571 scope.go:117] "RemoveContainer" containerID="bffa1bc981e250a5866da89994ed3aebf8a47e7edebf7a89e0525befc5043dc7"
Jul 01 21:17:25.030695 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:17:25.030672    2571 scope.go:117] "RemoveContainer" containerID="bffa1bc981e250a5866da89994ed3aebf8a47e7edebf7a89e0525befc5043dc7"
Jul 01 21:17:25.030988 ip-10-0-140-104 kubenswrapper[2571]: E0701 21:17:25.030971    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bffa1bc981e250a5866da89994ed3aebf8a47e7edebf7a89e0525befc5043dc7\": container with ID starting with bffa1bc981e250a5866da89994ed3aebf8a47e7edebf7a89e0525befc5043dc7 not found: ID does not exist" containerID="bffa1bc981e250a5866da89994ed3aebf8a47e7edebf7a89e0525befc5043dc7"
Jul 01 21:17:25.031071 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:17:25.030995    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bffa1bc981e250a5866da89994ed3aebf8a47e7edebf7a89e0525befc5043dc7"} err="failed to get container status \"bffa1bc981e250a5866da89994ed3aebf8a47e7edebf7a89e0525befc5043dc7\": rpc error: code = NotFound desc = could not find container \"bffa1bc981e250a5866da89994ed3aebf8a47e7edebf7a89e0525befc5043dc7\": container with ID starting with bffa1bc981e250a5866da89994ed3aebf8a47e7edebf7a89e0525befc5043dc7 not found: ID does not exist"
Jul 01 21:17:25.043750 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:17:25.043725    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz"]
Jul 01 21:17:25.047517 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:17:25.047496    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schejh6wz"]
Jul 01 21:17:25.838365 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:17:25.838334    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3bdf6e10-6de1-4906-9f43-a4d77900f38a" path="/var/lib/kubelet/pods/3bdf6e10-6de1-4906-9f43-a4d77900f38a/volumes"
Jul 01 21:18:00.102752 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:18:00.102671    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s"]
Jul 01 21:18:00.103177 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:18:00.102972    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bdf6e10-6de1-4906-9f43-a4d77900f38a" containerName="main"
Jul 01 21:18:00.103177 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:18:00.102982    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bdf6e10-6de1-4906-9f43-a4d77900f38a" containerName="main"
Jul 01 21:18:00.103177 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:18:00.103051    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bdf6e10-6de1-4906-9f43-a4d77900f38a" containerName="main"
Jul 01 21:18:00.105916 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:18:00.105898    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s"
Jul 01 21:18:00.118722 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:18:00.118689    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 01 21:18:00.118722 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:18:00.118703    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-pvc-router-manage-e8706282-epp-sa-dockercfg-v5frm\""
Jul 01 21:18:00.118944 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:18:00.118733    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 01 21:18:00.118944 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:18:00.118694    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvb19f98874e050eec8ca94d49676113f0-kserve-self-signed-certs\""
Jul 01 21:18:00.118944 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:18:00.118689    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-76tz7\""
Jul 01 21:18:00.133244 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:18:00.133222    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s"]
Jul 01 21:18:00.269483 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:18:00.269456    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bk7qr\" (UniqueName: \"kubernetes.io/projected/25bc1467-1a07-4879-a7dd-4eccbaaa3bae-kube-api-access-bk7qr\") pod \"llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s\" (UID: \"25bc1467-1a07-4879-a7dd-4eccbaaa3bae\") " pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s"
Jul 01 21:18:00.269615 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:18:00.269491    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/25bc1467-1a07-4879-a7dd-4eccbaaa3bae-tokenizer-uds\") pod \"llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s\" (UID: \"25bc1467-1a07-4879-a7dd-4eccbaaa3bae\") " pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s"
Jul 01 21:18:00.269615 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:18:00.269563    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25bc1467-1a07-4879-a7dd-4eccbaaa3bae-tls-certs\") pod \"llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s\" (UID: \"25bc1467-1a07-4879-a7dd-4eccbaaa3bae\") " pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s"
Jul 01 21:18:00.370132 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:18:00.370075    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk7qr\" (UniqueName: \"kubernetes.io/projected/25bc1467-1a07-4879-a7dd-4eccbaaa3bae-kube-api-access-bk7qr\") pod \"llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s\" (UID: \"25bc1467-1a07-4879-a7dd-4eccbaaa3bae\") " pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s"
Jul 01 21:18:00.370132 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:18:00.370108    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/25bc1467-1a07-4879-a7dd-4eccbaaa3bae-tokenizer-uds\") pod \"llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s\" (UID: \"25bc1467-1a07-4879-a7dd-4eccbaaa3bae\") " pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s"
Jul 01 21:18:00.370270 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:18:00.370158    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25bc1467-1a07-4879-a7dd-4eccbaaa3bae-tls-certs\") pod \"llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s\" (UID: \"25bc1467-1a07-4879-a7dd-4eccbaaa3bae\") " pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s"
Jul 01 21:18:00.370488 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:18:00.370473    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/25bc1467-1a07-4879-a7dd-4eccbaaa3bae-tokenizer-uds\") pod \"llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s\" (UID: \"25bc1467-1a07-4879-a7dd-4eccbaaa3bae\") " pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s"
Jul 01 21:18:00.372603 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:18:00.372585    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25bc1467-1a07-4879-a7dd-4eccbaaa3bae-tls-certs\") pod \"llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s\" (UID: \"25bc1467-1a07-4879-a7dd-4eccbaaa3bae\") " pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s"
Jul 01 21:18:00.379441 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:18:00.379420    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bk7qr\" (UniqueName: \"kubernetes.io/projected/25bc1467-1a07-4879-a7dd-4eccbaaa3bae-kube-api-access-bk7qr\") pod \"llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s\" (UID: \"25bc1467-1a07-4879-a7dd-4eccbaaa3bae\") " pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s"
Jul 01 21:18:00.414233 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:18:00.414205    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s"
Jul 01 21:18:00.546334 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:18:00.546303    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s"]
Jul 01 21:18:00.547238 ip-10-0-140-104 kubenswrapper[2571]: W0701 21:18:00.547214    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25bc1467_1a07_4879_a7dd_4eccbaaa3bae.slice/crio-a6faf630f6fb2e669ad131b36c5b13e0d15d9eabc8dbd90fde7d493baa2e08af WatchSource:0}: Error finding container a6faf630f6fb2e669ad131b36c5b13e0d15d9eabc8dbd90fde7d493baa2e08af: Status 404 returned error can't find the container with id a6faf630f6fb2e669ad131b36c5b13e0d15d9eabc8dbd90fde7d493baa2e08af
Jul 01 21:18:01.169738 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:18:01.169703    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s" event={"ID":"25bc1467-1a07-4879-a7dd-4eccbaaa3bae","Type":"ContainerStarted","Data":"7adc33f1aae5553f629a32a11b9afb4310e9fdedb27da5ba4fd83f31a02e9bc2"}
Jul 01 21:18:01.170078 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:18:01.169754    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s" event={"ID":"25bc1467-1a07-4879-a7dd-4eccbaaa3bae","Type":"ContainerStarted","Data":"a6faf630f6fb2e669ad131b36c5b13e0d15d9eabc8dbd90fde7d493baa2e08af"}
Jul 01 21:18:01.170078 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:18:01.169824    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s"
Jul 01 21:18:01.196908 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:18:01.196864    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s" podStartSLOduration=1.196851015 podStartE2EDuration="1.196851015s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 21:18:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 21:18:01.196724359 +0000 UTC m=+1899.709199558" watchObservedRunningTime="2026-07-01 21:18:01.196851015 +0000 UTC m=+1899.709326227"
Jul 01 21:18:32.175321 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:18:32.175293    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s"
Jul 01 21:21:27.850571 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:27.850541    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s"]
Jul 01 21:21:27.859492 ip-10-0-140-104 kubenswrapper[2571]: E0701 21:21:27.859444    2571 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 01 21:21:27.859492 ip-10-0-140-104 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 01 21:21:27.859492 ip-10-0-140-104 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 01 21:21:27.859492 ip-10-0-140-104 kubenswrapper[2571]:  > containerID="7adc33f1aae5553f629a32a11b9afb4310e9fdedb27da5ba4fd83f31a02e9bc2" cmd=["/bin/sleep","15"]
Jul 01 21:21:27.859757 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:27.859512    2571 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s" message="" err=<
Jul 01 21:21:27.859757 ip-10-0-140-104 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 01 21:21:27.859757 ip-10-0-140-104 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 01 21:21:27.859757 ip-10-0-140-104 kubenswrapper[2571]:  >
Jul 01 21:21:27.859757 ip-10-0-140-104 kubenswrapper[2571]: E0701 21:21:27.859529    2571 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 01 21:21:27.859757 ip-10-0-140-104 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 01 21:21:27.859757 ip-10-0-140-104 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 01 21:21:27.859757 ip-10-0-140-104 kubenswrapper[2571]:  > pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s" podUID="25bc1467-1a07-4879-a7dd-4eccbaaa3bae" containerName="main" containerID="cri-o://7adc33f1aae5553f629a32a11b9afb4310e9fdedb27da5ba4fd83f31a02e9bc2"
Jul 01 21:21:27.859757 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:27.859578    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s" podUID="25bc1467-1a07-4879-a7dd-4eccbaaa3bae" containerName="main" containerID="cri-o://7adc33f1aae5553f629a32a11b9afb4310e9fdedb27da5ba4fd83f31a02e9bc2" gracePeriod=60
Jul 01 21:21:28.059166 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:28.059139    2571 generic.go:356] "Generic (PLEG): container finished" podID="25bc1467-1a07-4879-a7dd-4eccbaaa3bae" containerID="7adc33f1aae5553f629a32a11b9afb4310e9fdedb27da5ba4fd83f31a02e9bc2" exitCode=0
Jul 01 21:21:28.059285 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:28.059191    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s" event={"ID":"25bc1467-1a07-4879-a7dd-4eccbaaa3bae","Type":"ContainerDied","Data":"7adc33f1aae5553f629a32a11b9afb4310e9fdedb27da5ba4fd83f31a02e9bc2"}
Jul 01 21:21:28.099726 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:28.099708    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s"
Jul 01 21:21:28.283691 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:28.283653    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bk7qr\" (UniqueName: \"kubernetes.io/projected/25bc1467-1a07-4879-a7dd-4eccbaaa3bae-kube-api-access-bk7qr\") pod \"25bc1467-1a07-4879-a7dd-4eccbaaa3bae\" (UID: \"25bc1467-1a07-4879-a7dd-4eccbaaa3bae\") "
Jul 01 21:21:28.283855 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:28.283738    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/25bc1467-1a07-4879-a7dd-4eccbaaa3bae-tokenizer-uds\") pod \"25bc1467-1a07-4879-a7dd-4eccbaaa3bae\" (UID: \"25bc1467-1a07-4879-a7dd-4eccbaaa3bae\") "
Jul 01 21:21:28.283855 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:28.283832    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25bc1467-1a07-4879-a7dd-4eccbaaa3bae-tls-certs\") pod \"25bc1467-1a07-4879-a7dd-4eccbaaa3bae\" (UID: \"25bc1467-1a07-4879-a7dd-4eccbaaa3bae\") "
Jul 01 21:21:28.284007 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:28.283985    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25bc1467-1a07-4879-a7dd-4eccbaaa3bae-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "25bc1467-1a07-4879-a7dd-4eccbaaa3bae" (UID: "25bc1467-1a07-4879-a7dd-4eccbaaa3bae"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 21:21:28.284134 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:28.284116    2571 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/25bc1467-1a07-4879-a7dd-4eccbaaa3bae-tokenizer-uds\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:21:28.285952 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:28.285930    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25bc1467-1a07-4879-a7dd-4eccbaaa3bae-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "25bc1467-1a07-4879-a7dd-4eccbaaa3bae" (UID: "25bc1467-1a07-4879-a7dd-4eccbaaa3bae"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 21:21:28.286036 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:28.285972    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25bc1467-1a07-4879-a7dd-4eccbaaa3bae-kube-api-access-bk7qr" (OuterVolumeSpecName: "kube-api-access-bk7qr") pod "25bc1467-1a07-4879-a7dd-4eccbaaa3bae" (UID: "25bc1467-1a07-4879-a7dd-4eccbaaa3bae"). InnerVolumeSpecName "kube-api-access-bk7qr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 21:21:28.385350 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:28.385329    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bk7qr\" (UniqueName: \"kubernetes.io/projected/25bc1467-1a07-4879-a7dd-4eccbaaa3bae-kube-api-access-bk7qr\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:21:28.385436 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:28.385350    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25bc1467-1a07-4879-a7dd-4eccbaaa3bae-tls-certs\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:21:29.064766 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:29.064733    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s"
Jul 01 21:21:29.065205 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:29.064760    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s" event={"ID":"25bc1467-1a07-4879-a7dd-4eccbaaa3bae","Type":"ContainerDied","Data":"a6faf630f6fb2e669ad131b36c5b13e0d15d9eabc8dbd90fde7d493baa2e08af"}
Jul 01 21:21:29.065205 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:29.064834    2571 scope.go:117] "RemoveContainer" containerID="7adc33f1aae5553f629a32a11b9afb4310e9fdedb27da5ba4fd83f31a02e9bc2"
Jul 01 21:21:29.085985 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:29.085956    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s"]
Jul 01 21:21:29.091168 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:29.091143    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheln89s"]
Jul 01 21:21:29.844151 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:29.844117    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25bc1467-1a07-4879-a7dd-4eccbaaa3bae" path="/var/lib/kubelet/pods/25bc1467-1a07-4879-a7dd-4eccbaaa3bae/volumes"
Jul 01 21:21:58.055973 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:58.055685    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8"]
Jul 01 21:21:58.056459 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:58.056434    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25bc1467-1a07-4879-a7dd-4eccbaaa3bae" containerName="main"
Jul 01 21:21:58.056549 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:58.056465    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="25bc1467-1a07-4879-a7dd-4eccbaaa3bae" containerName="main"
Jul 01 21:21:58.056682 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:58.056620    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="25bc1467-1a07-4879-a7dd-4eccbaaa3bae" containerName="main"
Jul 01 21:21:58.060692 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:58.060661    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8"
Jul 01 21:21:58.064856 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:58.064836    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98c209fa-e72b-416f-99eb-04f3feff04e1-tls-certs\") pod \"llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8\" (UID: \"98c209fa-e72b-416f-99eb-04f3feff04e1\") " pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8"
Jul 01 21:21:58.064961 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:58.064875    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/98c209fa-e72b-416f-99eb-04f3feff04e1-tokenizer-uds\") pod \"llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8\" (UID: \"98c209fa-e72b-416f-99eb-04f3feff04e1\") " pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8"
Jul 01 21:21:58.064961 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:58.064899    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dp22n\" (UniqueName: \"kubernetes.io/projected/98c209fa-e72b-416f-99eb-04f3feff04e1-kube-api-access-dp22n\") pod \"llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8\" (UID: \"98c209fa-e72b-416f-99eb-04f3feff04e1\") " pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8"
Jul 01 21:21:58.065526 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:58.065498    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv1dc4269d1ada5f2d28562215d180c57f-kserve-self-signed-certs\""
Jul 01 21:21:58.065526 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:58.065511    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-76tz7\""
Jul 01 21:21:58.065712 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:58.065562    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 01 21:21:58.065712 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:58.065511    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 01 21:21:58.065712 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:58.065639    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-pvc-router-manage-2577e794-epp-sa-dockercfg-f6z7r\""
Jul 01 21:21:58.076134 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:58.076115    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8"]
Jul 01 21:21:58.165431 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:58.165411    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/98c209fa-e72b-416f-99eb-04f3feff04e1-tokenizer-uds\") pod \"llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8\" (UID: \"98c209fa-e72b-416f-99eb-04f3feff04e1\") " pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8"
Jul 01 21:21:58.165530 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:58.165436    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dp22n\" (UniqueName: \"kubernetes.io/projected/98c209fa-e72b-416f-99eb-04f3feff04e1-kube-api-access-dp22n\") pod \"llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8\" (UID: \"98c209fa-e72b-416f-99eb-04f3feff04e1\") " pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8"
Jul 01 21:21:58.165530 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:58.165494    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98c209fa-e72b-416f-99eb-04f3feff04e1-tls-certs\") pod \"llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8\" (UID: \"98c209fa-e72b-416f-99eb-04f3feff04e1\") " pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8"
Jul 01 21:21:58.165796 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:58.165778    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/98c209fa-e72b-416f-99eb-04f3feff04e1-tokenizer-uds\") pod \"llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8\" (UID: \"98c209fa-e72b-416f-99eb-04f3feff04e1\") " pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8"
Jul 01 21:21:58.167901 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:58.167884    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98c209fa-e72b-416f-99eb-04f3feff04e1-tls-certs\") pod \"llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8\" (UID: \"98c209fa-e72b-416f-99eb-04f3feff04e1\") " pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8"
Jul 01 21:21:58.177038 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:58.177012    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dp22n\" (UniqueName: \"kubernetes.io/projected/98c209fa-e72b-416f-99eb-04f3feff04e1-kube-api-access-dp22n\") pod \"llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8\" (UID: \"98c209fa-e72b-416f-99eb-04f3feff04e1\") " pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8"
Jul 01 21:21:58.369578 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:58.369526    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8"
Jul 01 21:21:58.500683 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:58.500657    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8"]
Jul 01 21:21:58.502432 ip-10-0-140-104 kubenswrapper[2571]: W0701 21:21:58.502404    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98c209fa_e72b_416f_99eb_04f3feff04e1.slice/crio-8dd128a0588875138496c93f190bc3cb5f0a51c93402f28ef121d113ca32f837 WatchSource:0}: Error finding container 8dd128a0588875138496c93f190bc3cb5f0a51c93402f28ef121d113ca32f837: Status 404 returned error can't find the container with id 8dd128a0588875138496c93f190bc3cb5f0a51c93402f28ef121d113ca32f837
Jul 01 21:21:59.190567 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:59.190536    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8" event={"ID":"98c209fa-e72b-416f-99eb-04f3feff04e1","Type":"ContainerStarted","Data":"c2062fa25b1219d103e959b34ee018f5b308e31286fe39e34d8cd74941c050f5"}
Jul 01 21:21:59.191010 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:59.190580    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8" event={"ID":"98c209fa-e72b-416f-99eb-04f3feff04e1","Type":"ContainerStarted","Data":"8dd128a0588875138496c93f190bc3cb5f0a51c93402f28ef121d113ca32f837"}
Jul 01 21:21:59.191010 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:59.190675    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8"
Jul 01 21:21:59.209874 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:21:59.209828    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8" podStartSLOduration=1.20981582 podStartE2EDuration="1.20981582s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 21:21:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 21:21:59.20855769 +0000 UTC m=+2137.721032887" watchObservedRunningTime="2026-07-01 21:21:59.20981582 +0000 UTC m=+2137.722291008"
Jul 01 21:22:30.196466 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:22:30.196439    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8"
Jul 01 21:28:01.613928 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:28:01.613892    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8"]
Jul 01 21:28:01.622761 ip-10-0-140-104 kubenswrapper[2571]: E0701 21:28:01.622723    2571 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 01 21:28:01.622761 ip-10-0-140-104 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 01 21:28:01.622761 ip-10-0-140-104 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 01 21:28:01.622761 ip-10-0-140-104 kubenswrapper[2571]:  > containerID="c2062fa25b1219d103e959b34ee018f5b308e31286fe39e34d8cd74941c050f5" cmd=["/bin/sleep","15"]
Jul 01 21:28:01.623085 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:28:01.622775    2571 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8" message="" err=<
Jul 01 21:28:01.623085 ip-10-0-140-104 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 01 21:28:01.623085 ip-10-0-140-104 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 01 21:28:01.623085 ip-10-0-140-104 kubenswrapper[2571]:  >
Jul 01 21:28:01.623085 ip-10-0-140-104 kubenswrapper[2571]: E0701 21:28:01.622787    2571 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 01 21:28:01.623085 ip-10-0-140-104 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 01 21:28:01.623085 ip-10-0-140-104 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 01 21:28:01.623085 ip-10-0-140-104 kubenswrapper[2571]:  > pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8" podUID="98c209fa-e72b-416f-99eb-04f3feff04e1" containerName="main" containerID="cri-o://c2062fa25b1219d103e959b34ee018f5b308e31286fe39e34d8cd74941c050f5"
Jul 01 21:28:01.623085 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:28:01.622820    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8" podUID="98c209fa-e72b-416f-99eb-04f3feff04e1" containerName="main" containerID="cri-o://c2062fa25b1219d103e959b34ee018f5b308e31286fe39e34d8cd74941c050f5" gracePeriod=60
Jul 01 21:28:01.882674 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:28:01.882617    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8"
Jul 01 21:28:01.927598 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:28:01.927573    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dp22n\" (UniqueName: \"kubernetes.io/projected/98c209fa-e72b-416f-99eb-04f3feff04e1-kube-api-access-dp22n\") pod \"98c209fa-e72b-416f-99eb-04f3feff04e1\" (UID: \"98c209fa-e72b-416f-99eb-04f3feff04e1\") "
Jul 01 21:28:01.927718 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:28:01.927609    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/98c209fa-e72b-416f-99eb-04f3feff04e1-tokenizer-uds\") pod \"98c209fa-e72b-416f-99eb-04f3feff04e1\" (UID: \"98c209fa-e72b-416f-99eb-04f3feff04e1\") "
Jul 01 21:28:01.927718 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:28:01.927702    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98c209fa-e72b-416f-99eb-04f3feff04e1-tls-certs\") pod \"98c209fa-e72b-416f-99eb-04f3feff04e1\" (UID: \"98c209fa-e72b-416f-99eb-04f3feff04e1\") "
Jul 01 21:28:01.927922 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:28:01.927898    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98c209fa-e72b-416f-99eb-04f3feff04e1-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "98c209fa-e72b-416f-99eb-04f3feff04e1" (UID: "98c209fa-e72b-416f-99eb-04f3feff04e1"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 21:28:01.929671 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:28:01.929647    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98c209fa-e72b-416f-99eb-04f3feff04e1-kube-api-access-dp22n" (OuterVolumeSpecName: "kube-api-access-dp22n") pod "98c209fa-e72b-416f-99eb-04f3feff04e1" (UID: "98c209fa-e72b-416f-99eb-04f3feff04e1"). InnerVolumeSpecName "kube-api-access-dp22n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 21:28:01.929744 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:28:01.929725    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98c209fa-e72b-416f-99eb-04f3feff04e1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "98c209fa-e72b-416f-99eb-04f3feff04e1" (UID: "98c209fa-e72b-416f-99eb-04f3feff04e1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 21:28:02.028653 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:28:02.028609    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98c209fa-e72b-416f-99eb-04f3feff04e1-tls-certs\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:28:02.028653 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:28:02.028652    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dp22n\" (UniqueName: \"kubernetes.io/projected/98c209fa-e72b-416f-99eb-04f3feff04e1-kube-api-access-dp22n\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:28:02.028775 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:28:02.028666    2571 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/98c209fa-e72b-416f-99eb-04f3feff04e1-tokenizer-uds\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:28:02.653594 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:28:02.653561    2571 generic.go:356] "Generic (PLEG): container finished" podID="98c209fa-e72b-416f-99eb-04f3feff04e1" containerID="c2062fa25b1219d103e959b34ee018f5b308e31286fe39e34d8cd74941c050f5" exitCode=0
Jul 01 21:28:02.654104 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:28:02.653623    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8"
Jul 01 21:28:02.654104 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:28:02.653666    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8" event={"ID":"98c209fa-e72b-416f-99eb-04f3feff04e1","Type":"ContainerDied","Data":"c2062fa25b1219d103e959b34ee018f5b308e31286fe39e34d8cd74941c050f5"}
Jul 01 21:28:02.654104 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:28:02.653739    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8" event={"ID":"98c209fa-e72b-416f-99eb-04f3feff04e1","Type":"ContainerDied","Data":"8dd128a0588875138496c93f190bc3cb5f0a51c93402f28ef121d113ca32f837"}
Jul 01 21:28:02.654104 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:28:02.653765    2571 scope.go:117] "RemoveContainer" containerID="c2062fa25b1219d103e959b34ee018f5b308e31286fe39e34d8cd74941c050f5"
Jul 01 21:28:02.662361 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:28:02.662344    2571 scope.go:117] "RemoveContainer" containerID="c2062fa25b1219d103e959b34ee018f5b308e31286fe39e34d8cd74941c050f5"
Jul 01 21:28:02.662595 ip-10-0-140-104 kubenswrapper[2571]: E0701 21:28:02.662578    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2062fa25b1219d103e959b34ee018f5b308e31286fe39e34d8cd74941c050f5\": container with ID starting with c2062fa25b1219d103e959b34ee018f5b308e31286fe39e34d8cd74941c050f5 not found: ID does not exist" containerID="c2062fa25b1219d103e959b34ee018f5b308e31286fe39e34d8cd74941c050f5"
Jul 01 21:28:02.662682 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:28:02.662599    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2062fa25b1219d103e959b34ee018f5b308e31286fe39e34d8cd74941c050f5"} err="failed to get container status \"c2062fa25b1219d103e959b34ee018f5b308e31286fe39e34d8cd74941c050f5\": rpc error: code = NotFound desc = could not find container \"c2062fa25b1219d103e959b34ee018f5b308e31286fe39e34d8cd74941c050f5\": container with ID starting with c2062fa25b1219d103e959b34ee018f5b308e31286fe39e34d8cd74941c050f5 not found: ID does not exist"
Jul 01 21:28:02.674057 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:28:02.674034    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8"]
Jul 01 21:28:02.681202 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:28:02.681183    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheklks8"]
Jul 01 21:28:03.838186 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:28:03.838157    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98c209fa-e72b-416f-99eb-04f3feff04e1" path="/var/lib/kubelet/pods/98c209fa-e72b-416f-99eb-04f3feff04e1/volumes"
Jul 01 21:47:19.289424 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:19.289372    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf"]
Jul 01 21:47:19.290130 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:19.289704    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98c209fa-e72b-416f-99eb-04f3feff04e1" containerName="main"
Jul 01 21:47:19.290130 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:19.289714    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="98c209fa-e72b-416f-99eb-04f3feff04e1" containerName="main"
Jul 01 21:47:19.290130 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:19.289774    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="98c209fa-e72b-416f-99eb-04f3feff04e1" containerName="main"
Jul 01 21:47:19.292227 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:19.292206    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf"
Jul 01 21:47:19.295215 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:19.295197    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 01 21:47:19.296415 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:19.296397    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 01 21:47:19.296575 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:19.296559    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-inline-config-test-kserve-self-signed-certs\""
Jul 01 21:47:19.296683 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:19.296618    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-76tz7\""
Jul 01 21:47:19.301502 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:19.301476    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf"]
Jul 01 21:47:19.397775 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:19.397745    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-tmp-dir\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-c4czf\" (UID: \"35f7ef0e-3f8d-435d-b95f-e93e573b3f35\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf"
Jul 01 21:47:19.397898 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:19.397792    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-dshm\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-c4czf\" (UID: \"35f7ef0e-3f8d-435d-b95f-e93e573b3f35\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf"
Jul 01 21:47:19.397898 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:19.397820    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-model-cache\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-c4czf\" (UID: \"35f7ef0e-3f8d-435d-b95f-e93e573b3f35\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf"
Jul 01 21:47:19.397898 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:19.397887    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4p52t\" (UniqueName: \"kubernetes.io/projected/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-kube-api-access-4p52t\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-c4czf\" (UID: \"35f7ef0e-3f8d-435d-b95f-e93e573b3f35\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf"
Jul 01 21:47:19.398004 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:19.397981    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-home\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-c4czf\" (UID: \"35f7ef0e-3f8d-435d-b95f-e93e573b3f35\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf"
Jul 01 21:47:19.398040 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:19.398007    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-tls-certs\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-c4czf\" (UID: \"35f7ef0e-3f8d-435d-b95f-e93e573b3f35\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf"
Jul 01 21:47:19.499072 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:19.499044    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-home\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-c4czf\" (UID: \"35f7ef0e-3f8d-435d-b95f-e93e573b3f35\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf"
Jul 01 21:47:19.499182 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:19.499083    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-tls-certs\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-c4czf\" (UID: \"35f7ef0e-3f8d-435d-b95f-e93e573b3f35\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf"
Jul 01 21:47:19.499182 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:19.499120    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-tmp-dir\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-c4czf\" (UID: \"35f7ef0e-3f8d-435d-b95f-e93e573b3f35\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf"
Jul 01 21:47:19.499182 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:19.499158    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-dshm\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-c4czf\" (UID: \"35f7ef0e-3f8d-435d-b95f-e93e573b3f35\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf"
Jul 01 21:47:19.499337 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:19.499184    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-model-cache\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-c4czf\" (UID: \"35f7ef0e-3f8d-435d-b95f-e93e573b3f35\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf"
Jul 01 21:47:19.499337 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:19.499216    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p52t\" (UniqueName: \"kubernetes.io/projected/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-kube-api-access-4p52t\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-c4czf\" (UID: \"35f7ef0e-3f8d-435d-b95f-e93e573b3f35\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf"
Jul 01 21:47:19.499533 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:19.499506    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-tmp-dir\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-c4czf\" (UID: \"35f7ef0e-3f8d-435d-b95f-e93e573b3f35\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf"
Jul 01 21:47:19.499697 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:19.499618    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-home\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-c4czf\" (UID: \"35f7ef0e-3f8d-435d-b95f-e93e573b3f35\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf"
Jul 01 21:47:19.499697 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:19.499619    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-model-cache\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-c4czf\" (UID: \"35f7ef0e-3f8d-435d-b95f-e93e573b3f35\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf"
Jul 01 21:47:19.501441 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:19.501417    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-dshm\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-c4czf\" (UID: \"35f7ef0e-3f8d-435d-b95f-e93e573b3f35\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf"
Jul 01 21:47:19.501905 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:19.501883    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-tls-certs\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-c4czf\" (UID: \"35f7ef0e-3f8d-435d-b95f-e93e573b3f35\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf"
Jul 01 21:47:19.510702 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:19.510677    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4p52t\" (UniqueName: \"kubernetes.io/projected/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-kube-api-access-4p52t\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-c4czf\" (UID: \"35f7ef0e-3f8d-435d-b95f-e93e573b3f35\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf"
Jul 01 21:47:19.602359 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:19.602268    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf"
Jul 01 21:47:19.736049 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:19.736026    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf"]
Jul 01 21:47:19.737905 ip-10-0-140-104 kubenswrapper[2571]: W0701 21:47:19.737875    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35f7ef0e_3f8d_435d_b95f_e93e573b3f35.slice/crio-8f3e9f2b241f0aa42d267dbd386106b88c8c73b51e23f6579e9022048bdb1d5c WatchSource:0}: Error finding container 8f3e9f2b241f0aa42d267dbd386106b88c8c73b51e23f6579e9022048bdb1d5c: Status 404 returned error can't find the container with id 8f3e9f2b241f0aa42d267dbd386106b88c8c73b51e23f6579e9022048bdb1d5c
Jul 01 21:47:20.601455 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:20.601423    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf" event={"ID":"35f7ef0e-3f8d-435d-b95f-e93e573b3f35","Type":"ContainerStarted","Data":"7a1403aa403e4e9c7ae16be4864465e081800c82296f8732b586d9e9cf160894"}
Jul 01 21:47:20.601820 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:20.601469    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf" event={"ID":"35f7ef0e-3f8d-435d-b95f-e93e573b3f35","Type":"ContainerStarted","Data":"8f3e9f2b241f0aa42d267dbd386106b88c8c73b51e23f6579e9022048bdb1d5c"}
Jul 01 21:47:20.602682 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:20.602665    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf"
Jul 01 21:47:20.620073 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:20.620036    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf" podStartSLOduration=1.6200237259999999 podStartE2EDuration="1.620023726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 21:47:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 21:47:20.61904307 +0000 UTC m=+3659.131518257" watchObservedRunningTime="2026-07-01 21:47:20.620023726 +0000 UTC m=+3659.132498913"
Jul 01 21:47:29.602574 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:29.602537    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf"
Jul 01 21:47:29.614344 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:29.614323    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf"
Jul 01 21:47:29.651460 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:29.651437    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf"
Jul 01 21:47:52.489176 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:47:52.489101    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf"]
Jul 01 21:48:07.508876 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:07.508831    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf" podUID="35f7ef0e-3f8d-435d-b95f-e93e573b3f35" containerName="main" containerID="cri-o://7a1403aa403e4e9c7ae16be4864465e081800c82296f8732b586d9e9cf160894" gracePeriod=45
Jul 01 21:48:07.640760 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:07.640722    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf" podUID="35f7ef0e-3f8d-435d-b95f-e93e573b3f35" containerName="main" probeResult="failure" output="Get \"https://10.133.0.33:8000/health\": dial tcp 10.133.0.33:8000: connect: connection refused"
Jul 01 21:48:07.751019 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:07.750991    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf"
Jul 01 21:48:07.819804 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:07.819724    2571 generic.go:356] "Generic (PLEG): container finished" podID="35f7ef0e-3f8d-435d-b95f-e93e573b3f35" containerID="7a1403aa403e4e9c7ae16be4864465e081800c82296f8732b586d9e9cf160894" exitCode=0
Jul 01 21:48:07.819804 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:07.819761    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf" event={"ID":"35f7ef0e-3f8d-435d-b95f-e93e573b3f35","Type":"ContainerDied","Data":"7a1403aa403e4e9c7ae16be4864465e081800c82296f8732b586d9e9cf160894"}
Jul 01 21:48:07.820012 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:07.819829    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf" event={"ID":"35f7ef0e-3f8d-435d-b95f-e93e573b3f35","Type":"ContainerDied","Data":"8f3e9f2b241f0aa42d267dbd386106b88c8c73b51e23f6579e9022048bdb1d5c"}
Jul 01 21:48:07.820012 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:07.819778    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf"
Jul 01 21:48:07.820012 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:07.819859    2571 scope.go:117] "RemoveContainer" containerID="7a1403aa403e4e9c7ae16be4864465e081800c82296f8732b586d9e9cf160894"
Jul 01 21:48:07.828027 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:07.828009    2571 scope.go:117] "RemoveContainer" containerID="7a1403aa403e4e9c7ae16be4864465e081800c82296f8732b586d9e9cf160894"
Jul 01 21:48:07.828303 ip-10-0-140-104 kubenswrapper[2571]: E0701 21:48:07.828286    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a1403aa403e4e9c7ae16be4864465e081800c82296f8732b586d9e9cf160894\": container with ID starting with 7a1403aa403e4e9c7ae16be4864465e081800c82296f8732b586d9e9cf160894 not found: ID does not exist" containerID="7a1403aa403e4e9c7ae16be4864465e081800c82296f8732b586d9e9cf160894"
Jul 01 21:48:07.828357 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:07.828314    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a1403aa403e4e9c7ae16be4864465e081800c82296f8732b586d9e9cf160894"} err="failed to get container status \"7a1403aa403e4e9c7ae16be4864465e081800c82296f8732b586d9e9cf160894\": rpc error: code = NotFound desc = could not find container \"7a1403aa403e4e9c7ae16be4864465e081800c82296f8732b586d9e9cf160894\": container with ID starting with 7a1403aa403e4e9c7ae16be4864465e081800c82296f8732b586d9e9cf160894 not found: ID does not exist"
Jul 01 21:48:07.921003 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:07.920981    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-home\") pod \"35f7ef0e-3f8d-435d-b95f-e93e573b3f35\" (UID: \"35f7ef0e-3f8d-435d-b95f-e93e573b3f35\") "
Jul 01 21:48:07.921134 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:07.921015    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4p52t\" (UniqueName: \"kubernetes.io/projected/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-kube-api-access-4p52t\") pod \"35f7ef0e-3f8d-435d-b95f-e93e573b3f35\" (UID: \"35f7ef0e-3f8d-435d-b95f-e93e573b3f35\") "
Jul 01 21:48:07.921134 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:07.921048    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-tls-certs\") pod \"35f7ef0e-3f8d-435d-b95f-e93e573b3f35\" (UID: \"35f7ef0e-3f8d-435d-b95f-e93e573b3f35\") "
Jul 01 21:48:07.921134 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:07.921080    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-dshm\") pod \"35f7ef0e-3f8d-435d-b95f-e93e573b3f35\" (UID: \"35f7ef0e-3f8d-435d-b95f-e93e573b3f35\") "
Jul 01 21:48:07.921134 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:07.921109    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-model-cache\") pod \"35f7ef0e-3f8d-435d-b95f-e93e573b3f35\" (UID: \"35f7ef0e-3f8d-435d-b95f-e93e573b3f35\") "
Jul 01 21:48:07.921335 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:07.921154    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-tmp-dir\") pod \"35f7ef0e-3f8d-435d-b95f-e93e573b3f35\" (UID: \"35f7ef0e-3f8d-435d-b95f-e93e573b3f35\") "
Jul 01 21:48:07.921335 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:07.921229    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-home" (OuterVolumeSpecName: "home") pod "35f7ef0e-3f8d-435d-b95f-e93e573b3f35" (UID: "35f7ef0e-3f8d-435d-b95f-e93e573b3f35"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 21:48:07.921469 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:07.921355    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-model-cache" (OuterVolumeSpecName: "model-cache") pod "35f7ef0e-3f8d-435d-b95f-e93e573b3f35" (UID: "35f7ef0e-3f8d-435d-b95f-e93e573b3f35"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 21:48:07.921469 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:07.921432    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-home\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:48:07.921469 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:07.921448    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-model-cache\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:48:07.921469 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:07.921444    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "35f7ef0e-3f8d-435d-b95f-e93e573b3f35" (UID: "35f7ef0e-3f8d-435d-b95f-e93e573b3f35"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 21:48:07.923313 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:07.923288    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-kube-api-access-4p52t" (OuterVolumeSpecName: "kube-api-access-4p52t") pod "35f7ef0e-3f8d-435d-b95f-e93e573b3f35" (UID: "35f7ef0e-3f8d-435d-b95f-e93e573b3f35"). InnerVolumeSpecName "kube-api-access-4p52t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 21:48:07.923402 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:07.923311    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-dshm" (OuterVolumeSpecName: "dshm") pod "35f7ef0e-3f8d-435d-b95f-e93e573b3f35" (UID: "35f7ef0e-3f8d-435d-b95f-e93e573b3f35"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 21:48:07.923402 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:07.923353    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "35f7ef0e-3f8d-435d-b95f-e93e573b3f35" (UID: "35f7ef0e-3f8d-435d-b95f-e93e573b3f35"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 21:48:08.022479 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:08.022457    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4p52t\" (UniqueName: \"kubernetes.io/projected/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-kube-api-access-4p52t\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:48:08.022479 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:08.022476    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-tls-certs\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:48:08.022612 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:08.022484    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-dshm\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:48:08.022612 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:08.022492    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35f7ef0e-3f8d-435d-b95f-e93e573b3f35-tmp-dir\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:48:08.145180 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:08.145150    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf"]
Jul 01 21:48:08.152604 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:08.152577    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-c4czf"]
Jul 01 21:48:09.838435 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:09.838405    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35f7ef0e-3f8d-435d-b95f-e93e573b3f35" path="/var/lib/kubelet/pods/35f7ef0e-3f8d-435d-b95f-e93e573b3f35/volumes"
Jul 01 21:48:28.478525 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:28.478483    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv"]
Jul 01 21:48:28.479088 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:28.478965    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35f7ef0e-3f8d-435d-b95f-e93e573b3f35" containerName="main"
Jul 01 21:48:28.479088 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:28.478983    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="35f7ef0e-3f8d-435d-b95f-e93e573b3f35" containerName="main"
Jul 01 21:48:28.479194 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:28.479089    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="35f7ef0e-3f8d-435d-b95f-e93e573b3f35" containerName="main"
Jul 01 21:48:28.482212 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:28.482190    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv"
Jul 01 21:48:28.486522 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:28.486236    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv431d7e30f13554296364403a5b6365f5-kserve-self-signed-certs\""
Jul 01 21:48:28.486522 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:28.486245    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 01 21:48:28.486522 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:28.486288    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 01 21:48:28.486522 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:28.486351    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-76tz7\""
Jul 01 21:48:28.489051 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:28.489031    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv"]
Jul 01 21:48:28.592256 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:28.592235    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d088b04-3dcf-408f-a061-8f7577b6c74c-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv\" (UID: \"9d088b04-3dcf-408f-a061-8f7577b6c74c\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv"
Jul 01 21:48:28.592382 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:28.592264    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d088b04-3dcf-408f-a061-8f7577b6c74c-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv\" (UID: \"9d088b04-3dcf-408f-a061-8f7577b6c74c\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv"
Jul 01 21:48:28.592382 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:28.592281    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d088b04-3dcf-408f-a061-8f7577b6c74c-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv\" (UID: \"9d088b04-3dcf-408f-a061-8f7577b6c74c\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv"
Jul 01 21:48:28.592493 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:28.592410    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg4st\" (UniqueName: \"kubernetes.io/projected/9d088b04-3dcf-408f-a061-8f7577b6c74c-kube-api-access-mg4st\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv\" (UID: \"9d088b04-3dcf-408f-a061-8f7577b6c74c\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv"
Jul 01 21:48:28.592493 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:28.592454    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d088b04-3dcf-408f-a061-8f7577b6c74c-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv\" (UID: \"9d088b04-3dcf-408f-a061-8f7577b6c74c\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv"
Jul 01 21:48:28.592597 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:28.592552    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d088b04-3dcf-408f-a061-8f7577b6c74c-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv\" (UID: \"9d088b04-3dcf-408f-a061-8f7577b6c74c\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv"
Jul 01 21:48:28.693297 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:28.693261    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d088b04-3dcf-408f-a061-8f7577b6c74c-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv\" (UID: \"9d088b04-3dcf-408f-a061-8f7577b6c74c\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv"
Jul 01 21:48:28.693524 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:28.693505    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d088b04-3dcf-408f-a061-8f7577b6c74c-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv\" (UID: \"9d088b04-3dcf-408f-a061-8f7577b6c74c\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv"
Jul 01 21:48:28.693608 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:28.693545    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d088b04-3dcf-408f-a061-8f7577b6c74c-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv\" (UID: \"9d088b04-3dcf-408f-a061-8f7577b6c74c\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv"
Jul 01 21:48:28.693608 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:28.693567    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d088b04-3dcf-408f-a061-8f7577b6c74c-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv\" (UID: \"9d088b04-3dcf-408f-a061-8f7577b6c74c\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv"
Jul 01 21:48:28.693756 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:28.693659    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg4st\" (UniqueName: \"kubernetes.io/projected/9d088b04-3dcf-408f-a061-8f7577b6c74c-kube-api-access-mg4st\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv\" (UID: \"9d088b04-3dcf-408f-a061-8f7577b6c74c\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv"
Jul 01 21:48:28.693756 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:28.693697    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d088b04-3dcf-408f-a061-8f7577b6c74c-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv\" (UID: \"9d088b04-3dcf-408f-a061-8f7577b6c74c\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv"
Jul 01 21:48:28.694193 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:28.694107    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d088b04-3dcf-408f-a061-8f7577b6c74c-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv\" (UID: \"9d088b04-3dcf-408f-a061-8f7577b6c74c\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv"
Jul 01 21:48:28.694695 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:28.694671    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d088b04-3dcf-408f-a061-8f7577b6c74c-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv\" (UID: \"9d088b04-3dcf-408f-a061-8f7577b6c74c\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv"
Jul 01 21:48:28.694859 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:28.694840    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d088b04-3dcf-408f-a061-8f7577b6c74c-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv\" (UID: \"9d088b04-3dcf-408f-a061-8f7577b6c74c\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv"
Jul 01 21:48:28.695801 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:28.695780    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d088b04-3dcf-408f-a061-8f7577b6c74c-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv\" (UID: \"9d088b04-3dcf-408f-a061-8f7577b6c74c\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv"
Jul 01 21:48:28.696302 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:28.696275    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d088b04-3dcf-408f-a061-8f7577b6c74c-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv\" (UID: \"9d088b04-3dcf-408f-a061-8f7577b6c74c\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv"
Jul 01 21:48:28.707084 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:28.707063    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg4st\" (UniqueName: \"kubernetes.io/projected/9d088b04-3dcf-408f-a061-8f7577b6c74c-kube-api-access-mg4st\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv\" (UID: \"9d088b04-3dcf-408f-a061-8f7577b6c74c\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv"
Jul 01 21:48:28.794201 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:28.794109    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv"
Jul 01 21:48:28.936697 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:28.936672    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv"]
Jul 01 21:48:28.938698 ip-10-0-140-104 kubenswrapper[2571]: W0701 21:48:28.938667    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d088b04_3dcf_408f_a061_8f7577b6c74c.slice/crio-21fd4cbfa74ff35578de076253a7ded5fe49be69913e1cc10a8243b9568917cd WatchSource:0}: Error finding container 21fd4cbfa74ff35578de076253a7ded5fe49be69913e1cc10a8243b9568917cd: Status 404 returned error can't find the container with id 21fd4cbfa74ff35578de076253a7ded5fe49be69913e1cc10a8243b9568917cd
Jul 01 21:48:29.922538 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:29.922494    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv" event={"ID":"9d088b04-3dcf-408f-a061-8f7577b6c74c","Type":"ContainerStarted","Data":"f7b50dcb6576b03883e5aba2fe99668db7d235b6ca07976e68e73e36c24a9d49"}
Jul 01 21:48:29.922952 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:29.922561    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv" event={"ID":"9d088b04-3dcf-408f-a061-8f7577b6c74c","Type":"ContainerStarted","Data":"21fd4cbfa74ff35578de076253a7ded5fe49be69913e1cc10a8243b9568917cd"}
Jul 01 21:48:29.924323 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:29.924302    2571 generic.go:356] "Generic (PLEG): container finished" podID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13" containerID="dfb41cc2a4c264ca1199e6658fb2d4b84e508515ab2af334913b5184552d4de3" exitCode=137
Jul 01 21:48:29.924410 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:29.924342    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" event={"ID":"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13","Type":"ContainerDied","Data":"dfb41cc2a4c264ca1199e6658fb2d4b84e508515ab2af334913b5184552d4de3"}
Jul 01 21:48:29.924410 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:29.924382    2571 scope.go:117] "RemoveContainer" containerID="c71445b5ddf9bcfc1081317b4de334650e5c94af28db02a35fd0d6cea7cd6317"
Jul 01 21:48:29.924670 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:29.924654    2571 scope.go:117] "RemoveContainer" containerID="dfb41cc2a4c264ca1199e6658fb2d4b84e508515ab2af334913b5184552d4de3"
Jul 01 21:48:29.925712 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:29.925675    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 01 21:48:29.925812 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:29.925770    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 01 21:48:29.925863 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:29.925812    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 01 21:48:29.941788 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:29.941745    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv" podStartSLOduration=1.941728149 podStartE2EDuration="1.941728149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 21:48:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 21:48:29.940137032 +0000 UTC m=+3728.452612220" watchObservedRunningTime="2026-07-01 21:48:29.941728149 +0000 UTC m=+3728.454203338"
Jul 01 21:48:30.794459 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:30.794425    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv"
Jul 01 21:48:30.931118 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:30.931084    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" event={"ID":"f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13","Type":"ContainerStarted","Data":"326c883d138db7246fa953b892394eaef4ea02b8ae9f6fee4d7d7989e82ac50f"}
Jul 01 21:48:31.722915 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:31.722890    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 21:48:31.723965 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:31.723936    2571 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-655f7c58b4-57g8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body=
Jul 01 21:48:31.724069 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:31.723977    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" podUID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused"
Jul 01 21:48:32.723271 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:32.723232    2571 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-655f7c58b4-57g8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body=
Jul 01 21:48:32.723654 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:32.723321    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" podUID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused"
Jul 01 21:48:33.723073 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:33.723042    2571 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-655f7c58b4-57g8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body=
Jul 01 21:48:33.723241 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:33.723092    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" podUID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused"
Jul 01 21:48:34.723047 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:34.723012    2571 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-655f7c58b4-57g8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body=
Jul 01 21:48:34.723415 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:34.723063    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" podUID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused"
Jul 01 21:48:35.376424 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:35.376389    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 21:48:36.724269 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:36.724234    2571 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-655f7c58b4-57g8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 01 21:48:36.724801 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:36.724329    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x" podUID="f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 01 21:48:37.285718 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:37.285689    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 21:48:37.286988 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:37.286961    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-655f7c58b4-57g8x"
Jul 01 21:48:38.795094 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:38.795065    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv"
Jul 01 21:48:38.806725 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:38.806703    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv"
Jul 01 21:48:38.979009 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:48:38.978982    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv"
Jul 01 21:55:41.575825 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:55:41.575789    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv"]
Jul 01 21:55:56.594735 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:55:56.594687    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv" podUID="9d088b04-3dcf-408f-a061-8f7577b6c74c" containerName="main" containerID="cri-o://f7b50dcb6576b03883e5aba2fe99668db7d235b6ca07976e68e73e36c24a9d49" gracePeriod=45
Jul 01 21:55:56.833945 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:55:56.833918    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv"
Jul 01 21:55:56.898104 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:55:56.898045    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d088b04-3dcf-408f-a061-8f7577b6c74c-tls-certs\") pod \"9d088b04-3dcf-408f-a061-8f7577b6c74c\" (UID: \"9d088b04-3dcf-408f-a061-8f7577b6c74c\") "
Jul 01 21:55:56.898104 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:55:56.898095    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d088b04-3dcf-408f-a061-8f7577b6c74c-dshm\") pod \"9d088b04-3dcf-408f-a061-8f7577b6c74c\" (UID: \"9d088b04-3dcf-408f-a061-8f7577b6c74c\") "
Jul 01 21:55:56.898279 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:55:56.898118    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d088b04-3dcf-408f-a061-8f7577b6c74c-tmp-dir\") pod \"9d088b04-3dcf-408f-a061-8f7577b6c74c\" (UID: \"9d088b04-3dcf-408f-a061-8f7577b6c74c\") "
Jul 01 21:55:56.898279 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:55:56.898160    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d088b04-3dcf-408f-a061-8f7577b6c74c-home\") pod \"9d088b04-3dcf-408f-a061-8f7577b6c74c\" (UID: \"9d088b04-3dcf-408f-a061-8f7577b6c74c\") "
Jul 01 21:55:56.898279 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:55:56.898193    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d088b04-3dcf-408f-a061-8f7577b6c74c-model-cache\") pod \"9d088b04-3dcf-408f-a061-8f7577b6c74c\" (UID: \"9d088b04-3dcf-408f-a061-8f7577b6c74c\") "
Jul 01 21:55:56.898279 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:55:56.898217    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mg4st\" (UniqueName: \"kubernetes.io/projected/9d088b04-3dcf-408f-a061-8f7577b6c74c-kube-api-access-mg4st\") pod \"9d088b04-3dcf-408f-a061-8f7577b6c74c\" (UID: \"9d088b04-3dcf-408f-a061-8f7577b6c74c\") "
Jul 01 21:55:56.898490 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:55:56.898360    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d088b04-3dcf-408f-a061-8f7577b6c74c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9d088b04-3dcf-408f-a061-8f7577b6c74c" (UID: "9d088b04-3dcf-408f-a061-8f7577b6c74c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 21:55:56.898490 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:55:56.898410    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d088b04-3dcf-408f-a061-8f7577b6c74c-model-cache" (OuterVolumeSpecName: "model-cache") pod "9d088b04-3dcf-408f-a061-8f7577b6c74c" (UID: "9d088b04-3dcf-408f-a061-8f7577b6c74c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 21:55:56.898490 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:55:56.898462    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d088b04-3dcf-408f-a061-8f7577b6c74c-home" (OuterVolumeSpecName: "home") pod "9d088b04-3dcf-408f-a061-8f7577b6c74c" (UID: "9d088b04-3dcf-408f-a061-8f7577b6c74c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 21:55:56.898669 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:55:56.898517    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d088b04-3dcf-408f-a061-8f7577b6c74c-tmp-dir\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:55:56.898669 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:55:56.898530    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d088b04-3dcf-408f-a061-8f7577b6c74c-model-cache\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:55:56.900172 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:55:56.900153    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d088b04-3dcf-408f-a061-8f7577b6c74c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9d088b04-3dcf-408f-a061-8f7577b6c74c" (UID: "9d088b04-3dcf-408f-a061-8f7577b6c74c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 21:55:56.900172 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:55:56.900158    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d088b04-3dcf-408f-a061-8f7577b6c74c-dshm" (OuterVolumeSpecName: "dshm") pod "9d088b04-3dcf-408f-a061-8f7577b6c74c" (UID: "9d088b04-3dcf-408f-a061-8f7577b6c74c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 21:55:56.900287 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:55:56.900202    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d088b04-3dcf-408f-a061-8f7577b6c74c-kube-api-access-mg4st" (OuterVolumeSpecName: "kube-api-access-mg4st") pod "9d088b04-3dcf-408f-a061-8f7577b6c74c" (UID: "9d088b04-3dcf-408f-a061-8f7577b6c74c"). InnerVolumeSpecName "kube-api-access-mg4st". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 21:55:56.927451 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:55:56.927427    2571 generic.go:356] "Generic (PLEG): container finished" podID="9d088b04-3dcf-408f-a061-8f7577b6c74c" containerID="f7b50dcb6576b03883e5aba2fe99668db7d235b6ca07976e68e73e36c24a9d49" exitCode=0
Jul 01 21:55:56.927547 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:55:56.927464    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv" event={"ID":"9d088b04-3dcf-408f-a061-8f7577b6c74c","Type":"ContainerDied","Data":"f7b50dcb6576b03883e5aba2fe99668db7d235b6ca07976e68e73e36c24a9d49"}
Jul 01 21:55:56.927547 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:55:56.927471    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv"
Jul 01 21:55:56.927547 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:55:56.927515    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv" event={"ID":"9d088b04-3dcf-408f-a061-8f7577b6c74c","Type":"ContainerDied","Data":"21fd4cbfa74ff35578de076253a7ded5fe49be69913e1cc10a8243b9568917cd"}
Jul 01 21:55:56.927547 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:55:56.927536    2571 scope.go:117] "RemoveContainer" containerID="f7b50dcb6576b03883e5aba2fe99668db7d235b6ca07976e68e73e36c24a9d49"
Jul 01 21:55:56.935615 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:55:56.935598    2571 scope.go:117] "RemoveContainer" containerID="f7b50dcb6576b03883e5aba2fe99668db7d235b6ca07976e68e73e36c24a9d49"
Jul 01 21:55:56.935869 ip-10-0-140-104 kubenswrapper[2571]: E0701 21:55:56.935849    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7b50dcb6576b03883e5aba2fe99668db7d235b6ca07976e68e73e36c24a9d49\": container with ID starting with f7b50dcb6576b03883e5aba2fe99668db7d235b6ca07976e68e73e36c24a9d49 not found: ID does not exist" containerID="f7b50dcb6576b03883e5aba2fe99668db7d235b6ca07976e68e73e36c24a9d49"
Jul 01 21:55:56.935930 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:55:56.935878    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7b50dcb6576b03883e5aba2fe99668db7d235b6ca07976e68e73e36c24a9d49"} err="failed to get container status \"f7b50dcb6576b03883e5aba2fe99668db7d235b6ca07976e68e73e36c24a9d49\": rpc error: code = NotFound desc = could not find container \"f7b50dcb6576b03883e5aba2fe99668db7d235b6ca07976e68e73e36c24a9d49\": container with ID starting with f7b50dcb6576b03883e5aba2fe99668db7d235b6ca07976e68e73e36c24a9d49 not found: ID does not exist"
Jul 01 21:55:56.949607 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:55:56.949580    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv"]
Jul 01 21:55:56.956446 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:55:56.956424    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4bzrdv"]
Jul 01 21:55:56.999149 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:55:56.999131    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d088b04-3dcf-408f-a061-8f7577b6c74c-tls-certs\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:55:56.999149 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:55:56.999150    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d088b04-3dcf-408f-a061-8f7577b6c74c-dshm\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:55:56.999259 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:55:56.999158    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d088b04-3dcf-408f-a061-8f7577b6c74c-home\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:55:56.999259 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:55:56.999166    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mg4st\" (UniqueName: \"kubernetes.io/projected/9d088b04-3dcf-408f-a061-8f7577b6c74c-kube-api-access-mg4st\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 21:55:57.838454 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:55:57.838430    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d088b04-3dcf-408f-a061-8f7577b6c74c" path="/var/lib/kubelet/pods/9d088b04-3dcf-408f-a061-8f7577b6c74c/volumes"
Jul 01 21:56:04.677070 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:04.677037    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn"]
Jul 01 21:56:04.677434 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:04.677315    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d088b04-3dcf-408f-a061-8f7577b6c74c" containerName="main"
Jul 01 21:56:04.677434 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:04.677325    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d088b04-3dcf-408f-a061-8f7577b6c74c" containerName="main"
Jul 01 21:56:04.677434 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:04.677416    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d088b04-3dcf-408f-a061-8f7577b6c74c" containerName="main"
Jul 01 21:56:04.681771 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:04.681756    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn"
Jul 01 21:56:04.685892 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:04.685864    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-76tz7\""
Jul 01 21:56:04.686024 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:04.685922    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 01 21:56:04.686024 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:04.685994    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 01 21:56:04.686024 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:04.686003    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-configmap-ref-test-kserve-self-signed-certs\""
Jul 01 21:56:04.688818 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:04.688604    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn"]
Jul 01 21:56:04.754380 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:04.754359    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0470911-5d99-4cf3-bf51-f509a2ec3184-dshm\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn\" (UID: \"f0470911-5d99-4cf3-bf51-f509a2ec3184\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn"
Jul 01 21:56:04.754480 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:04.754387    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0470911-5d99-4cf3-bf51-f509a2ec3184-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn\" (UID: \"f0470911-5d99-4cf3-bf51-f509a2ec3184\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn"
Jul 01 21:56:04.754480 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:04.754416    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0470911-5d99-4cf3-bf51-f509a2ec3184-model-cache\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn\" (UID: \"f0470911-5d99-4cf3-bf51-f509a2ec3184\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn"
Jul 01 21:56:04.754560 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:04.754488    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvw8j\" (UniqueName: \"kubernetes.io/projected/f0470911-5d99-4cf3-bf51-f509a2ec3184-kube-api-access-vvw8j\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn\" (UID: \"f0470911-5d99-4cf3-bf51-f509a2ec3184\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn"
Jul 01 21:56:04.754560 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:04.754531    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0470911-5d99-4cf3-bf51-f509a2ec3184-tmp-dir\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn\" (UID: \"f0470911-5d99-4cf3-bf51-f509a2ec3184\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn"
Jul 01 21:56:04.754560 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:04.754554    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0470911-5d99-4cf3-bf51-f509a2ec3184-home\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn\" (UID: \"f0470911-5d99-4cf3-bf51-f509a2ec3184\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn"
Jul 01 21:56:04.854791 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:04.854772    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0470911-5d99-4cf3-bf51-f509a2ec3184-dshm\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn\" (UID: \"f0470911-5d99-4cf3-bf51-f509a2ec3184\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn"
Jul 01 21:56:04.854896 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:04.854797    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0470911-5d99-4cf3-bf51-f509a2ec3184-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn\" (UID: \"f0470911-5d99-4cf3-bf51-f509a2ec3184\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn"
Jul 01 21:56:04.854896 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:04.854824    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0470911-5d99-4cf3-bf51-f509a2ec3184-model-cache\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn\" (UID: \"f0470911-5d99-4cf3-bf51-f509a2ec3184\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn"
Jul 01 21:56:04.854896 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:04.854856    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvw8j\" (UniqueName: \"kubernetes.io/projected/f0470911-5d99-4cf3-bf51-f509a2ec3184-kube-api-access-vvw8j\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn\" (UID: \"f0470911-5d99-4cf3-bf51-f509a2ec3184\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn"
Jul 01 21:56:04.854896 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:04.854880    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0470911-5d99-4cf3-bf51-f509a2ec3184-tmp-dir\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn\" (UID: \"f0470911-5d99-4cf3-bf51-f509a2ec3184\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn"
Jul 01 21:56:04.855093 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:04.854910    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0470911-5d99-4cf3-bf51-f509a2ec3184-home\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn\" (UID: \"f0470911-5d99-4cf3-bf51-f509a2ec3184\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn"
Jul 01 21:56:04.855226 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:04.855203    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0470911-5d99-4cf3-bf51-f509a2ec3184-model-cache\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn\" (UID: \"f0470911-5d99-4cf3-bf51-f509a2ec3184\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn"
Jul 01 21:56:04.855300 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:04.855235    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0470911-5d99-4cf3-bf51-f509a2ec3184-home\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn\" (UID: \"f0470911-5d99-4cf3-bf51-f509a2ec3184\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn"
Jul 01 21:56:04.855300 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:04.855289    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0470911-5d99-4cf3-bf51-f509a2ec3184-tmp-dir\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn\" (UID: \"f0470911-5d99-4cf3-bf51-f509a2ec3184\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn"
Jul 01 21:56:04.856952 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:04.856934    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0470911-5d99-4cf3-bf51-f509a2ec3184-dshm\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn\" (UID: \"f0470911-5d99-4cf3-bf51-f509a2ec3184\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn"
Jul 01 21:56:04.857275 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:04.857253    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0470911-5d99-4cf3-bf51-f509a2ec3184-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn\" (UID: \"f0470911-5d99-4cf3-bf51-f509a2ec3184\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn"
Jul 01 21:56:04.863639 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:04.863603    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvw8j\" (UniqueName: \"kubernetes.io/projected/f0470911-5d99-4cf3-bf51-f509a2ec3184-kube-api-access-vvw8j\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn\" (UID: \"f0470911-5d99-4cf3-bf51-f509a2ec3184\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn"
Jul 01 21:56:04.992567 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:04.992510    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn"
Jul 01 21:56:05.116968 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:05.116943    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn"]
Jul 01 21:56:05.119663 ip-10-0-140-104 kubenswrapper[2571]: W0701 21:56:05.119621    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0470911_5d99_4cf3_bf51_f509a2ec3184.slice/crio-cab34dfd4f11d09b545a85bb3e55694ea8942cfc04fd39301bf399d25ddde1f5 WatchSource:0}: Error finding container cab34dfd4f11d09b545a85bb3e55694ea8942cfc04fd39301bf399d25ddde1f5: Status 404 returned error can't find the container with id cab34dfd4f11d09b545a85bb3e55694ea8942cfc04fd39301bf399d25ddde1f5
Jul 01 21:56:05.971554 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:05.971522    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn" event={"ID":"f0470911-5d99-4cf3-bf51-f509a2ec3184","Type":"ContainerStarted","Data":"20277e80ab9e25ff4fd59cb9972bae534160cfdfdc6669987a9f7a8e65f01ae2"}
Jul 01 21:56:05.971924 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:05.971569    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn" event={"ID":"f0470911-5d99-4cf3-bf51-f509a2ec3184","Type":"ContainerStarted","Data":"cab34dfd4f11d09b545a85bb3e55694ea8942cfc04fd39301bf399d25ddde1f5"}
Jul 01 21:56:05.988356 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:05.988312    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn" podStartSLOduration=1.9882979349999998 podStartE2EDuration="1.988297935s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 21:56:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 21:56:05.987740686 +0000 UTC m=+4184.500215874" watchObservedRunningTime="2026-07-01 21:56:05.988297935 +0000 UTC m=+4184.500773122"
Jul 01 21:56:05.992793 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:05.992775    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn"
Jul 01 21:56:14.993543 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:14.993510    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn"
Jul 01 21:56:15.005120 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:15.005100    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn"
Jul 01 21:56:15.019835 ip-10-0-140-104 kubenswrapper[2571]: I0701 21:56:15.019816    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn"
Jul 01 22:01:08.793544 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:08.793511    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn"]
Jul 01 22:01:23.813427 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:23.813378    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn" podUID="f0470911-5d99-4cf3-bf51-f509a2ec3184" containerName="main" containerID="cri-o://20277e80ab9e25ff4fd59cb9972bae534160cfdfdc6669987a9f7a8e65f01ae2" gracePeriod=45
Jul 01 22:01:24.049856 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:24.049835    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn"
Jul 01 22:01:24.167003 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:24.166981    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0470911-5d99-4cf3-bf51-f509a2ec3184-home\") pod \"f0470911-5d99-4cf3-bf51-f509a2ec3184\" (UID: \"f0470911-5d99-4cf3-bf51-f509a2ec3184\") "
Jul 01 22:01:24.167141 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:24.167013    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0470911-5d99-4cf3-bf51-f509a2ec3184-tls-certs\") pod \"f0470911-5d99-4cf3-bf51-f509a2ec3184\" (UID: \"f0470911-5d99-4cf3-bf51-f509a2ec3184\") "
Jul 01 22:01:24.167141 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:24.167040    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvw8j\" (UniqueName: \"kubernetes.io/projected/f0470911-5d99-4cf3-bf51-f509a2ec3184-kube-api-access-vvw8j\") pod \"f0470911-5d99-4cf3-bf51-f509a2ec3184\" (UID: \"f0470911-5d99-4cf3-bf51-f509a2ec3184\") "
Jul 01 22:01:24.167141 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:24.167062    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0470911-5d99-4cf3-bf51-f509a2ec3184-dshm\") pod \"f0470911-5d99-4cf3-bf51-f509a2ec3184\" (UID: \"f0470911-5d99-4cf3-bf51-f509a2ec3184\") "
Jul 01 22:01:24.167141 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:24.167084    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0470911-5d99-4cf3-bf51-f509a2ec3184-model-cache\") pod \"f0470911-5d99-4cf3-bf51-f509a2ec3184\" (UID: \"f0470911-5d99-4cf3-bf51-f509a2ec3184\") "
Jul 01 22:01:24.167141 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:24.167114    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0470911-5d99-4cf3-bf51-f509a2ec3184-tmp-dir\") pod \"f0470911-5d99-4cf3-bf51-f509a2ec3184\" (UID: \"f0470911-5d99-4cf3-bf51-f509a2ec3184\") "
Jul 01 22:01:24.167390 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:24.167270    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0470911-5d99-4cf3-bf51-f509a2ec3184-home" (OuterVolumeSpecName: "home") pod "f0470911-5d99-4cf3-bf51-f509a2ec3184" (UID: "f0470911-5d99-4cf3-bf51-f509a2ec3184"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 22:01:24.167456 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:24.167417    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0470911-5d99-4cf3-bf51-f509a2ec3184-model-cache" (OuterVolumeSpecName: "model-cache") pod "f0470911-5d99-4cf3-bf51-f509a2ec3184" (UID: "f0470911-5d99-4cf3-bf51-f509a2ec3184"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 22:01:24.167508 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:24.167433    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0470911-5d99-4cf3-bf51-f509a2ec3184-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f0470911-5d99-4cf3-bf51-f509a2ec3184" (UID: "f0470911-5d99-4cf3-bf51-f509a2ec3184"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 22:01:24.169041 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:24.169011    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0470911-5d99-4cf3-bf51-f509a2ec3184-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f0470911-5d99-4cf3-bf51-f509a2ec3184" (UID: "f0470911-5d99-4cf3-bf51-f509a2ec3184"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 22:01:24.169204 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:24.169181    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0470911-5d99-4cf3-bf51-f509a2ec3184-dshm" (OuterVolumeSpecName: "dshm") pod "f0470911-5d99-4cf3-bf51-f509a2ec3184" (UID: "f0470911-5d99-4cf3-bf51-f509a2ec3184"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 22:01:24.169204 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:24.169194    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0470911-5d99-4cf3-bf51-f509a2ec3184-kube-api-access-vvw8j" (OuterVolumeSpecName: "kube-api-access-vvw8j") pod "f0470911-5d99-4cf3-bf51-f509a2ec3184" (UID: "f0470911-5d99-4cf3-bf51-f509a2ec3184"). InnerVolumeSpecName "kube-api-access-vvw8j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 22:01:24.268124 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:24.268103    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0470911-5d99-4cf3-bf51-f509a2ec3184-home\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 22:01:24.268230 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:24.268126    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0470911-5d99-4cf3-bf51-f509a2ec3184-tls-certs\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 22:01:24.268230 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:24.268140    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvw8j\" (UniqueName: \"kubernetes.io/projected/f0470911-5d99-4cf3-bf51-f509a2ec3184-kube-api-access-vvw8j\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 22:01:24.268230 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:24.268150    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0470911-5d99-4cf3-bf51-f509a2ec3184-dshm\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 22:01:24.268230 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:24.268163    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0470911-5d99-4cf3-bf51-f509a2ec3184-model-cache\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 22:01:24.268230 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:24.268197    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0470911-5d99-4cf3-bf51-f509a2ec3184-tmp-dir\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 22:01:24.280109 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:24.280086    2571 generic.go:356] "Generic (PLEG): container finished" podID="f0470911-5d99-4cf3-bf51-f509a2ec3184" containerID="20277e80ab9e25ff4fd59cb9972bae534160cfdfdc6669987a9f7a8e65f01ae2" exitCode=0
Jul 01 22:01:24.280195 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:24.280133    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn"
Jul 01 22:01:24.280195 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:24.280164    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn" event={"ID":"f0470911-5d99-4cf3-bf51-f509a2ec3184","Type":"ContainerDied","Data":"20277e80ab9e25ff4fd59cb9972bae534160cfdfdc6669987a9f7a8e65f01ae2"}
Jul 01 22:01:24.280263 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:24.280211    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn" event={"ID":"f0470911-5d99-4cf3-bf51-f509a2ec3184","Type":"ContainerDied","Data":"cab34dfd4f11d09b545a85bb3e55694ea8942cfc04fd39301bf399d25ddde1f5"}
Jul 01 22:01:24.280263 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:24.280232    2571 scope.go:117] "RemoveContainer" containerID="20277e80ab9e25ff4fd59cb9972bae534160cfdfdc6669987a9f7a8e65f01ae2"
Jul 01 22:01:24.288883 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:24.288858    2571 scope.go:117] "RemoveContainer" containerID="20277e80ab9e25ff4fd59cb9972bae534160cfdfdc6669987a9f7a8e65f01ae2"
Jul 01 22:01:24.289110 ip-10-0-140-104 kubenswrapper[2571]: E0701 22:01:24.289094    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20277e80ab9e25ff4fd59cb9972bae534160cfdfdc6669987a9f7a8e65f01ae2\": container with ID starting with 20277e80ab9e25ff4fd59cb9972bae534160cfdfdc6669987a9f7a8e65f01ae2 not found: ID does not exist" containerID="20277e80ab9e25ff4fd59cb9972bae534160cfdfdc6669987a9f7a8e65f01ae2"
Jul 01 22:01:24.289164 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:24.289115    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20277e80ab9e25ff4fd59cb9972bae534160cfdfdc6669987a9f7a8e65f01ae2"} err="failed to get container status \"20277e80ab9e25ff4fd59cb9972bae534160cfdfdc6669987a9f7a8e65f01ae2\": rpc error: code = NotFound desc = could not find container \"20277e80ab9e25ff4fd59cb9972bae534160cfdfdc6669987a9f7a8e65f01ae2\": container with ID starting with 20277e80ab9e25ff4fd59cb9972bae534160cfdfdc6669987a9f7a8e65f01ae2 not found: ID does not exist"
Jul 01 22:01:24.301094 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:24.301070    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn"]
Jul 01 22:01:24.304991 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:24.304972    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn"]
Jul 01 22:01:25.011438 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:25.011405    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-w4dpn" podUID="f0470911-5d99-4cf3-bf51-f509a2ec3184" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 01 22:01:25.266825 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:25.266777    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8"]
Jul 01 22:01:25.267062 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:25.267050    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0470911-5d99-4cf3-bf51-f509a2ec3184" containerName="main"
Jul 01 22:01:25.267109 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:25.267064    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0470911-5d99-4cf3-bf51-f509a2ec3184" containerName="main"
Jul 01 22:01:25.267142 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:25.267127    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0470911-5d99-4cf3-bf51-f509a2ec3184" containerName="main"
Jul 01 22:01:25.271361 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:25.271347    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8"
Jul 01 22:01:25.274011 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:25.273991    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-76tz7\""
Jul 01 22:01:25.274147 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:25.274131    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 01 22:01:25.275308 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:25.275293    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"rolling-upgrade-test-kserve-self-signed-certs\""
Jul 01 22:01:25.275398 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:25.275330    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 01 22:01:25.278264 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:25.278249    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8"]
Jul 01 22:01:25.376879 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:25.376860    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54ed781f-d7d0-4dae-a301-0693093e8227-tmp-dir\") pod \"rolling-upgrade-test-kserve-859df57bcb-glml8\" (UID: \"54ed781f-d7d0-4dae-a301-0693093e8227\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8"
Jul 01 22:01:25.376975 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:25.376898    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54ed781f-d7d0-4dae-a301-0693093e8227-tls-certs\") pod \"rolling-upgrade-test-kserve-859df57bcb-glml8\" (UID: \"54ed781f-d7d0-4dae-a301-0693093e8227\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8"
Jul 01 22:01:25.376975 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:25.376917    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54ed781f-d7d0-4dae-a301-0693093e8227-model-cache\") pod \"rolling-upgrade-test-kserve-859df57bcb-glml8\" (UID: \"54ed781f-d7d0-4dae-a301-0693093e8227\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8"
Jul 01 22:01:25.376975 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:25.376942    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hj8m\" (UniqueName: \"kubernetes.io/projected/54ed781f-d7d0-4dae-a301-0693093e8227-kube-api-access-9hj8m\") pod \"rolling-upgrade-test-kserve-859df57bcb-glml8\" (UID: \"54ed781f-d7d0-4dae-a301-0693093e8227\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8"
Jul 01 22:01:25.377094 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:25.376989    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54ed781f-d7d0-4dae-a301-0693093e8227-home\") pod \"rolling-upgrade-test-kserve-859df57bcb-glml8\" (UID: \"54ed781f-d7d0-4dae-a301-0693093e8227\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8"
Jul 01 22:01:25.377094 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:25.377012    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54ed781f-d7d0-4dae-a301-0693093e8227-dshm\") pod \"rolling-upgrade-test-kserve-859df57bcb-glml8\" (UID: \"54ed781f-d7d0-4dae-a301-0693093e8227\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8"
Jul 01 22:01:25.477820 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:25.477797    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hj8m\" (UniqueName: \"kubernetes.io/projected/54ed781f-d7d0-4dae-a301-0693093e8227-kube-api-access-9hj8m\") pod \"rolling-upgrade-test-kserve-859df57bcb-glml8\" (UID: \"54ed781f-d7d0-4dae-a301-0693093e8227\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8"
Jul 01 22:01:25.477917 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:25.477833    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54ed781f-d7d0-4dae-a301-0693093e8227-home\") pod \"rolling-upgrade-test-kserve-859df57bcb-glml8\" (UID: \"54ed781f-d7d0-4dae-a301-0693093e8227\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8"
Jul 01 22:01:25.477917 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:25.477856    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54ed781f-d7d0-4dae-a301-0693093e8227-dshm\") pod \"rolling-upgrade-test-kserve-859df57bcb-glml8\" (UID: \"54ed781f-d7d0-4dae-a301-0693093e8227\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8"
Jul 01 22:01:25.478027 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:25.477910    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54ed781f-d7d0-4dae-a301-0693093e8227-tmp-dir\") pod \"rolling-upgrade-test-kserve-859df57bcb-glml8\" (UID: \"54ed781f-d7d0-4dae-a301-0693093e8227\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8"
Jul 01 22:01:25.478027 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:25.478002    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54ed781f-d7d0-4dae-a301-0693093e8227-tls-certs\") pod \"rolling-upgrade-test-kserve-859df57bcb-glml8\" (UID: \"54ed781f-d7d0-4dae-a301-0693093e8227\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8"
Jul 01 22:01:25.478027 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:25.478025    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54ed781f-d7d0-4dae-a301-0693093e8227-model-cache\") pod \"rolling-upgrade-test-kserve-859df57bcb-glml8\" (UID: \"54ed781f-d7d0-4dae-a301-0693093e8227\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8"
Jul 01 22:01:25.478187 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:25.478168    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54ed781f-d7d0-4dae-a301-0693093e8227-home\") pod \"rolling-upgrade-test-kserve-859df57bcb-glml8\" (UID: \"54ed781f-d7d0-4dae-a301-0693093e8227\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8"
Jul 01 22:01:25.478309 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:25.478292    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54ed781f-d7d0-4dae-a301-0693093e8227-tmp-dir\") pod \"rolling-upgrade-test-kserve-859df57bcb-glml8\" (UID: \"54ed781f-d7d0-4dae-a301-0693093e8227\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8"
Jul 01 22:01:25.478309 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:25.478304    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54ed781f-d7d0-4dae-a301-0693093e8227-model-cache\") pod \"rolling-upgrade-test-kserve-859df57bcb-glml8\" (UID: \"54ed781f-d7d0-4dae-a301-0693093e8227\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8"
Jul 01 22:01:25.480032 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:25.480011    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54ed781f-d7d0-4dae-a301-0693093e8227-dshm\") pod \"rolling-upgrade-test-kserve-859df57bcb-glml8\" (UID: \"54ed781f-d7d0-4dae-a301-0693093e8227\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8"
Jul 01 22:01:25.480289 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:25.480272    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54ed781f-d7d0-4dae-a301-0693093e8227-tls-certs\") pod \"rolling-upgrade-test-kserve-859df57bcb-glml8\" (UID: \"54ed781f-d7d0-4dae-a301-0693093e8227\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8"
Jul 01 22:01:25.486024 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:25.486003    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hj8m\" (UniqueName: \"kubernetes.io/projected/54ed781f-d7d0-4dae-a301-0693093e8227-kube-api-access-9hj8m\") pod \"rolling-upgrade-test-kserve-859df57bcb-glml8\" (UID: \"54ed781f-d7d0-4dae-a301-0693093e8227\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8"
Jul 01 22:01:25.581280 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:25.581235    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8"
Jul 01 22:01:25.701121 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:25.701097    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8"]
Jul 01 22:01:25.702732 ip-10-0-140-104 kubenswrapper[2571]: W0701 22:01:25.702706    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54ed781f_d7d0_4dae_a301_0693093e8227.slice/crio-fed47edacd17cefd8fe4a6d4c1cfe2fdbf26344df3063dcbc580ada8e2268c02 WatchSource:0}: Error finding container fed47edacd17cefd8fe4a6d4c1cfe2fdbf26344df3063dcbc580ada8e2268c02: Status 404 returned error can't find the container with id fed47edacd17cefd8fe4a6d4c1cfe2fdbf26344df3063dcbc580ada8e2268c02
Jul 01 22:01:25.838649 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:25.838561    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0470911-5d99-4cf3-bf51-f509a2ec3184" path="/var/lib/kubelet/pods/f0470911-5d99-4cf3-bf51-f509a2ec3184/volumes"
Jul 01 22:01:26.290126 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:26.290103    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8" event={"ID":"54ed781f-d7d0-4dae-a301-0693093e8227","Type":"ContainerStarted","Data":"b5cb4c571c98f23d5a464e18ae2a3eb6c81652a93b887423311d7d329483872f"}
Jul 01 22:01:26.290449 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:26.290144    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8" event={"ID":"54ed781f-d7d0-4dae-a301-0693093e8227","Type":"ContainerStarted","Data":"fed47edacd17cefd8fe4a6d4c1cfe2fdbf26344df3063dcbc580ada8e2268c02"}
Jul 01 22:01:26.307346 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:26.307297    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8" podStartSLOduration=1.307279071 podStartE2EDuration="1.307279071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:01:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:01:26.307275772 +0000 UTC m=+4504.819750960" watchObservedRunningTime="2026-07-01 22:01:26.307279071 +0000 UTC m=+4504.819754259"
Jul 01 22:01:26.582253 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:26.582188    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8"
Jul 01 22:01:35.582275 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:35.582244    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8"
Jul 01 22:01:35.593714 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:35.593690    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8"
Jul 01 22:01:36.341448 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:36.341403    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8"
Jul 01 22:01:50.280047 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:50.280014    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc"]
Jul 01 22:01:50.283193 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:50.283178    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc"
Jul 01 22:01:50.287445 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:50.287425    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-ha-replicas-test-epp-sa-dockercfg-zbsdm\""
Jul 01 22:01:50.287445 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:50.287437    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-ha-replicas-test-kserve-self-signed-certs\""
Jul 01 22:01:50.292443 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:50.292420    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc"]
Jul 01 22:01:50.354233 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:50.354207    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-886bq\" (UniqueName: \"kubernetes.io/projected/87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2-kube-api-access-886bq\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc\" (UID: \"87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc"
Jul 01 22:01:50.354340 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:50.354269    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc\" (UID: \"87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc"
Jul 01 22:01:50.354340 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:50.354311    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2-tokenizer-uds\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc\" (UID: \"87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc"
Jul 01 22:01:50.455328 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:50.455305    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc\" (UID: \"87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc"
Jul 01 22:01:50.455428 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:50.455345    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2-tokenizer-uds\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc\" (UID: \"87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc"
Jul 01 22:01:50.455428 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:50.455367    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-886bq\" (UniqueName: \"kubernetes.io/projected/87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2-kube-api-access-886bq\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc\" (UID: \"87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc"
Jul 01 22:01:50.455759 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:50.455740    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2-tokenizer-uds\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc\" (UID: \"87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc"
Jul 01 22:01:50.457777 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:50.457758    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc\" (UID: \"87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc"
Jul 01 22:01:50.472595 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:50.472575    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-886bq\" (UniqueName: \"kubernetes.io/projected/87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2-kube-api-access-886bq\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc\" (UID: \"87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc"
Jul 01 22:01:50.592777 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:50.592721    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc"
Jul 01 22:01:50.721846 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:50.721816    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc"]
Jul 01 22:01:50.723756 ip-10-0-140-104 kubenswrapper[2571]: W0701 22:01:50.723726    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87dbd6f0_8bbe_4ad6_a354_3c142bbc0df2.slice/crio-eae4ce62507864055f8050c6191029ab7a4e1ca547b4fdcc07cee914681758d9 WatchSource:0}: Error finding container eae4ce62507864055f8050c6191029ab7a4e1ca547b4fdcc07cee914681758d9: Status 404 returned error can't find the container with id eae4ce62507864055f8050c6191029ab7a4e1ca547b4fdcc07cee914681758d9
Jul 01 22:01:51.394189 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:51.394161    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc" event={"ID":"87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2","Type":"ContainerStarted","Data":"ebefcb29af565e86f4376f70ba846bce50121b0abd07974ed860303b9a4fcbce"}
Jul 01 22:01:51.394599 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:51.394203    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc" event={"ID":"87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2","Type":"ContainerStarted","Data":"eae4ce62507864055f8050c6191029ab7a4e1ca547b4fdcc07cee914681758d9"}
Jul 01 22:01:51.394599 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:51.394234    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc"
Jul 01 22:01:51.418586 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:51.418545    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc" podStartSLOduration=1.418529199 podStartE2EDuration="1.418529199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:01:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:01:51.417154281 +0000 UTC m=+4529.929629468" watchObservedRunningTime="2026-07-01 22:01:51.418529199 +0000 UTC m=+4529.931004383"
Jul 01 22:01:52.489974 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:01:52.489941    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8"]
Jul 01 22:02:07.518434 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:02:07.518400    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8" podUID="54ed781f-d7d0-4dae-a301-0693093e8227" containerName="main" containerID="cri-o://b5cb4c571c98f23d5a464e18ae2a3eb6c81652a93b887423311d7d329483872f" gracePeriod=45
Jul 01 22:02:07.757065 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:02:07.757039    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8"
Jul 01 22:02:07.787423 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:02:07.787373    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54ed781f-d7d0-4dae-a301-0693093e8227-tmp-dir\") pod \"54ed781f-d7d0-4dae-a301-0693093e8227\" (UID: \"54ed781f-d7d0-4dae-a301-0693093e8227\") "
Jul 01 22:02:07.787423 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:02:07.787404    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9hj8m\" (UniqueName: \"kubernetes.io/projected/54ed781f-d7d0-4dae-a301-0693093e8227-kube-api-access-9hj8m\") pod \"54ed781f-d7d0-4dae-a301-0693093e8227\" (UID: \"54ed781f-d7d0-4dae-a301-0693093e8227\") "
Jul 01 22:02:07.787562 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:02:07.787427    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54ed781f-d7d0-4dae-a301-0693093e8227-home\") pod \"54ed781f-d7d0-4dae-a301-0693093e8227\" (UID: \"54ed781f-d7d0-4dae-a301-0693093e8227\") "
Jul 01 22:02:07.787562 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:02:07.787549    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54ed781f-d7d0-4dae-a301-0693093e8227-tls-certs\") pod \"54ed781f-d7d0-4dae-a301-0693093e8227\" (UID: \"54ed781f-d7d0-4dae-a301-0693093e8227\") "
Jul 01 22:02:07.787673 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:02:07.787651    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54ed781f-d7d0-4dae-a301-0693093e8227-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "54ed781f-d7d0-4dae-a301-0693093e8227" (UID: "54ed781f-d7d0-4dae-a301-0693093e8227"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 22:02:07.787673 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:02:07.787669    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54ed781f-d7d0-4dae-a301-0693093e8227-model-cache\") pod \"54ed781f-d7d0-4dae-a301-0693093e8227\" (UID: \"54ed781f-d7d0-4dae-a301-0693093e8227\") "
Jul 01 22:02:07.787785 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:02:07.787678    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54ed781f-d7d0-4dae-a301-0693093e8227-home" (OuterVolumeSpecName: "home") pod "54ed781f-d7d0-4dae-a301-0693093e8227" (UID: "54ed781f-d7d0-4dae-a301-0693093e8227"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 22:02:07.787785 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:02:07.787692    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54ed781f-d7d0-4dae-a301-0693093e8227-dshm\") pod \"54ed781f-d7d0-4dae-a301-0693093e8227\" (UID: \"54ed781f-d7d0-4dae-a301-0693093e8227\") "
Jul 01 22:02:07.787884 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:02:07.787840    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54ed781f-d7d0-4dae-a301-0693093e8227-model-cache" (OuterVolumeSpecName: "model-cache") pod "54ed781f-d7d0-4dae-a301-0693093e8227" (UID: "54ed781f-d7d0-4dae-a301-0693093e8227"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 22:02:07.787953 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:02:07.787940    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54ed781f-d7d0-4dae-a301-0693093e8227-model-cache\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 22:02:07.788009 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:02:07.787963    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54ed781f-d7d0-4dae-a301-0693093e8227-tmp-dir\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 22:02:07.788009 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:02:07.787980    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54ed781f-d7d0-4dae-a301-0693093e8227-home\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 22:02:07.789642 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:02:07.789597    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54ed781f-d7d0-4dae-a301-0693093e8227-kube-api-access-9hj8m" (OuterVolumeSpecName: "kube-api-access-9hj8m") pod "54ed781f-d7d0-4dae-a301-0693093e8227" (UID: "54ed781f-d7d0-4dae-a301-0693093e8227"). InnerVolumeSpecName "kube-api-access-9hj8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 22:02:07.789762 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:02:07.789711    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54ed781f-d7d0-4dae-a301-0693093e8227-dshm" (OuterVolumeSpecName: "dshm") pod "54ed781f-d7d0-4dae-a301-0693093e8227" (UID: "54ed781f-d7d0-4dae-a301-0693093e8227"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 22:02:07.789833 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:02:07.789778    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54ed781f-d7d0-4dae-a301-0693093e8227-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "54ed781f-d7d0-4dae-a301-0693093e8227" (UID: "54ed781f-d7d0-4dae-a301-0693093e8227"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 22:02:07.888921 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:02:07.888901    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54ed781f-d7d0-4dae-a301-0693093e8227-tls-certs\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 22:02:07.888921 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:02:07.888921    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54ed781f-d7d0-4dae-a301-0693093e8227-dshm\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 22:02:07.889052 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:02:07.888929    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hj8m\" (UniqueName: \"kubernetes.io/projected/54ed781f-d7d0-4dae-a301-0693093e8227-kube-api-access-9hj8m\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 22:02:08.473216 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:02:08.473187    2571 generic.go:356] "Generic (PLEG): container finished" podID="54ed781f-d7d0-4dae-a301-0693093e8227" containerID="b5cb4c571c98f23d5a464e18ae2a3eb6c81652a93b887423311d7d329483872f" exitCode=0
Jul 01 22:02:08.473358 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:02:08.473258    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8" event={"ID":"54ed781f-d7d0-4dae-a301-0693093e8227","Type":"ContainerDied","Data":"b5cb4c571c98f23d5a464e18ae2a3eb6c81652a93b887423311d7d329483872f"}
Jul 01 22:02:08.473358 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:02:08.473286    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8"
Jul 01 22:02:08.473358 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:02:08.473295    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8" event={"ID":"54ed781f-d7d0-4dae-a301-0693093e8227","Type":"ContainerDied","Data":"fed47edacd17cefd8fe4a6d4c1cfe2fdbf26344df3063dcbc580ada8e2268c02"}
Jul 01 22:02:08.473358 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:02:08.473315    2571 scope.go:117] "RemoveContainer" containerID="b5cb4c571c98f23d5a464e18ae2a3eb6c81652a93b887423311d7d329483872f"
Jul 01 22:02:08.481367 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:02:08.481343    2571 scope.go:117] "RemoveContainer" containerID="b5cb4c571c98f23d5a464e18ae2a3eb6c81652a93b887423311d7d329483872f"
Jul 01 22:02:08.481651 ip-10-0-140-104 kubenswrapper[2571]: E0701 22:02:08.481619    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5cb4c571c98f23d5a464e18ae2a3eb6c81652a93b887423311d7d329483872f\": container with ID starting with b5cb4c571c98f23d5a464e18ae2a3eb6c81652a93b887423311d7d329483872f not found: ID does not exist" containerID="b5cb4c571c98f23d5a464e18ae2a3eb6c81652a93b887423311d7d329483872f"
Jul 01 22:02:08.481703 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:02:08.481658    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5cb4c571c98f23d5a464e18ae2a3eb6c81652a93b887423311d7d329483872f"} err="failed to get container status \"b5cb4c571c98f23d5a464e18ae2a3eb6c81652a93b887423311d7d329483872f\": rpc error: code = NotFound desc = could not find container \"b5cb4c571c98f23d5a464e18ae2a3eb6c81652a93b887423311d7d329483872f\": container with ID starting with b5cb4c571c98f23d5a464e18ae2a3eb6c81652a93b887423311d7d329483872f not found: ID does not exist"
Jul 01 22:02:08.490365 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:02:08.490344    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8"]
Jul 01 22:02:08.493438 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:02:08.493417    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-glml8"]
Jul 01 22:02:09.838607 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:02:09.838574    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54ed781f-d7d0-4dae-a301-0693093e8227" path="/var/lib/kubelet/pods/54ed781f-d7d0-4dae-a301-0693093e8227/volumes"
Jul 01 22:02:22.401347 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:02:22.401323    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc"
Jul 01 22:03:26.280829 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:26.280802    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk"]
Jul 01 22:03:26.281214 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:26.281117    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54ed781f-d7d0-4dae-a301-0693093e8227" containerName="main"
Jul 01 22:03:26.281214 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:26.281128    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="54ed781f-d7d0-4dae-a301-0693093e8227" containerName="main"
Jul 01 22:03:26.281214 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:26.281186    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="54ed781f-d7d0-4dae-a301-0693093e8227" containerName="main"
Jul 01 22:03:26.283994 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:26.283980    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk"
Jul 01 22:03:26.286971 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:26.286950    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-v06-pd-migration-test-kserve-self-signed-certs\""
Jul 01 22:03:26.295053 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:26.295034    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk"]
Jul 01 22:03:26.346323 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:26.346299    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44254112-edab-4268-8b4b-9c43cb7e371b-home\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk\" (UID: \"44254112-edab-4268-8b4b-9c43cb7e371b\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk"
Jul 01 22:03:26.346420 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:26.346328    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44254112-edab-4268-8b4b-9c43cb7e371b-dshm\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk\" (UID: \"44254112-edab-4268-8b4b-9c43cb7e371b\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk"
Jul 01 22:03:26.346420 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:26.346344    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn9x2\" (UniqueName: \"kubernetes.io/projected/44254112-edab-4268-8b4b-9c43cb7e371b-kube-api-access-cn9x2\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk\" (UID: \"44254112-edab-4268-8b4b-9c43cb7e371b\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk"
Jul 01 22:03:26.346420 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:26.346368    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44254112-edab-4268-8b4b-9c43cb7e371b-model-cache\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk\" (UID: \"44254112-edab-4268-8b4b-9c43cb7e371b\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk"
Jul 01 22:03:26.346553 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:26.346450    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44254112-edab-4268-8b4b-9c43cb7e371b-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk\" (UID: \"44254112-edab-4268-8b4b-9c43cb7e371b\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk"
Jul 01 22:03:26.346553 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:26.346483    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44254112-edab-4268-8b4b-9c43cb7e371b-tmp-dir\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk\" (UID: \"44254112-edab-4268-8b4b-9c43cb7e371b\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk"
Jul 01 22:03:26.446869 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:26.446844    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44254112-edab-4268-8b4b-9c43cb7e371b-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk\" (UID: \"44254112-edab-4268-8b4b-9c43cb7e371b\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk"
Jul 01 22:03:26.446963 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:26.446878    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44254112-edab-4268-8b4b-9c43cb7e371b-tmp-dir\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk\" (UID: \"44254112-edab-4268-8b4b-9c43cb7e371b\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk"
Jul 01 22:03:26.446963 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:26.446945    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44254112-edab-4268-8b4b-9c43cb7e371b-home\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk\" (UID: \"44254112-edab-4268-8b4b-9c43cb7e371b\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk"
Jul 01 22:03:26.447073 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:26.446975    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44254112-edab-4268-8b4b-9c43cb7e371b-dshm\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk\" (UID: \"44254112-edab-4268-8b4b-9c43cb7e371b\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk"
Jul 01 22:03:26.447073 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:26.446998    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn9x2\" (UniqueName: \"kubernetes.io/projected/44254112-edab-4268-8b4b-9c43cb7e371b-kube-api-access-cn9x2\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk\" (UID: \"44254112-edab-4268-8b4b-9c43cb7e371b\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk"
Jul 01 22:03:26.447073 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:26.447037    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44254112-edab-4268-8b4b-9c43cb7e371b-model-cache\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk\" (UID: \"44254112-edab-4268-8b4b-9c43cb7e371b\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk"
Jul 01 22:03:26.447401 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:26.447371    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44254112-edab-4268-8b4b-9c43cb7e371b-tmp-dir\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk\" (UID: \"44254112-edab-4268-8b4b-9c43cb7e371b\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk"
Jul 01 22:03:26.447505 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:26.447377    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44254112-edab-4268-8b4b-9c43cb7e371b-model-cache\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk\" (UID: \"44254112-edab-4268-8b4b-9c43cb7e371b\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk"
Jul 01 22:03:26.447505 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:26.447420    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44254112-edab-4268-8b4b-9c43cb7e371b-home\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk\" (UID: \"44254112-edab-4268-8b4b-9c43cb7e371b\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk"
Jul 01 22:03:26.449249 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:26.449230    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44254112-edab-4268-8b4b-9c43cb7e371b-dshm\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk\" (UID: \"44254112-edab-4268-8b4b-9c43cb7e371b\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk"
Jul 01 22:03:26.449344 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:26.449327    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44254112-edab-4268-8b4b-9c43cb7e371b-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk\" (UID: \"44254112-edab-4268-8b4b-9c43cb7e371b\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk"
Jul 01 22:03:26.455912 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:26.455891    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn9x2\" (UniqueName: \"kubernetes.io/projected/44254112-edab-4268-8b4b-9c43cb7e371b-kube-api-access-cn9x2\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk\" (UID: \"44254112-edab-4268-8b4b-9c43cb7e371b\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk"
Jul 01 22:03:26.593771 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:26.593722    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk"
Jul 01 22:03:26.719774 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:26.719748    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk"]
Jul 01 22:03:26.721475 ip-10-0-140-104 kubenswrapper[2571]: W0701 22:03:26.721449    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44254112_edab_4268_8b4b_9c43cb7e371b.slice/crio-97ce099ff6c2869ce8059c78cefa956ccb811d10527e2bbc8edd0062b9da1083 WatchSource:0}: Error finding container 97ce099ff6c2869ce8059c78cefa956ccb811d10527e2bbc8edd0062b9da1083: Status 404 returned error can't find the container with id 97ce099ff6c2869ce8059c78cefa956ccb811d10527e2bbc8edd0062b9da1083
Jul 01 22:03:26.789885 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:26.789851    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk" event={"ID":"44254112-edab-4268-8b4b-9c43cb7e371b","Type":"ContainerStarted","Data":"1feff14011ec6b13646ec3319f35411f7f7d0c939c4a5edacb16159c214a43dc"}
Jul 01 22:03:26.789996 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:26.789907    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk" event={"ID":"44254112-edab-4268-8b4b-9c43cb7e371b","Type":"ContainerStarted","Data":"97ce099ff6c2869ce8059c78cefa956ccb811d10527e2bbc8edd0062b9da1083"}
Jul 01 22:03:26.808898 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:26.808855    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk" podStartSLOduration=0.808839617 podStartE2EDuration="808.839617ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:03:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:03:26.808101177 +0000 UTC m=+4625.320576365" watchObservedRunningTime="2026-07-01 22:03:26.808839617 +0000 UTC m=+4625.321314815"
Jul 01 22:03:27.594584 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:27.594562    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk"
Jul 01 22:03:29.679112 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:29.679080    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc"]
Jul 01 22:03:29.688407 ip-10-0-140-104 kubenswrapper[2571]: E0701 22:03:29.688357    2571 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 01 22:03:29.688407 ip-10-0-140-104 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 01 22:03:29.688407 ip-10-0-140-104 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 01 22:03:29.688407 ip-10-0-140-104 kubenswrapper[2571]:  > containerID="ebefcb29af565e86f4376f70ba846bce50121b0abd07974ed860303b9a4fcbce" cmd=["/bin/sleep","15"]
Jul 01 22:03:29.688674 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:29.688427    2571 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc" message="" err=<
Jul 01 22:03:29.688674 ip-10-0-140-104 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 01 22:03:29.688674 ip-10-0-140-104 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 01 22:03:29.688674 ip-10-0-140-104 kubenswrapper[2571]:  >
Jul 01 22:03:29.688674 ip-10-0-140-104 kubenswrapper[2571]: E0701 22:03:29.688441    2571 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 01 22:03:29.688674 ip-10-0-140-104 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 01 22:03:29.688674 ip-10-0-140-104 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 01 22:03:29.688674 ip-10-0-140-104 kubenswrapper[2571]:  > pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc" podUID="87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2" containerName="main" containerID="cri-o://ebefcb29af565e86f4376f70ba846bce50121b0abd07974ed860303b9a4fcbce"
Jul 01 22:03:29.688674 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:29.688492    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc" podUID="87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2" containerName="main" containerID="cri-o://ebefcb29af565e86f4376f70ba846bce50121b0abd07974ed860303b9a4fcbce" gracePeriod=60
Jul 01 22:03:29.806400 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:29.806368    2571 generic.go:356] "Generic (PLEG): container finished" podID="87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2" containerID="ebefcb29af565e86f4376f70ba846bce50121b0abd07974ed860303b9a4fcbce" exitCode=0
Jul 01 22:03:29.806535 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:29.806442    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc" event={"ID":"87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2","Type":"ContainerDied","Data":"ebefcb29af565e86f4376f70ba846bce50121b0abd07974ed860303b9a4fcbce"}
Jul 01 22:03:29.937491 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:29.937440    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc"
Jul 01 22:03:29.975014 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:29.974995    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2-tokenizer-uds\") pod \"87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2\" (UID: \"87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2\") "
Jul 01 22:03:29.975107 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:29.975054    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-886bq\" (UniqueName: \"kubernetes.io/projected/87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2-kube-api-access-886bq\") pod \"87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2\" (UID: \"87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2\") "
Jul 01 22:03:29.975107 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:29.975100    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2-tls-certs\") pod \"87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2\" (UID: \"87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2\") "
Jul 01 22:03:29.975253 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:29.975228    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2" (UID: "87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 22:03:29.977042 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:29.977017    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2-kube-api-access-886bq" (OuterVolumeSpecName: "kube-api-access-886bq") pod "87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2" (UID: "87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2"). InnerVolumeSpecName "kube-api-access-886bq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 22:03:29.977120 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:29.977099    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2" (UID: "87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 22:03:30.076497 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:30.076478    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2-tls-certs\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 22:03:30.076497 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:30.076498    2571 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2-tokenizer-uds\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 22:03:30.076619 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:30.076507    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-886bq\" (UniqueName: \"kubernetes.io/projected/87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2-kube-api-access-886bq\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 22:03:30.813329 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:30.813299    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc"
Jul 01 22:03:30.813329 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:30.813306    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc" event={"ID":"87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2","Type":"ContainerDied","Data":"eae4ce62507864055f8050c6191029ab7a4e1ca547b4fdcc07cee914681758d9"}
Jul 01 22:03:30.813918 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:30.813373    2571 scope.go:117] "RemoveContainer" containerID="ebefcb29af565e86f4376f70ba846bce50121b0abd07974ed860303b9a4fcbce"
Jul 01 22:03:30.834964 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:30.834943    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc"]
Jul 01 22:03:30.837578 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:30.837560    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dhg8tc"]
Jul 01 22:03:31.839472 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:31.839440    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2" path="/var/lib/kubelet/pods/87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2/volumes"
Jul 01 22:03:36.594861 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:36.594826    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk"
Jul 01 22:03:36.606536 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:36.606498    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk"
Jul 01 22:03:36.849784 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:03:36.849729    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk"
Jul 01 22:04:03.414760 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:03.414729    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567jsr2c"]
Jul 01 22:04:03.415311 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:03.415098    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2" containerName="main"
Jul 01 22:04:03.415311 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:03.415114    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2" containerName="main"
Jul 01 22:04:03.415311 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:03.415181    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="87dbd6f0-8bbe-4ad6-a354-3c142bbc0df2" containerName="main"
Jul 01 22:04:03.417895 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:03.417876    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567jsr2c"
Jul 01 22:04:03.420842 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:03.420822    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-custom-template-test-kserve-self-signed-certs\""
Jul 01 22:04:03.421961 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:03.421938    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-custom-template-test-epp-sa-dockercfg-k5774\""
Jul 01 22:04:03.428364 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:03.428343    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567jsr2c"]
Jul 01 22:04:03.510880 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:03.510855    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5001e767-8581-44db-b2b9-4a6b1d787bbc-tls-certs\") pod \"scheduler-custom-template-test-kserve-router-scheduler-567jsr2c\" (UID: \"5001e767-8581-44db-b2b9-4a6b1d787bbc\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567jsr2c"
Jul 01 22:04:03.510983 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:03.510900    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5001e767-8581-44db-b2b9-4a6b1d787bbc-tokenizer-uds\") pod \"scheduler-custom-template-test-kserve-router-scheduler-567jsr2c\" (UID: \"5001e767-8581-44db-b2b9-4a6b1d787bbc\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567jsr2c"
Jul 01 22:04:03.510983 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:03.510937    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ztpc\" (UniqueName: \"kubernetes.io/projected/5001e767-8581-44db-b2b9-4a6b1d787bbc-kube-api-access-6ztpc\") pod \"scheduler-custom-template-test-kserve-router-scheduler-567jsr2c\" (UID: \"5001e767-8581-44db-b2b9-4a6b1d787bbc\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567jsr2c"
Jul 01 22:04:03.510983 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:03.510980    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/5001e767-8581-44db-b2b9-4a6b1d787bbc-tokenizer-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-567jsr2c\" (UID: \"5001e767-8581-44db-b2b9-4a6b1d787bbc\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567jsr2c"
Jul 01 22:04:03.511112 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:03.511070    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/5001e767-8581-44db-b2b9-4a6b1d787bbc-cachi2-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-567jsr2c\" (UID: \"5001e767-8581-44db-b2b9-4a6b1d787bbc\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567jsr2c"
Jul 01 22:04:03.611847 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:03.611815    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/5001e767-8581-44db-b2b9-4a6b1d787bbc-cachi2-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-567jsr2c\" (UID: \"5001e767-8581-44db-b2b9-4a6b1d787bbc\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567jsr2c"
Jul 01 22:04:03.611940 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:03.611858    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5001e767-8581-44db-b2b9-4a6b1d787bbc-tls-certs\") pod \"scheduler-custom-template-test-kserve-router-scheduler-567jsr2c\" (UID: \"5001e767-8581-44db-b2b9-4a6b1d787bbc\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567jsr2c"
Jul 01 22:04:03.611940 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:03.611879    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5001e767-8581-44db-b2b9-4a6b1d787bbc-tokenizer-uds\") pod \"scheduler-custom-template-test-kserve-router-scheduler-567jsr2c\" (UID: \"5001e767-8581-44db-b2b9-4a6b1d787bbc\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567jsr2c"
Jul 01 22:04:03.611940 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:03.611899    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ztpc\" (UniqueName: \"kubernetes.io/projected/5001e767-8581-44db-b2b9-4a6b1d787bbc-kube-api-access-6ztpc\") pod \"scheduler-custom-template-test-kserve-router-scheduler-567jsr2c\" (UID: \"5001e767-8581-44db-b2b9-4a6b1d787bbc\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567jsr2c"
Jul 01 22:04:03.612054 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:03.612012    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/5001e767-8581-44db-b2b9-4a6b1d787bbc-tokenizer-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-567jsr2c\" (UID: \"5001e767-8581-44db-b2b9-4a6b1d787bbc\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567jsr2c"
Jul 01 22:04:03.612195 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:03.612175    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/5001e767-8581-44db-b2b9-4a6b1d787bbc-cachi2-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-567jsr2c\" (UID: \"5001e767-8581-44db-b2b9-4a6b1d787bbc\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567jsr2c"
Jul 01 22:04:03.612263 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:03.612196    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5001e767-8581-44db-b2b9-4a6b1d787bbc-tokenizer-uds\") pod \"scheduler-custom-template-test-kserve-router-scheduler-567jsr2c\" (UID: \"5001e767-8581-44db-b2b9-4a6b1d787bbc\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567jsr2c"
Jul 01 22:04:03.612362 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:03.612343    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/5001e767-8581-44db-b2b9-4a6b1d787bbc-tokenizer-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-567jsr2c\" (UID: \"5001e767-8581-44db-b2b9-4a6b1d787bbc\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567jsr2c"
Jul 01 22:04:03.614530 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:03.614512    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5001e767-8581-44db-b2b9-4a6b1d787bbc-tls-certs\") pod \"scheduler-custom-template-test-kserve-router-scheduler-567jsr2c\" (UID: \"5001e767-8581-44db-b2b9-4a6b1d787bbc\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567jsr2c"
Jul 01 22:04:03.620328 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:03.620307    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ztpc\" (UniqueName: \"kubernetes.io/projected/5001e767-8581-44db-b2b9-4a6b1d787bbc-kube-api-access-6ztpc\") pod \"scheduler-custom-template-test-kserve-router-scheduler-567jsr2c\" (UID: \"5001e767-8581-44db-b2b9-4a6b1d787bbc\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567jsr2c"
Jul 01 22:04:03.727259 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:03.727211    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567jsr2c"
Jul 01 22:04:03.851105 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:03.851083    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567jsr2c"]
Jul 01 22:04:03.853156 ip-10-0-140-104 kubenswrapper[2571]: W0701 22:04:03.853133    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5001e767_8581_44db_b2b9_4a6b1d787bbc.slice/crio-330bed9edc4c9ba5869e3999b1ffbfa02d9695fe4e7c26069f77364146c6af3a WatchSource:0}: Error finding container 330bed9edc4c9ba5869e3999b1ffbfa02d9695fe4e7c26069f77364146c6af3a: Status 404 returned error can't find the container with id 330bed9edc4c9ba5869e3999b1ffbfa02d9695fe4e7c26069f77364146c6af3a
Jul 01 22:04:03.948573 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:03.948546    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567jsr2c" event={"ID":"5001e767-8581-44db-b2b9-4a6b1d787bbc","Type":"ContainerStarted","Data":"d54c45cee0eb09272be08d442a780bc1c4f605c8398b053da8d96c476adc85f0"}
Jul 01 22:04:03.948732 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:03.948588    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567jsr2c" event={"ID":"5001e767-8581-44db-b2b9-4a6b1d787bbc","Type":"ContainerStarted","Data":"330bed9edc4c9ba5869e3999b1ffbfa02d9695fe4e7c26069f77364146c6af3a"}
Jul 01 22:04:03.948732 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:03.948720    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567jsr2c"
Jul 01 22:04:03.965762 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:03.965718    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567jsr2c" podStartSLOduration=0.965705369 podStartE2EDuration="965.705369ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:04:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:04:03.96496713 +0000 UTC m=+4662.477442319" watchObservedRunningTime="2026-07-01 22:04:03.965705369 +0000 UTC m=+4662.478180557"
Jul 01 22:04:34.965123 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:34.965095    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567jsr2c"
Jul 01 22:04:36.413279 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:36.413245    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567jsr2c"]
Jul 01 22:04:36.422544 ip-10-0-140-104 kubenswrapper[2571]: E0701 22:04:36.422497    2571 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 01 22:04:36.422544 ip-10-0-140-104 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 01 22:04:36.422544 ip-10-0-140-104 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 01 22:04:36.422544 ip-10-0-140-104 kubenswrapper[2571]:  > containerID="d54c45cee0eb09272be08d442a780bc1c4f605c8398b053da8d96c476adc85f0" cmd=["/bin/sleep","15"]
Jul 01 22:04:36.422860 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:36.422564    2571 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567jsr2c" message="" err=<
Jul 01 22:04:36.422860 ip-10-0-140-104 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 01 22:04:36.422860 ip-10-0-140-104 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 01 22:04:36.422860 ip-10-0-140-104 kubenswrapper[2571]:  >
Jul 01 22:04:36.422860 ip-10-0-140-104 kubenswrapper[2571]: E0701 22:04:36.422578    2571 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 01 22:04:36.422860 ip-10-0-140-104 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 01 22:04:36.422860 ip-10-0-140-104 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 01 22:04:36.422860 ip-10-0-140-104 kubenswrapper[2571]:  > pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567jsr2c" podUID="5001e767-8581-44db-b2b9-4a6b1d787bbc" containerName="main" containerID="cri-o://d54c45cee0eb09272be08d442a780bc1c4f605c8398b053da8d96c476adc85f0"
Jul 01 22:04:36.422860 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:36.422769    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567jsr2c" podUID="5001e767-8581-44db-b2b9-4a6b1d787bbc" containerName="main" containerID="cri-o://d54c45cee0eb09272be08d442a780bc1c4f605c8398b053da8d96c476adc85f0" gracePeriod=60
Jul 01 22:04:36.671096 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:36.671033    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567jsr2c"
Jul 01 22:04:36.783171 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:36.783124    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5001e767-8581-44db-b2b9-4a6b1d787bbc-tokenizer-uds\") pod \"5001e767-8581-44db-b2b9-4a6b1d787bbc\" (UID: \"5001e767-8581-44db-b2b9-4a6b1d787bbc\") "
Jul 01 22:04:36.783373 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:36.783198    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5001e767-8581-44db-b2b9-4a6b1d787bbc-tls-certs\") pod \"5001e767-8581-44db-b2b9-4a6b1d787bbc\" (UID: \"5001e767-8581-44db-b2b9-4a6b1d787bbc\") "
Jul 01 22:04:36.783373 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:36.783240    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/5001e767-8581-44db-b2b9-4a6b1d787bbc-tokenizer-cache\") pod \"5001e767-8581-44db-b2b9-4a6b1d787bbc\" (UID: \"5001e767-8581-44db-b2b9-4a6b1d787bbc\") "
Jul 01 22:04:36.783373 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:36.783298    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6ztpc\" (UniqueName: \"kubernetes.io/projected/5001e767-8581-44db-b2b9-4a6b1d787bbc-kube-api-access-6ztpc\") pod \"5001e767-8581-44db-b2b9-4a6b1d787bbc\" (UID: \"5001e767-8581-44db-b2b9-4a6b1d787bbc\") "
Jul 01 22:04:36.783373 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:36.783332    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/5001e767-8581-44db-b2b9-4a6b1d787bbc-cachi2-cache\") pod \"5001e767-8581-44db-b2b9-4a6b1d787bbc\" (UID: \"5001e767-8581-44db-b2b9-4a6b1d787bbc\") "
Jul 01 22:04:36.783596 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:36.783460    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5001e767-8581-44db-b2b9-4a6b1d787bbc-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "5001e767-8581-44db-b2b9-4a6b1d787bbc" (UID: "5001e767-8581-44db-b2b9-4a6b1d787bbc"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 22:04:36.783672 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:36.783658    2571 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5001e767-8581-44db-b2b9-4a6b1d787bbc-tokenizer-uds\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 22:04:36.783731 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:36.783703    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5001e767-8581-44db-b2b9-4a6b1d787bbc-cachi2-cache" (OuterVolumeSpecName: "cachi2-cache") pod "5001e767-8581-44db-b2b9-4a6b1d787bbc" (UID: "5001e767-8581-44db-b2b9-4a6b1d787bbc"). InnerVolumeSpecName "cachi2-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 22:04:36.783845 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:36.783807    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5001e767-8581-44db-b2b9-4a6b1d787bbc-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "5001e767-8581-44db-b2b9-4a6b1d787bbc" (UID: "5001e767-8581-44db-b2b9-4a6b1d787bbc"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 22:04:36.785616 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:36.785580    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5001e767-8581-44db-b2b9-4a6b1d787bbc-kube-api-access-6ztpc" (OuterVolumeSpecName: "kube-api-access-6ztpc") pod "5001e767-8581-44db-b2b9-4a6b1d787bbc" (UID: "5001e767-8581-44db-b2b9-4a6b1d787bbc"). InnerVolumeSpecName "kube-api-access-6ztpc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 22:04:36.785732 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:36.785667    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5001e767-8581-44db-b2b9-4a6b1d787bbc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5001e767-8581-44db-b2b9-4a6b1d787bbc" (UID: "5001e767-8581-44db-b2b9-4a6b1d787bbc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 22:04:36.884467 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:36.884441    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5001e767-8581-44db-b2b9-4a6b1d787bbc-tls-certs\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 22:04:36.884467 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:36.884469    2571 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/5001e767-8581-44db-b2b9-4a6b1d787bbc-tokenizer-cache\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 22:04:36.884702 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:36.884484    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ztpc\" (UniqueName: \"kubernetes.io/projected/5001e767-8581-44db-b2b9-4a6b1d787bbc-kube-api-access-6ztpc\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 22:04:36.884702 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:36.884499    2571 reconciler_common.go:299] "Volume detached for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/5001e767-8581-44db-b2b9-4a6b1d787bbc-cachi2-cache\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 22:04:37.109879 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:37.109797    2571 generic.go:356] "Generic (PLEG): container finished" podID="5001e767-8581-44db-b2b9-4a6b1d787bbc" containerID="d54c45cee0eb09272be08d442a780bc1c4f605c8398b053da8d96c476adc85f0" exitCode=0
Jul 01 22:04:37.109879 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:37.109832    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567jsr2c" event={"ID":"5001e767-8581-44db-b2b9-4a6b1d787bbc","Type":"ContainerDied","Data":"d54c45cee0eb09272be08d442a780bc1c4f605c8398b053da8d96c476adc85f0"}
Jul 01 22:04:37.110087 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:37.109848    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567jsr2c"
Jul 01 22:04:37.110087 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:37.109897    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567jsr2c" event={"ID":"5001e767-8581-44db-b2b9-4a6b1d787bbc","Type":"ContainerDied","Data":"330bed9edc4c9ba5869e3999b1ffbfa02d9695fe4e7c26069f77364146c6af3a"}
Jul 01 22:04:37.110087 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:37.109923    2571 scope.go:117] "RemoveContainer" containerID="d54c45cee0eb09272be08d442a780bc1c4f605c8398b053da8d96c476adc85f0"
Jul 01 22:04:37.119012 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:37.118998    2571 scope.go:117] "RemoveContainer" containerID="d54c45cee0eb09272be08d442a780bc1c4f605c8398b053da8d96c476adc85f0"
Jul 01 22:04:37.119266 ip-10-0-140-104 kubenswrapper[2571]: E0701 22:04:37.119249    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d54c45cee0eb09272be08d442a780bc1c4f605c8398b053da8d96c476adc85f0\": container with ID starting with d54c45cee0eb09272be08d442a780bc1c4f605c8398b053da8d96c476adc85f0 not found: ID does not exist" containerID="d54c45cee0eb09272be08d442a780bc1c4f605c8398b053da8d96c476adc85f0"
Jul 01 22:04:37.119331 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:37.119274    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d54c45cee0eb09272be08d442a780bc1c4f605c8398b053da8d96c476adc85f0"} err="failed to get container status \"d54c45cee0eb09272be08d442a780bc1c4f605c8398b053da8d96c476adc85f0\": rpc error: code = NotFound desc = could not find container \"d54c45cee0eb09272be08d442a780bc1c4f605c8398b053da8d96c476adc85f0\": container with ID starting with d54c45cee0eb09272be08d442a780bc1c4f605c8398b053da8d96c476adc85f0 not found: ID does not exist"
Jul 01 22:04:37.130546 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:37.130521    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567jsr2c"]
Jul 01 22:04:37.135181 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:37.135160    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567jsr2c"]
Jul 01 22:04:37.838745 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:04:37.838715    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5001e767-8581-44db-b2b9-4a6b1d787bbc" path="/var/lib/kubelet/pods/5001e767-8581-44db-b2b9-4a6b1d787bbc/volumes"
Jul 01 22:25:23.848916 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:25:23.848888    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk_44254112-edab-4268-8b4b-9c43cb7e371b/main/0.log"
Jul 01 22:25:44.877563 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:25:44.877532    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched94m92"]
Jul 01 22:25:44.878001 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:25:44.877872    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5001e767-8581-44db-b2b9-4a6b1d787bbc" containerName="main"
Jul 01 22:25:44.878001 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:25:44.877884    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5001e767-8581-44db-b2b9-4a6b1d787bbc" containerName="main"
Jul 01 22:25:44.878001 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:25:44.877939    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="5001e767-8581-44db-b2b9-4a6b1d787bbc" containerName="main"
Jul 01 22:25:44.880454 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:25:44.880436    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched94m92"
Jul 01 22:25:44.885011 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:25:44.884990    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-v06-threshold-migration-test-epp-sa-dockercfg-glgz8\""
Jul 01 22:25:44.885110 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:25:44.885037    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-v06-threshold-migration-test-kserve-self-signed-certs\""
Jul 01 22:25:44.892933 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:25:44.892915    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched94m92"]
Jul 01 22:25:44.997090 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:25:44.997066    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-router-sched94m92\" (UID: \"cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched94m92"
Jul 01 22:25:44.997213 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:25:44.997109    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srvcs\" (UniqueName: \"kubernetes.io/projected/cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57-kube-api-access-srvcs\") pod \"scheduler-v06-threshold-migration-test-kserve-router-sched94m92\" (UID: \"cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched94m92"
Jul 01 22:25:44.997279 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:25:44.997244    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57-tokenizer-uds\") pod \"scheduler-v06-threshold-migration-test-kserve-router-sched94m92\" (UID: \"cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched94m92"
Jul 01 22:25:45.098094 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:25:45.098072    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57-tokenizer-uds\") pod \"scheduler-v06-threshold-migration-test-kserve-router-sched94m92\" (UID: \"cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched94m92"
Jul 01 22:25:45.098213 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:25:45.098129    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-router-sched94m92\" (UID: \"cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched94m92"
Jul 01 22:25:45.098213 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:25:45.098168    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srvcs\" (UniqueName: \"kubernetes.io/projected/cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57-kube-api-access-srvcs\") pod \"scheduler-v06-threshold-migration-test-kserve-router-sched94m92\" (UID: \"cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched94m92"
Jul 01 22:25:45.098428 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:25:45.098409    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57-tokenizer-uds\") pod \"scheduler-v06-threshold-migration-test-kserve-router-sched94m92\" (UID: \"cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched94m92"
Jul 01 22:25:45.100515 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:25:45.100493    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-router-sched94m92\" (UID: \"cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched94m92"
Jul 01 22:25:45.107098 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:25:45.107079    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-srvcs\" (UniqueName: \"kubernetes.io/projected/cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57-kube-api-access-srvcs\") pod \"scheduler-v06-threshold-migration-test-kserve-router-sched94m92\" (UID: \"cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched94m92"
Jul 01 22:25:45.191478 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:25:45.191460    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched94m92"
Jul 01 22:25:45.326728 ip-10-0-140-104 kubenswrapper[2571]: W0701 22:25:45.326696    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc3f09cb_fa94_4a4e_9f8f_f8bdc33e4d57.slice/crio-c270bdb67bada3dcf8a43e1832330b4b2f4d64441ef97a23d31f5d0c0f9f8af6 WatchSource:0}: Error finding container c270bdb67bada3dcf8a43e1832330b4b2f4d64441ef97a23d31f5d0c0f9f8af6: Status 404 returned error can't find the container with id c270bdb67bada3dcf8a43e1832330b4b2f4d64441ef97a23d31f5d0c0f9f8af6
Jul 01 22:25:45.329194 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:25:45.329174    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched94m92"]
Jul 01 22:25:46.249685 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:25:46.249654    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched94m92" event={"ID":"cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57","Type":"ContainerStarted","Data":"b6ff2772e226a839700a612d26d94c110f52850c367d5ebdb0fc431cab218d22"}
Jul 01 22:25:46.250061 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:25:46.249695    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched94m92" event={"ID":"cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57","Type":"ContainerStarted","Data":"c270bdb67bada3dcf8a43e1832330b4b2f4d64441ef97a23d31f5d0c0f9f8af6"}
Jul 01 22:25:46.250061 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:25:46.249753    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched94m92"
Jul 01 22:25:46.274271 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:25:46.274217    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched94m92" podStartSLOduration=2.274202271 podStartE2EDuration="2.274202271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:25:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:25:46.270717333 +0000 UTC m=+5964.783192521" watchObservedRunningTime="2026-07-01 22:25:46.274202271 +0000 UTC m=+5964.786677459"
Jul 01 22:26:17.256925 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:26:17.256901    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched94m92"
Jul 01 22:32:18.580557 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:32:18.580527    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched94m92"]
Jul 01 22:32:18.589315 ip-10-0-140-104 kubenswrapper[2571]: E0701 22:32:18.589285    2571 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 01 22:32:18.589315 ip-10-0-140-104 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 01 22:32:18.589315 ip-10-0-140-104 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 01 22:32:18.589315 ip-10-0-140-104 kubenswrapper[2571]:  > containerID="b6ff2772e226a839700a612d26d94c110f52850c367d5ebdb0fc431cab218d22" cmd=["/bin/sleep","15"]
Jul 01 22:32:18.589576 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:32:18.589328    2571 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched94m92" message="" err=<
Jul 01 22:32:18.589576 ip-10-0-140-104 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 01 22:32:18.589576 ip-10-0-140-104 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 01 22:32:18.589576 ip-10-0-140-104 kubenswrapper[2571]:  >
Jul 01 22:32:18.589576 ip-10-0-140-104 kubenswrapper[2571]: E0701 22:32:18.589345    2571 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 01 22:32:18.589576 ip-10-0-140-104 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 01 22:32:18.589576 ip-10-0-140-104 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 01 22:32:18.589576 ip-10-0-140-104 kubenswrapper[2571]:  > pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched94m92" podUID="cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57" containerName="main" containerID="cri-o://b6ff2772e226a839700a612d26d94c110f52850c367d5ebdb0fc431cab218d22"
Jul 01 22:32:18.589576 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:32:18.589403    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched94m92" podUID="cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57" containerName="main" containerID="cri-o://b6ff2772e226a839700a612d26d94c110f52850c367d5ebdb0fc431cab218d22" gracePeriod=60
Jul 01 22:32:18.825247 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:32:18.825225    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched94m92"
Jul 01 22:32:18.864743 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:32:18.864678    2571 generic.go:356] "Generic (PLEG): container finished" podID="cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57" containerID="b6ff2772e226a839700a612d26d94c110f52850c367d5ebdb0fc431cab218d22" exitCode=0
Jul 01 22:32:18.864743 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:32:18.864719    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched94m92"
Jul 01 22:32:18.864870 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:32:18.864715    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched94m92" event={"ID":"cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57","Type":"ContainerDied","Data":"b6ff2772e226a839700a612d26d94c110f52850c367d5ebdb0fc431cab218d22"}
Jul 01 22:32:18.864870 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:32:18.864834    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched94m92" event={"ID":"cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57","Type":"ContainerDied","Data":"c270bdb67bada3dcf8a43e1832330b4b2f4d64441ef97a23d31f5d0c0f9f8af6"}
Jul 01 22:32:18.864870 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:32:18.864859    2571 scope.go:117] "RemoveContainer" containerID="b6ff2772e226a839700a612d26d94c110f52850c367d5ebdb0fc431cab218d22"
Jul 01 22:32:18.872754 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:32:18.872739    2571 scope.go:117] "RemoveContainer" containerID="b6ff2772e226a839700a612d26d94c110f52850c367d5ebdb0fc431cab218d22"
Jul 01 22:32:18.872991 ip-10-0-140-104 kubenswrapper[2571]: E0701 22:32:18.872975    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6ff2772e226a839700a612d26d94c110f52850c367d5ebdb0fc431cab218d22\": container with ID starting with b6ff2772e226a839700a612d26d94c110f52850c367d5ebdb0fc431cab218d22 not found: ID does not exist" containerID="b6ff2772e226a839700a612d26d94c110f52850c367d5ebdb0fc431cab218d22"
Jul 01 22:32:18.873044 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:32:18.873002    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6ff2772e226a839700a612d26d94c110f52850c367d5ebdb0fc431cab218d22"} err="failed to get container status \"b6ff2772e226a839700a612d26d94c110f52850c367d5ebdb0fc431cab218d22\": rpc error: code = NotFound desc = could not find container \"b6ff2772e226a839700a612d26d94c110f52850c367d5ebdb0fc431cab218d22\": container with ID starting with b6ff2772e226a839700a612d26d94c110f52850c367d5ebdb0fc431cab218d22 not found: ID does not exist"
Jul 01 22:32:18.924813 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:32:18.924794    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57-tls-certs\") pod \"cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57\" (UID: \"cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57\") "
Jul 01 22:32:18.924911 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:32:18.924823    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57-tokenizer-uds\") pod \"cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57\" (UID: \"cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57\") "
Jul 01 22:32:18.924911 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:32:18.924871    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-srvcs\" (UniqueName: \"kubernetes.io/projected/cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57-kube-api-access-srvcs\") pod \"cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57\" (UID: \"cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57\") "
Jul 01 22:32:18.925104 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:32:18.925086    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57" (UID: "cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 22:32:18.926873 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:32:18.926850    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57" (UID: "cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 22:32:18.926873 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:32:18.926854    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57-kube-api-access-srvcs" (OuterVolumeSpecName: "kube-api-access-srvcs") pod "cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57" (UID: "cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57"). InnerVolumeSpecName "kube-api-access-srvcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 22:32:19.025777 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:32:19.025747    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57-tls-certs\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 22:32:19.025777 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:32:19.025771    2571 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57-tokenizer-uds\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 22:32:19.025777 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:32:19.025779    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-srvcs\" (UniqueName: \"kubernetes.io/projected/cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57-kube-api-access-srvcs\") on node \"ip-10-0-140-104.ec2.internal\" DevicePath \"\""
Jul 01 22:32:19.188564 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:32:19.188539    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched94m92"]
Jul 01 22:32:19.195612 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:32:19.195594    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched94m92"]
Jul 01 22:32:19.838161 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:32:19.838129    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57" path="/var/lib/kubelet/pods/cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57/volumes"
Jul 01 22:32:55.620323 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:32:55.620235    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk_44254112-edab-4268-8b4b-9c43cb7e371b/main/0.log"
Jul 01 22:32:57.195945 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:32:57.195915    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk_44254112-edab-4268-8b4b-9c43cb7e371b/main/0.log"
Jul 01 22:32:58.748090 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:32:58.748066    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk_44254112-edab-4268-8b4b-9c43cb7e371b/main/0.log"
Jul 01 22:33:00.255042 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:00.255018    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk_44254112-edab-4268-8b4b-9c43cb7e371b/main/0.log"
Jul 01 22:33:01.761267 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:01.761240    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk_44254112-edab-4268-8b4b-9c43cb7e371b/main/0.log"
Jul 01 22:33:03.334805 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:03.334768    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk_44254112-edab-4268-8b4b-9c43cb7e371b/main/0.log"
Jul 01 22:33:04.820824 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:04.820797    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk_44254112-edab-4268-8b4b-9c43cb7e371b/main/0.log"
Jul 01 22:33:06.299583 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:06.299552    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk_44254112-edab-4268-8b4b-9c43cb7e371b/main/0.log"
Jul 01 22:33:07.917257 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:07.917227    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk_44254112-edab-4268-8b4b-9c43cb7e371b/main/0.log"
Jul 01 22:33:09.499643 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:09.499605    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk_44254112-edab-4268-8b4b-9c43cb7e371b/main/0.log"
Jul 01 22:33:10.991293 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:10.991262    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk_44254112-edab-4268-8b4b-9c43cb7e371b/main/0.log"
Jul 01 22:33:12.580470 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:12.580431    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk_44254112-edab-4268-8b4b-9c43cb7e371b/main/0.log"
Jul 01 22:33:14.078409 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:14.078385    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk_44254112-edab-4268-8b4b-9c43cb7e371b/main/0.log"
Jul 01 22:33:15.578003 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:15.577974    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk_44254112-edab-4268-8b4b-9c43cb7e371b/main/0.log"
Jul 01 22:33:17.057743 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:17.057706    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk_44254112-edab-4268-8b4b-9c43cb7e371b/main/0.log"
Jul 01 22:33:18.529300 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:18.529268    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hhhsk_44254112-edab-4268-8b4b-9c43cb7e371b/main/0.log"
Jul 01 22:33:20.719485 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:20.719448    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-655f7c58b4-57g8x_f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13/istio-proxy/5.log"
Jul 01 22:33:20.734133 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:20.734105    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-655f7c58b4-57g8x_f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13/istio-proxy/4.log"
Jul 01 22:33:22.393798 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:22.393760    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-655f7c58b4-57g8x_f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13/istio-proxy/5.log"
Jul 01 22:33:22.409010 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:22.408982    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-655f7c58b4-57g8x_f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13/istio-proxy/4.log"
Jul 01 22:33:24.051706 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:24.051670    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-655f7c58b4-57g8x_f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13/istio-proxy/5.log"
Jul 01 22:33:24.065716 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:24.065692    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-655f7c58b4-57g8x_f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13/istio-proxy/4.log"
Jul 01 22:33:25.736291 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:25.736262    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-655f7c58b4-57g8x_f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13/istio-proxy/5.log"
Jul 01 22:33:25.750746 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:25.750712    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-655f7c58b4-57g8x_f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13/istio-proxy/4.log"
Jul 01 22:33:26.724736 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:26.724700    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-c59jh_a2454155-5137-4ea9-9b3a-6656ed62603c/manager/0.log"
Jul 01 22:33:26.739343 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:26.739319    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-pgv8h_b7720a2b-19f8-4d1a-b0f6-a481c4dc0f62/kuadrant-console-plugin/0.log"
Jul 01 22:33:26.972529 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:26.972503    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-8mmn4_f0cb5c83-be3d-4f3d-8bd4-ea2170710461/limitador/0.log"
Jul 01 22:33:27.996914 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:27.996889    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-c59jh_a2454155-5137-4ea9-9b3a-6656ed62603c/manager/0.log"
Jul 01 22:33:28.011762 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:28.011737    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-pgv8h_b7720a2b-19f8-4d1a-b0f6-a481c4dc0f62/kuadrant-console-plugin/0.log"
Jul 01 22:33:28.251143 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:28.251059    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-8mmn4_f0cb5c83-be3d-4f3d-8bd4-ea2170710461/limitador/0.log"
Jul 01 22:33:29.337079 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:29.337050    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-c59jh_a2454155-5137-4ea9-9b3a-6656ed62603c/manager/0.log"
Jul 01 22:33:29.352468 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:29.352446    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-pgv8h_b7720a2b-19f8-4d1a-b0f6-a481c4dc0f62/kuadrant-console-plugin/0.log"
Jul 01 22:33:29.607376 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:29.607299    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-8mmn4_f0cb5c83-be3d-4f3d-8bd4-ea2170710461/limitador/0.log"
Jul 01 22:33:30.624725 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:30.624692    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-c59jh_a2454155-5137-4ea9-9b3a-6656ed62603c/manager/0.log"
Jul 01 22:33:30.640414 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:30.640382    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-pgv8h_b7720a2b-19f8-4d1a-b0f6-a481c4dc0f62/kuadrant-console-plugin/0.log"
Jul 01 22:33:30.877802 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:30.877708    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-8mmn4_f0cb5c83-be3d-4f3d-8bd4-ea2170710461/limitador/0.log"
Jul 01 22:33:31.891890 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:31.891862    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-c59jh_a2454155-5137-4ea9-9b3a-6656ed62603c/manager/0.log"
Jul 01 22:33:31.908754 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:31.908730    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-pgv8h_b7720a2b-19f8-4d1a-b0f6-a481c4dc0f62/kuadrant-console-plugin/0.log"
Jul 01 22:33:32.177330 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:32.177296    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-8mmn4_f0cb5c83-be3d-4f3d-8bd4-ea2170710461/limitador/0.log"
Jul 01 22:33:33.209565 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:33.209501    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-c59jh_a2454155-5137-4ea9-9b3a-6656ed62603c/manager/0.log"
Jul 01 22:33:33.223459 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:33.223437    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-pgv8h_b7720a2b-19f8-4d1a-b0f6-a481c4dc0f62/kuadrant-console-plugin/0.log"
Jul 01 22:33:33.461836 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:33.461752    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-8mmn4_f0cb5c83-be3d-4f3d-8bd4-ea2170710461/limitador/0.log"
Jul 01 22:33:34.515610 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:34.515560    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-c59jh_a2454155-5137-4ea9-9b3a-6656ed62603c/manager/0.log"
Jul 01 22:33:34.530517 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:34.530492    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-pgv8h_b7720a2b-19f8-4d1a-b0f6-a481c4dc0f62/kuadrant-console-plugin/0.log"
Jul 01 22:33:34.761351 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:34.761321    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-8mmn4_f0cb5c83-be3d-4f3d-8bd4-ea2170710461/limitador/0.log"
Jul 01 22:33:35.797788 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:35.797746    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-c59jh_a2454155-5137-4ea9-9b3a-6656ed62603c/manager/0.log"
Jul 01 22:33:35.812569 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:35.812545    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-pgv8h_b7720a2b-19f8-4d1a-b0f6-a481c4dc0f62/kuadrant-console-plugin/0.log"
Jul 01 22:33:36.046497 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:36.046465    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-8mmn4_f0cb5c83-be3d-4f3d-8bd4-ea2170710461/limitador/0.log"
Jul 01 22:33:37.155551 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:37.155521    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-c59jh_a2454155-5137-4ea9-9b3a-6656ed62603c/manager/0.log"
Jul 01 22:33:37.168605 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:37.168583    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-pgv8h_b7720a2b-19f8-4d1a-b0f6-a481c4dc0f62/kuadrant-console-plugin/0.log"
Jul 01 22:33:37.397272 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:37.397244    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-8mmn4_f0cb5c83-be3d-4f3d-8bd4-ea2170710461/limitador/0.log"
Jul 01 22:33:38.414884 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:38.414858    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-c59jh_a2454155-5137-4ea9-9b3a-6656ed62603c/manager/0.log"
Jul 01 22:33:38.428880 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:38.428859    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-pgv8h_b7720a2b-19f8-4d1a-b0f6-a481c4dc0f62/kuadrant-console-plugin/0.log"
Jul 01 22:33:38.670134 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:38.670048    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-8mmn4_f0cb5c83-be3d-4f3d-8bd4-ea2170710461/limitador/0.log"
Jul 01 22:33:39.697219 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:39.697181    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-c59jh_a2454155-5137-4ea9-9b3a-6656ed62603c/manager/0.log"
Jul 01 22:33:39.711941 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:39.711918    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-pgv8h_b7720a2b-19f8-4d1a-b0f6-a481c4dc0f62/kuadrant-console-plugin/0.log"
Jul 01 22:33:39.948969 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:39.948891    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-8mmn4_f0cb5c83-be3d-4f3d-8bd4-ea2170710461/limitador/0.log"
Jul 01 22:33:45.355235 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:45.355188    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-t6lw6_1dfa26e7-23bf-4e56-808d-e59cee52aee7/global-pull-secret-syncer/0.log"
Jul 01 22:33:45.553699 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:45.553669    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-njzrp_6cb9481e-9331-4e53-abc4-3336c04034c2/konnectivity-agent/0.log"
Jul 01 22:33:45.643186 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:45.643128    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-104.ec2.internal_289aec27ab6274a41b4614fd9786336a/haproxy/0.log"
Jul 01 22:33:50.451187 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:50.451150    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-c59jh_a2454155-5137-4ea9-9b3a-6656ed62603c/manager/0.log"
Jul 01 22:33:50.493462 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:50.493429    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-pgv8h_b7720a2b-19f8-4d1a-b0f6-a481c4dc0f62/kuadrant-console-plugin/0.log"
Jul 01 22:33:50.780831 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:50.780730    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-8mmn4_f0cb5c83-be3d-4f3d-8bd4-ea2170710461/limitador/0.log"
Jul 01 22:33:52.115716 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:52.115690    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_eb5351f3-0f90-4da6-846a-11ff1ba5f6bb/alertmanager/0.log"
Jul 01 22:33:52.146011 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:52.145987    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_eb5351f3-0f90-4da6-846a-11ff1ba5f6bb/config-reloader/0.log"
Jul 01 22:33:52.169884 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:52.169862    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_eb5351f3-0f90-4da6-846a-11ff1ba5f6bb/kube-rbac-proxy-web/0.log"
Jul 01 22:33:52.195027 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:52.195003    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_eb5351f3-0f90-4da6-846a-11ff1ba5f6bb/kube-rbac-proxy/0.log"
Jul 01 22:33:52.219023 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:52.218996    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_eb5351f3-0f90-4da6-846a-11ff1ba5f6bb/kube-rbac-proxy-metric/0.log"
Jul 01 22:33:52.250573 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:52.250546    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_eb5351f3-0f90-4da6-846a-11ff1ba5f6bb/prom-label-proxy/0.log"
Jul 01 22:33:52.278524 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:52.278501    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_eb5351f3-0f90-4da6-846a-11ff1ba5f6bb/init-config-reloader/0.log"
Jul 01 22:33:52.629880 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:52.629855    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f7ddq_639b6494-9238-4659-a094-d1fe4b13021f/node-exporter/0.log"
Jul 01 22:33:52.653586 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:52.653558    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f7ddq_639b6494-9238-4659-a094-d1fe4b13021f/kube-rbac-proxy/0.log"
Jul 01 22:33:52.696155 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:52.696124    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f7ddq_639b6494-9238-4659-a094-d1fe4b13021f/init-textfile/0.log"
Jul 01 22:33:52.925881 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:52.925845    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8a1b7312-62c8-47fe-a17d-2384295f4db6/prometheus/0.log"
Jul 01 22:33:52.943695 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:52.943622    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8a1b7312-62c8-47fe-a17d-2384295f4db6/config-reloader/0.log"
Jul 01 22:33:52.966476 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:52.966446    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8a1b7312-62c8-47fe-a17d-2384295f4db6/thanos-sidecar/0.log"
Jul 01 22:33:52.990545 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:52.990517    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8a1b7312-62c8-47fe-a17d-2384295f4db6/kube-rbac-proxy-web/0.log"
Jul 01 22:33:53.015133 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:53.015111    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8a1b7312-62c8-47fe-a17d-2384295f4db6/kube-rbac-proxy/0.log"
Jul 01 22:33:53.043677 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:53.043622    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8a1b7312-62c8-47fe-a17d-2384295f4db6/kube-rbac-proxy-thanos/0.log"
Jul 01 22:33:53.067587 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:53.067560    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8a1b7312-62c8-47fe-a17d-2384295f4db6/init-config-reloader/0.log"
Jul 01 22:33:53.188375 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:53.188351    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-789f894974-2v8t6_a7e8a91f-7786-4107-8c33-8592e49a15c8/telemeter-client/0.log"
Jul 01 22:33:53.213460 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:53.213438    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-789f894974-2v8t6_a7e8a91f-7786-4107-8c33-8592e49a15c8/reload/0.log"
Jul 01 22:33:53.237844 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:53.237822    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-789f894974-2v8t6_a7e8a91f-7786-4107-8c33-8592e49a15c8/kube-rbac-proxy/0.log"
Jul 01 22:33:54.845676 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:54.845640    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-z7v9q/perf-node-gather-daemonset-76fpm"]
Jul 01 22:33:54.846083 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:54.845940    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57" containerName="main"
Jul 01 22:33:54.846083 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:54.845951    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57" containerName="main"
Jul 01 22:33:54.846083 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:54.846021    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc3f09cb-fa94-4a4e-9f8f-f8bdc33e4d57" containerName="main"
Jul 01 22:33:54.849039 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:54.849019    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z7v9q/perf-node-gather-daemonset-76fpm"
Jul 01 22:33:54.851855 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:54.851833    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-z7v9q\"/\"kube-root-ca.crt\""
Jul 01 22:33:54.852050 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:54.852032    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-z7v9q\"/\"openshift-service-ca.crt\""
Jul 01 22:33:54.853100 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:54.853084    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-z7v9q\"/\"default-dockercfg-pqvjh\""
Jul 01 22:33:54.859472 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:54.859445    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z7v9q/perf-node-gather-daemonset-76fpm"]
Jul 01 22:33:54.899276 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:54.899240    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fd75453f-1171-4db5-ab42-ddfa52be4bcc-proc\") pod \"perf-node-gather-daemonset-76fpm\" (UID: \"fd75453f-1171-4db5-ab42-ddfa52be4bcc\") " pod="openshift-must-gather-z7v9q/perf-node-gather-daemonset-76fpm"
Jul 01 22:33:54.899441 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:54.899316    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6q4wg\" (UniqueName: \"kubernetes.io/projected/fd75453f-1171-4db5-ab42-ddfa52be4bcc-kube-api-access-6q4wg\") pod \"perf-node-gather-daemonset-76fpm\" (UID: \"fd75453f-1171-4db5-ab42-ddfa52be4bcc\") " pod="openshift-must-gather-z7v9q/perf-node-gather-daemonset-76fpm"
Jul 01 22:33:54.899441 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:54.899344    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fd75453f-1171-4db5-ab42-ddfa52be4bcc-sys\") pod \"perf-node-gather-daemonset-76fpm\" (UID: \"fd75453f-1171-4db5-ab42-ddfa52be4bcc\") " pod="openshift-must-gather-z7v9q/perf-node-gather-daemonset-76fpm"
Jul 01 22:33:54.899441 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:54.899370    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fd75453f-1171-4db5-ab42-ddfa52be4bcc-podres\") pod \"perf-node-gather-daemonset-76fpm\" (UID: \"fd75453f-1171-4db5-ab42-ddfa52be4bcc\") " pod="openshift-must-gather-z7v9q/perf-node-gather-daemonset-76fpm"
Jul 01 22:33:54.899605 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:54.899464    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fd75453f-1171-4db5-ab42-ddfa52be4bcc-lib-modules\") pod \"perf-node-gather-daemonset-76fpm\" (UID: \"fd75453f-1171-4db5-ab42-ddfa52be4bcc\") " pod="openshift-must-gather-z7v9q/perf-node-gather-daemonset-76fpm"
Jul 01 22:33:54.999991 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:54.999948    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fd75453f-1171-4db5-ab42-ddfa52be4bcc-proc\") pod \"perf-node-gather-daemonset-76fpm\" (UID: \"fd75453f-1171-4db5-ab42-ddfa52be4bcc\") " pod="openshift-must-gather-z7v9q/perf-node-gather-daemonset-76fpm"
Jul 01 22:33:55.000194 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:55.000014    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6q4wg\" (UniqueName: \"kubernetes.io/projected/fd75453f-1171-4db5-ab42-ddfa52be4bcc-kube-api-access-6q4wg\") pod \"perf-node-gather-daemonset-76fpm\" (UID: \"fd75453f-1171-4db5-ab42-ddfa52be4bcc\") " pod="openshift-must-gather-z7v9q/perf-node-gather-daemonset-76fpm"
Jul 01 22:33:55.000194 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:55.000042    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fd75453f-1171-4db5-ab42-ddfa52be4bcc-sys\") pod \"perf-node-gather-daemonset-76fpm\" (UID: \"fd75453f-1171-4db5-ab42-ddfa52be4bcc\") " pod="openshift-must-gather-z7v9q/perf-node-gather-daemonset-76fpm"
Jul 01 22:33:55.000194 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:55.000071    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fd75453f-1171-4db5-ab42-ddfa52be4bcc-podres\") pod \"perf-node-gather-daemonset-76fpm\" (UID: \"fd75453f-1171-4db5-ab42-ddfa52be4bcc\") " pod="openshift-must-gather-z7v9q/perf-node-gather-daemonset-76fpm"
Jul 01 22:33:55.000194 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:55.000088    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fd75453f-1171-4db5-ab42-ddfa52be4bcc-proc\") pod \"perf-node-gather-daemonset-76fpm\" (UID: \"fd75453f-1171-4db5-ab42-ddfa52be4bcc\") " pod="openshift-must-gather-z7v9q/perf-node-gather-daemonset-76fpm"
Jul 01 22:33:55.000194 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:55.000105    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fd75453f-1171-4db5-ab42-ddfa52be4bcc-lib-modules\") pod \"perf-node-gather-daemonset-76fpm\" (UID: \"fd75453f-1171-4db5-ab42-ddfa52be4bcc\") " pod="openshift-must-gather-z7v9q/perf-node-gather-daemonset-76fpm"
Jul 01 22:33:55.000194 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:55.000157    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fd75453f-1171-4db5-ab42-ddfa52be4bcc-sys\") pod \"perf-node-gather-daemonset-76fpm\" (UID: \"fd75453f-1171-4db5-ab42-ddfa52be4bcc\") " pod="openshift-must-gather-z7v9q/perf-node-gather-daemonset-76fpm"
Jul 01 22:33:55.000424 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:55.000212    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fd75453f-1171-4db5-ab42-ddfa52be4bcc-podres\") pod \"perf-node-gather-daemonset-76fpm\" (UID: \"fd75453f-1171-4db5-ab42-ddfa52be4bcc\") " pod="openshift-must-gather-z7v9q/perf-node-gather-daemonset-76fpm"
Jul 01 22:33:55.000424 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:55.000221    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fd75453f-1171-4db5-ab42-ddfa52be4bcc-lib-modules\") pod \"perf-node-gather-daemonset-76fpm\" (UID: \"fd75453f-1171-4db5-ab42-ddfa52be4bcc\") " pod="openshift-must-gather-z7v9q/perf-node-gather-daemonset-76fpm"
Jul 01 22:33:55.009669 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:55.009618    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6q4wg\" (UniqueName: \"kubernetes.io/projected/fd75453f-1171-4db5-ab42-ddfa52be4bcc-kube-api-access-6q4wg\") pod \"perf-node-gather-daemonset-76fpm\" (UID: \"fd75453f-1171-4db5-ab42-ddfa52be4bcc\") " pod="openshift-must-gather-z7v9q/perf-node-gather-daemonset-76fpm"
Jul 01 22:33:55.159676 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:55.159643    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z7v9q/perf-node-gather-daemonset-76fpm"
Jul 01 22:33:55.307883 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:55.307853    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z7v9q/perf-node-gather-daemonset-76fpm"]
Jul 01 22:33:55.311112 ip-10-0-140-104 kubenswrapper[2571]: W0701 22:33:55.311084    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podfd75453f_1171_4db5_ab42_ddfa52be4bcc.slice/crio-5fe8d33353c2c1127895d5e76ff7a05027ac119c7e27407fc6879d3a45a6a743 WatchSource:0}: Error finding container 5fe8d33353c2c1127895d5e76ff7a05027ac119c7e27407fc6879d3a45a6a743: Status 404 returned error can't find the container with id 5fe8d33353c2c1127895d5e76ff7a05027ac119c7e27407fc6879d3a45a6a743
Jul 01 22:33:56.274544 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:56.274512    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z7v9q/perf-node-gather-daemonset-76fpm" event={"ID":"fd75453f-1171-4db5-ab42-ddfa52be4bcc","Type":"ContainerStarted","Data":"0f92b22f30aef5505a3187292f08d81118b4f5e4317167b5ed739a52ac353930"}
Jul 01 22:33:56.274975 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:56.274564    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z7v9q/perf-node-gather-daemonset-76fpm" event={"ID":"fd75453f-1171-4db5-ab42-ddfa52be4bcc","Type":"ContainerStarted","Data":"5fe8d33353c2c1127895d5e76ff7a05027ac119c7e27407fc6879d3a45a6a743"}
Jul 01 22:33:56.274975 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:56.274592    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-z7v9q/perf-node-gather-daemonset-76fpm"
Jul 01 22:33:56.293609 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:56.293507    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-z7v9q/perf-node-gather-daemonset-76fpm" podStartSLOduration=2.293489816 podStartE2EDuration="2.293489816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 22:33:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:33:56.292579934 +0000 UTC m=+6454.805055139" watchObservedRunningTime="2026-07-01 22:33:56.293489816 +0000 UTC m=+6454.805965006"
Jul 01 22:33:57.176226 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:57.176193    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6xhrz_6ac76942-02a5-4206-97e1-3329bcea5c93/dns/0.log"
Jul 01 22:33:57.195165 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:57.195133    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6xhrz_6ac76942-02a5-4206-97e1-3329bcea5c93/kube-rbac-proxy/0.log"
Jul 01 22:33:57.365145 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:57.365115    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ncfhm_eaa37651-7fb2-4625-a15d-e7427a744196/dns-node-resolver/0.log"
Jul 01 22:33:57.878434 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:57.878409    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-54dbbfbdcd-xq7ll_5dec1da6-cda0-4fb5-832a-36cea048f4d4/registry/0.log"
Jul 01 22:33:57.954783 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:57.954756    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-swv7c_111e269e-a761-4eee-9a78-3fd554431c39/node-ca/0.log"
Jul 01 22:33:59.582604 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:59.582561    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-655f7c58b4-57g8x_f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13/istio-proxy/5.log"
Jul 01 22:33:59.598455 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:33:59.598419    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-655f7c58b4-57g8x_f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13/istio-proxy/4.log"
Jul 01 22:34:00.229264 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:34:00.229237    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-r64tx_5df26038-9551-4259-af17-826c1c3480f7/serve-healthcheck-canary/0.log"
Jul 01 22:34:00.882717 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:34:00.882684    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wpqxs_da6e12d2-79f9-4aef-a545-2329fae14c68/kube-rbac-proxy/0.log"
Jul 01 22:34:00.904407 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:34:00.904382    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wpqxs_da6e12d2-79f9-4aef-a545-2329fae14c68/exporter/0.log"
Jul 01 22:34:00.928150 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:34:00.928126    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wpqxs_da6e12d2-79f9-4aef-a545-2329fae14c68/extractor/0.log"
Jul 01 22:34:02.289145 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:34:02.289122    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-z7v9q/perf-node-gather-daemonset-76fpm"
Jul 01 22:34:03.829612 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:34:03.829531    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-55f5fcc78b-5zh2r_91ab49c8-06a7-44a4-9c12-25f87767dca3/manager/0.log"
Jul 01 22:34:03.890530 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:34:03.890510    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-ktmkv_f3e9046b-d3fa-41ab-8a22-d3ea9adbdc4e/openshift-lws-operator/0.log"
Jul 01 22:34:06.968194 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:34:06.968159    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-655f7c58b4-57g8x_f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13/istio-proxy/5.log"
Jul 01 22:34:06.984113 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:34:06.984093    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-655f7c58b4-57g8x_f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13/istio-proxy/4.log"
Jul 01 22:34:08.753716 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:34:08.753685    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-655f7c58b4-57g8x_f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13/istio-proxy/5.log"
Jul 01 22:34:08.769078 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:34:08.769047    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-655f7c58b4-57g8x_f1d4abbb-fa80-4dff-8760-4c5e5f0d7c13/istio-proxy/4.log"
Jul 01 22:34:11.970885 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:34:11.970859    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qbnbc_9a45d9e0-af21-4075-9578-09d1913a1082/kube-multus-additional-cni-plugins/0.log"
Jul 01 22:34:11.992810 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:34:11.992780    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qbnbc_9a45d9e0-af21-4075-9578-09d1913a1082/egress-router-binary-copy/0.log"
Jul 01 22:34:12.014032 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:34:12.014003    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qbnbc_9a45d9e0-af21-4075-9578-09d1913a1082/cni-plugins/0.log"
Jul 01 22:34:12.037403 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:34:12.037367    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qbnbc_9a45d9e0-af21-4075-9578-09d1913a1082/bond-cni-plugin/0.log"
Jul 01 22:34:12.060295 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:34:12.060269    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qbnbc_9a45d9e0-af21-4075-9578-09d1913a1082/routeoverride-cni/0.log"
Jul 01 22:34:12.085966 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:34:12.085942    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qbnbc_9a45d9e0-af21-4075-9578-09d1913a1082/whereabouts-cni-bincopy/0.log"
Jul 01 22:34:12.107795 ip-10-0-140-104 kubenswrapper[2571]: I0701 22:34:12.107763    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qbnbc_9a45d9e0-af21-4075-9578-09d1913a1082/whereabouts-cni/0.log"