Jul 13 14:46:27.644880 ip-10-0-136-96 systemd[1]: Starting Kubernetes Kubelet... Jul 13 14:46:28.040168 ip-10-0-136-96 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 14:46:28.040168 ip-10-0-136-96 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 13 14:46:28.040168 ip-10-0-136-96 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 14:46:28.040168 ip-10-0-136-96 kubenswrapper[2568]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 13 14:46:28.040168 ip-10-0-136-96 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 14:46:28.040168 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.029562 2568 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 13 14:46:28.040168 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035006 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 14:46:28.040168 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035022 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 14:46:28.040168 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035027 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 14:46:28.040168 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035032 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 14:46:28.040168 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035036 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 14:46:28.040168 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035040 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 14:46:28.040168 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035044 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 14:46:28.040168 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035048 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 14:46:28.042549 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035052 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 14:46:28.042549 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035056 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 14:46:28.042549 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035061 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 14:46:28.042549 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035065 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 14:46:28.042549 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035070 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 14:46:28.042549 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035073 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 14:46:28.042549 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035077 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 14:46:28.042549 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035082 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 14:46:28.042549 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035086 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 14:46:28.042549 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035090 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 14:46:28.042549 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035093 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 14:46:28.042549 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035108 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 14:46:28.042549 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035112 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 14:46:28.042549 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035116 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 14:46:28.042549 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035120 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 14:46:28.042549 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035124 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 14:46:28.042549 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035128 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 14:46:28.042549 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035132 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 14:46:28.042549 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035136 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 14:46:28.042549 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035140 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 14:46:28.043115 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035144 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 14:46:28.043115 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035147 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 14:46:28.043115 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035151 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 14:46:28.043115 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035155 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 14:46:28.043115 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035159 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 14:46:28.043115 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035163 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 14:46:28.043115 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035168 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 14:46:28.043115 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035171 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 14:46:28.043115 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035175 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 14:46:28.043115 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035179 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 14:46:28.043115 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035184 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 14:46:28.043115 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035188 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 14:46:28.043115 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035192 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 14:46:28.043115 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035196 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 14:46:28.043115 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035200 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 14:46:28.043115 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035203 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 14:46:28.043115 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035207 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 14:46:28.043115 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035211 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 14:46:28.043115 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035215 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 14:46:28.043115 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035219 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 14:46:28.043679 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035225 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 14:46:28.043679 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035232 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 14:46:28.043679 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035238 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 14:46:28.043679 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035242 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 14:46:28.043679 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035246 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 14:46:28.043679 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035250 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 14:46:28.043679 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035254 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 14:46:28.043679 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035258 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 14:46:28.043679 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035262 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 14:46:28.043679 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035266 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 14:46:28.043679 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035271 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 14:46:28.043679 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035275 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 14:46:28.043679 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035279 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 14:46:28.043679 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035283 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 14:46:28.043679 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035287 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 14:46:28.043679 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035292 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 14:46:28.043679 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035296 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 13 14:46:28.043679 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035300 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 14:46:28.043679 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035305 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 14:46:28.044260 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035309 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 14:46:28.044260 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035313 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 14:46:28.044260 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035317 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 14:46:28.044260 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035321 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 14:46:28.044260 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035326 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 14:46:28.044260 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035330 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 14:46:28.044260 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035333 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 14:46:28.044260 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035337 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 14:46:28.044260 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035341 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 14:46:28.044260 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035345 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 14:46:28.044260 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035349 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 14:46:28.044260 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035352 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 14:46:28.044260 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035356 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 14:46:28.044260 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035359 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 14:46:28.044260 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035363 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 14:46:28.044260 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035367 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 14:46:28.044260 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035370 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 14:46:28.044260 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035374 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 14:46:28.044260 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035377 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 14:46:28.044260 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035381 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 14:46:28.226690 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035385 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 14:46:28.226690 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035388 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 14:46:28.226690 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035394 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 14:46:28.226690 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035399 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 14:46:28.226690 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035404 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 14:46:28.226690 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035410 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 14:46:28.226690 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035414 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 14:46:28.226690 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035419 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 14:46:28.226690 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035422 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 14:46:28.226690 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035426 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 14:46:28.226690 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035430 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 14:46:28.226690 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.035434 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 14:46:28.226690 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036466 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 14:46:28.226690 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036474 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 14:46:28.226690 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036479 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 14:46:28.226690 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036483 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 14:46:28.226690 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036487 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 14:46:28.226690 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036498 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 14:46:28.226690 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036503 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 14:46:28.107021 ip-10-0-136-96 systemd[1]: Started Kubernetes Kubelet. Jul 13 14:46:28.239282 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036507 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 14:46:28.239282 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036511 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 14:46:28.239282 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036515 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 14:46:28.239282 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036519 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 14:46:28.239282 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036523 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 14:46:28.239282 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036527 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 14:46:28.239282 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036530 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 14:46:28.239282 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036534 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 14:46:28.239282 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036537 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 14:46:28.239282 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036542 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 14:46:28.239282 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036546 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 14:46:28.239282 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036550 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 14:46:28.239282 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036554 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 14:46:28.239282 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036558 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 14:46:28.239282 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036562 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 14:46:28.239282 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036566 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 14:46:28.239282 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036584 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 14:46:28.239282 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036588 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 14:46:28.239282 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036592 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 14:46:28.239282 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036595 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 14:46:28.240063 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036600 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 14:46:28.240063 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036603 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 14:46:28.240063 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036607 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 14:46:28.240063 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036611 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 14:46:28.240063 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036614 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 14:46:28.240063 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036618 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 14:46:28.240063 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036622 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 14:46:28.240063 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036626 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 14:46:28.240063 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036629 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 14:46:28.240063 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036635 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 14:46:28.240063 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036641 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 14:46:28.240063 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036653 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 14:46:28.240063 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036658 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 14:46:28.240063 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036663 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 14:46:28.240063 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036666 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 14:46:28.240063 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036671 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 14:46:28.240063 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036674 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 14:46:28.240063 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036678 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 14:46:28.240063 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036684 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 14:46:28.240063 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036688 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 14:46:28.240861 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036692 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 14:46:28.240861 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036695 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 14:46:28.240861 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036699 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 14:46:28.240861 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036702 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 14:46:28.240861 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036706 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 14:46:28.240861 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036710 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 14:46:28.240861 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036714 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 13 14:46:28.240861 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036718 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 14:46:28.240861 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036722 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 14:46:28.240861 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036726 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 14:46:28.240861 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036730 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 14:46:28.240861 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036734 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 14:46:28.240861 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036738 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 14:46:28.240861 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036741 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 14:46:28.240861 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036745 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 14:46:28.240861 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036751 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 14:46:28.240861 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036756 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 14:46:28.240861 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036776 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 14:46:28.240861 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036781 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 14:46:28.241480 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036785 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 14:46:28.241480 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036789 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 14:46:28.241480 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036793 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 14:46:28.241480 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036797 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 14:46:28.241480 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036800 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 14:46:28.241480 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036805 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 14:46:28.241480 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036810 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 14:46:28.241480 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036814 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 14:46:28.241480 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036817 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 14:46:28.241480 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036821 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 14:46:28.241480 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036825 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 14:46:28.241480 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036828 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 14:46:28.241480 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036832 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 14:46:28.241480 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036837 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 14:46:28.241480 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036841 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 14:46:28.241480 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036844 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 14:46:28.241480 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036848 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 14:46:28.241480 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036852 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 14:46:28.241480 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036856 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 14:46:28.242097 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036860 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 14:46:28.242097 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036864 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 14:46:28.242097 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036867 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 14:46:28.242097 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036871 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 14:46:28.242097 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036874 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 14:46:28.242097 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036878 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 14:46:28.242097 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036882 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 14:46:28.242097 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036886 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 14:46:28.242097 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036892 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 14:46:28.242097 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036897 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 14:46:28.242097 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036901 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 14:46:28.242097 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036905 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 14:46:28.242097 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036910 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 14:46:28.242097 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.036915 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 14:46:28.242097 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037533 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 14:46:28.242097 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037539 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 14:46:28.242097 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037543 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 14:46:28.242097 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037548 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 14:46:28.242097 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037554 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 14:46:28.242714 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037558 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 14:46:28.242714 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037562 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 14:46:28.242714 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037566 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 14:46:28.242714 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037587 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 14:46:28.242714 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037591 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 14:46:28.242714 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037595 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 14:46:28.242714 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037600 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 14:46:28.242714 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037603 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 14:46:28.242714 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037607 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 14:46:28.242714 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037611 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 13 14:46:28.242714 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037615 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 14:46:28.242714 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037619 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 14:46:28.242714 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037623 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 14:46:28.242714 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037627 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 14:46:28.242714 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037631 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 14:46:28.242714 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037636 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 14:46:28.242714 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037640 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 14:46:28.242714 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037644 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 14:46:28.242714 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037647 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 14:46:28.242714 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037651 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 14:46:28.243490 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037655 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 14:46:28.243490 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037659 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 14:46:28.243490 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037663 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 14:46:28.243490 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037667 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 14:46:28.243490 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037670 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 14:46:28.243490 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037674 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 14:46:28.243490 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037677 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 14:46:28.243490 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037681 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 14:46:28.243490 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037684 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 14:46:28.243490 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037688 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 14:46:28.243490 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037691 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 14:46:28.243490 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037696 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 14:46:28.243490 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037700 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 14:46:28.243490 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037705 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 14:46:28.243490 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037710 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 14:46:28.243490 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037714 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 14:46:28.243490 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037719 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 14:46:28.243490 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037723 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 14:46:28.243490 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037726 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 14:46:28.244070 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037730 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 14:46:28.244070 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037733 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 14:46:28.244070 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037737 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 14:46:28.244070 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037740 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 14:46:28.244070 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037744 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 14:46:28.244070 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037748 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 14:46:28.244070 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037751 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 14:46:28.244070 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037755 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 14:46:28.244070 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037758 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 14:46:28.244070 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037765 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 14:46:28.244070 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037768 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 14:46:28.244070 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037772 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 14:46:28.244070 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037775 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 14:46:28.244070 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037779 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 14:46:28.244070 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037785 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 14:46:28.244070 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037788 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 14:46:28.244070 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037792 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 14:46:28.244070 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037796 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 14:46:28.244070 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037799 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 14:46:28.244070 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037804 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 14:46:28.244698 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037807 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 14:46:28.244698 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037811 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 14:46:28.244698 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037815 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 14:46:28.244698 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037819 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 14:46:28.244698 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037822 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 14:46:28.244698 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037826 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 14:46:28.244698 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037830 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 14:46:28.244698 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037834 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 14:46:28.244698 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037837 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 14:46:28.244698 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037841 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 14:46:28.244698 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037844 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 14:46:28.244698 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037850 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 14:46:28.244698 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037856 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 14:46:28.244698 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037860 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 14:46:28.244698 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037863 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 14:46:28.244698 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037869 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 14:46:28.244698 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037872 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 14:46:28.244698 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037876 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 14:46:28.244698 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037881 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 14:46:28.245236 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037884 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 14:46:28.245236 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037888 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 14:46:28.245236 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037894 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 14:46:28.245236 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037898 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 14:46:28.245236 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037902 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 14:46:28.245236 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037906 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 14:46:28.245236 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037912 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 14:46:28.245236 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037917 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 14:46:28.245236 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037921 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 14:46:28.245236 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037928 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 14:46:28.245236 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037932 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 14:46:28.245236 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037936 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 14:46:28.245236 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037940 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 14:46:28.245236 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037943 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 14:46:28.245236 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037947 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 14:46:28.245236 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.037950 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 14:46:28.245236 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038045 2568 flags.go:64] FLAG: --address="0.0.0.0" Jul 13 14:46:28.245236 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038055 2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 13 14:46:28.245236 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038064 2568 flags.go:64] FLAG: --anonymous-auth="true" Jul 13 14:46:28.245236 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038071 2568 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 13 14:46:28.245236 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038079 2568 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 13 14:46:28.245912 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038084 2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 13 14:46:28.245912 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038090 2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 13 14:46:28.245912 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038096 2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 13 14:46:28.245912 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038101 2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 13 14:46:28.245912 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038105 2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 13 14:46:28.245912 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038111 2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 13 14:46:28.245912 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038116 2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 13 14:46:28.245912 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038120 2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 13 14:46:28.245912 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038125 2568 flags.go:64] FLAG: --cgroup-root="" Jul 13 14:46:28.245912 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038129 2568 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 13 14:46:28.245912 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038133 2568 flags.go:64] FLAG: --client-ca-file="" Jul 13 14:46:28.245912 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038137 2568 flags.go:64] FLAG: --cloud-provider="external" Jul 13 14:46:28.245912 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038141 2568 flags.go:64] FLAG: --cluster-dns="[]" Jul 13 14:46:28.245912 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038152 2568 flags.go:64] FLAG: --cluster-domain="" Jul 13 14:46:28.245912 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038157 2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 13 14:46:28.245912 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038162 2568 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 13 14:46:28.245912 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038166 2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 13 14:46:28.245912 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038171 2568 flags.go:64] FLAG: --container-log-max-files="5" Jul 13 14:46:28.245912 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038177 2568 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 13 14:46:28.245912 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038181 2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 13 14:46:28.245912 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038187 2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 13 14:46:28.245912 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038192 2568 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 13 14:46:28.245912 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038196 2568 flags.go:64] FLAG: --contention-profiling="false" Jul 13 14:46:28.245912 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038201 2568 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 13 14:46:28.246562 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038206 2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 13 14:46:28.246562 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038211 2568 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 13 14:46:28.246562 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038215 2568 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 13 14:46:28.246562 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038221 2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 13 14:46:28.246562 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038226 2568 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 13 14:46:28.246562 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038229 2568 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 13 14:46:28.246562 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038234 2568 flags.go:64] FLAG: --enable-load-reader="false" Jul 13 14:46:28.246562 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038238 2568 flags.go:64] FLAG: --enable-server="true" Jul 13 14:46:28.246562 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038243 2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 13 14:46:28.246562 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038251 2568 flags.go:64] FLAG: --event-burst="100" Jul 13 14:46:28.246562 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038256 2568 flags.go:64] FLAG: --event-qps="50" Jul 13 14:46:28.246562 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038260 2568 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 13 14:46:28.246562 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038265 2568 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 13 14:46:28.246562 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038269 2568 flags.go:64] FLAG: --eviction-hard="" Jul 13 14:46:28.246562 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038275 2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 13 14:46:28.246562 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038279 2568 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 13 14:46:28.246562 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038284 2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 13 14:46:28.246562 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038288 2568 flags.go:64] FLAG: --eviction-soft="" Jul 13 14:46:28.246562 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038293 2568 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 13 14:46:28.246562 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038297 2568 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 13 14:46:28.246562 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038301 2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 13 14:46:28.246562 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038307 2568 flags.go:64] FLAG: --experimental-mounter-path="" Jul 13 14:46:28.246562 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038311 2568 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 13 14:46:28.246562 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038315 2568 flags.go:64] FLAG: --fail-swap-on="true" Jul 13 14:46:28.246562 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038320 2568 flags.go:64] FLAG: --feature-gates="" Jul 13 14:46:28.247331 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038325 2568 flags.go:64] FLAG: --file-check-frequency="20s" Jul 13 14:46:28.247331 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038329 2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 13 14:46:28.247331 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038334 2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 13 14:46:28.247331 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038339 2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 13 14:46:28.247331 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038344 2568 flags.go:64] FLAG: --healthz-port="10248" Jul 13 14:46:28.247331 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038349 2568 flags.go:64] FLAG: --help="false" Jul 13 14:46:28.247331 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038353 2568 flags.go:64] FLAG: --hostname-override="ip-10-0-136-96.ec2.internal" Jul 13 14:46:28.247331 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038358 2568 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 13 14:46:28.247331 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038362 2568 flags.go:64] FLAG: --http-check-frequency="20s" Jul 13 14:46:28.247331 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038366 2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 13 14:46:28.247331 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038371 2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 13 14:46:28.247331 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038376 2568 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 13 14:46:28.247331 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038381 2568 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 13 14:46:28.247331 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038385 2568 flags.go:64] FLAG: --image-service-endpoint="" Jul 13 14:46:28.247331 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038389 2568 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 13 14:46:28.247331 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038392 2568 flags.go:64] FLAG: --kube-api-burst="100" Jul 13 14:46:28.247331 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038398 2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 13 14:46:28.247331 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038403 2568 flags.go:64] FLAG: --kube-api-qps="50" Jul 13 14:46:28.247331 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038408 2568 flags.go:64] FLAG: --kube-reserved="" Jul 13 14:46:28.247331 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038412 2568 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 13 14:46:28.247331 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038416 2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 13 14:46:28.247331 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038421 2568 flags.go:64] FLAG: --kubelet-cgroups="" Jul 13 14:46:28.247331 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038425 2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 13 14:46:28.247331 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038430 2568 flags.go:64] FLAG: --lock-file="" Jul 13 14:46:28.248014 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038434 2568 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 13 14:46:28.248014 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038438 2568 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 13 14:46:28.248014 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038442 2568 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 13 14:46:28.248014 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038451 2568 flags.go:64] FLAG: --log-json-split-stream="false" Jul 13 14:46:28.248014 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038457 2568 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 13 14:46:28.248014 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038461 2568 flags.go:64] FLAG: --log-text-split-stream="false" Jul 13 14:46:28.248014 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038466 2568 flags.go:64] FLAG: --logging-format="text" Jul 13 14:46:28.248014 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038470 2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 13 14:46:28.248014 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038475 2568 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 13 14:46:28.248014 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038479 2568 flags.go:64] FLAG: --manifest-url="" Jul 13 14:46:28.248014 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038483 2568 flags.go:64] FLAG: --manifest-url-header="" Jul 13 14:46:28.248014 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038491 2568 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 13 14:46:28.248014 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038495 2568 flags.go:64] FLAG: --max-open-files="1000000" Jul 13 14:46:28.248014 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038502 2568 flags.go:64] FLAG: --max-pods="110" Jul 13 14:46:28.248014 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038506 2568 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 13 14:46:28.248014 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038511 2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 13 14:46:28.248014 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038515 2568 flags.go:64] FLAG: --memory-manager-policy="None" Jul 13 14:46:28.248014 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038520 2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 13 14:46:28.248014 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038524 2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 13 14:46:28.248014 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038529 2568 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 13 14:46:28.248014 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038533 2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 13 14:46:28.248014 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038545 2568 flags.go:64] FLAG: --node-status-max-images="50" Jul 13 14:46:28.248014 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038549 2568 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 13 14:46:28.248014 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038554 2568 flags.go:64] FLAG: --oom-score-adj="-999" Jul 13 14:46:28.248014 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038558 2568 flags.go:64] FLAG: --pod-cidr="" Jul 13 14:46:28.248731 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038563 2568 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 13 14:46:28.248731 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038586 2568 flags.go:64] FLAG: --pod-manifest-path="" Jul 13 14:46:28.248731 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038590 2568 flags.go:64] FLAG: --pod-max-pids="-1" Jul 13 14:46:28.248731 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038594 2568 flags.go:64] FLAG: --pods-per-core="0" Jul 13 14:46:28.248731 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038599 2568 flags.go:64] FLAG: --port="10250" Jul 13 14:46:28.248731 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038603 2568 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 13 14:46:28.248731 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038607 2568 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03f9aaa118412fa6f" Jul 13 14:46:28.248731 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038612 2568 flags.go:64] FLAG: --qos-reserved="" Jul 13 14:46:28.248731 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038617 2568 flags.go:64] FLAG: --read-only-port="10255" Jul 13 14:46:28.248731 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038621 2568 flags.go:64] FLAG: --register-node="true" Jul 13 14:46:28.248731 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038625 2568 flags.go:64] FLAG: --register-with-taints="" Jul 13 14:46:28.248731 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038632 2568 flags.go:64] FLAG: --registry-burst="10" Jul 13 14:46:28.248731 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038636 2568 flags.go:64] FLAG: --registry-qps="5" Jul 13 14:46:28.248731 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038640 2568 flags.go:64] FLAG: --reserved-cpus="" Jul 13 14:46:28.248731 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038645 2568 flags.go:64] FLAG: --reserved-memory="" Jul 13 14:46:28.248731 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038650 2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 13 14:46:28.248731 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038655 2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 13 14:46:28.248731 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038659 2568 flags.go:64] FLAG: --rotate-certificates="false" Jul 13 14:46:28.248731 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038663 2568 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 13 14:46:28.248731 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038669 2568 flags.go:64] FLAG: --runonce="false" Jul 13 14:46:28.248731 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038673 2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 13 14:46:28.248731 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038678 2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 13 14:46:28.248731 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038682 2568 flags.go:64] FLAG: --seccomp-default="false" Jul 13 14:46:28.248731 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038687 2568 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 13 14:46:28.248731 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038691 2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 13 14:46:28.249750 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038695 2568 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 13 14:46:28.249750 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038700 2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 13 14:46:28.249750 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038704 2568 flags.go:64] FLAG: --storage-driver-password="root" Jul 13 14:46:28.249750 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038708 2568 flags.go:64] FLAG: --storage-driver-secure="false" Jul 13 14:46:28.249750 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038712 2568 flags.go:64] FLAG: --storage-driver-table="stats" Jul 13 14:46:28.249750 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038717 2568 flags.go:64] FLAG: --storage-driver-user="root" Jul 13 14:46:28.249750 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038721 2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 13 14:46:28.249750 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038725 2568 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 13 14:46:28.249750 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038730 2568 flags.go:64] FLAG: --system-cgroups="" Jul 13 14:46:28.249750 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038734 2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 13 14:46:28.249750 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038742 2568 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 13 14:46:28.249750 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038746 2568 flags.go:64] FLAG: --tls-cert-file="" Jul 13 14:46:28.249750 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038751 2568 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 13 14:46:28.249750 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038756 2568 flags.go:64] FLAG: --tls-min-version="" Jul 13 14:46:28.249750 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038760 2568 flags.go:64] FLAG: --tls-private-key-file="" Jul 13 14:46:28.249750 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038764 2568 flags.go:64] FLAG: --topology-manager-policy="none" Jul 13 14:46:28.249750 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038769 2568 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 13 14:46:28.249750 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038773 2568 flags.go:64] FLAG: --topology-manager-scope="container" Jul 13 14:46:28.249750 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038780 2568 flags.go:64] FLAG: --v="2" Jul 13 14:46:28.249750 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038786 2568 flags.go:64] FLAG: --version="false" Jul 13 14:46:28.249750 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038792 2568 flags.go:64] FLAG: --vmodule="" Jul 13 14:46:28.249750 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038797 2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 13 14:46:28.249750 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.038802 2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 13 14:46:28.249750 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.038971 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 14:46:28.249750 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.038978 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 14:46:28.250449 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.038982 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 14:46:28.250449 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.038987 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 14:46:28.250449 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.038992 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 14:46:28.250449 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.038995 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 14:46:28.250449 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039000 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 14:46:28.250449 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039005 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 14:46:28.250449 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039011 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 14:46:28.250449 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039015 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 14:46:28.250449 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039019 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 14:46:28.250449 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039023 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 14:46:28.250449 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039027 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 14:46:28.250449 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039031 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 14:46:28.250449 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039035 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 14:46:28.250449 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039039 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 14:46:28.250449 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039042 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 14:46:28.250449 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039048 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 14:46:28.250449 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039053 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 14:46:28.250449 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039057 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 14:46:28.250449 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039061 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 14:46:28.453233 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039065 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 14:46:28.453233 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039068 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 14:46:28.453233 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039072 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 14:46:28.453233 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039076 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 14:46:28.453233 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039080 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 14:46:28.453233 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039084 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 14:46:28.453233 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039089 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 14:46:28.453233 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039093 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 14:46:28.453233 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039097 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 14:46:28.453233 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039101 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 14:46:28.453233 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039105 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 14:46:28.453233 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039109 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 14:46:28.453233 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039112 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 14:46:28.453233 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039116 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 14:46:28.453233 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039121 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 14:46:28.453233 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039125 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 14:46:28.453233 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039128 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 14:46:28.453233 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039132 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 14:46:28.453233 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039136 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 14:46:28.454418 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039140 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 14:46:28.454418 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039144 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 14:46:28.454418 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039148 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 14:46:28.454418 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039152 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 14:46:28.454418 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039156 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 14:46:28.454418 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039160 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 14:46:28.454418 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039164 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 14:46:28.454418 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039167 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 14:46:28.454418 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039171 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 14:46:28.454418 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039175 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 14:46:28.454418 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039179 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 14:46:28.454418 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039183 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 14:46:28.454418 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039187 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 14:46:28.454418 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039190 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 14:46:28.454418 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039195 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 14:46:28.454418 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039198 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 14:46:28.454418 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039203 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 14:46:28.454418 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039206 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 14:46:28.454418 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039210 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 14:46:28.454418 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039215 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 14:46:28.455166 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039219 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 14:46:28.455166 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039222 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 14:46:28.455166 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039226 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 14:46:28.455166 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039230 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 14:46:28.455166 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039234 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 14:46:28.455166 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039237 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 14:46:28.455166 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039242 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 14:46:28.455166 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039247 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 14:46:28.455166 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039251 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 14:46:28.455166 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039257 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 14:46:28.455166 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039262 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 14:46:28.455166 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039267 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 14:46:28.455166 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039271 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 14:46:28.455166 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039276 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 14:46:28.455166 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039280 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 14:46:28.455166 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039284 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 14:46:28.455166 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039287 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 14:46:28.455166 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039291 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 13 14:46:28.455166 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039295 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 14:46:28.455782 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039299 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 14:46:28.455782 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039303 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 14:46:28.455782 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039306 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 14:46:28.455782 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039310 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 14:46:28.455782 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039313 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 14:46:28.455782 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039319 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 14:46:28.455782 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039323 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 14:46:28.455782 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039327 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 14:46:28.455782 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039331 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 14:46:28.455782 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039335 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 14:46:28.455782 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039339 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 14:46:28.455782 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039343 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 14:46:28.455782 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039348 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 14:46:28.455782 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039352 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 14:46:28.455782 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039355 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 14:46:28.455782 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039359 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 14:46:28.455782 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039363 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 14:46:28.455782 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039367 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 14:46:28.455782 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039371 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 14:46:28.455782 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.039374 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 14:46:28.456359 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.040008 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 14:46:28.456359 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.047039 2568 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 13 14:46:28.456359 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.047055 2568 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 13 14:46:28.456359 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.047650 2568 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 13 14:46:28.456359 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.047657 2568 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 13 14:46:28.456359 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047717 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 14:46:28.456359 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047722 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 14:46:28.456359 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047726 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 14:46:28.456359 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047730 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 14:46:28.456359 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047733 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 14:46:28.456359 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047736 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 14:46:28.456359 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047738 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 14:46:28.456359 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047741 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 14:46:28.456359 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047744 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 14:46:28.456359 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047748 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 14:46:28.456866 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047750 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 14:46:28.456866 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047753 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 14:46:28.456866 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047756 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 14:46:28.456866 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047759 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 14:46:28.456866 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047762 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 14:46:28.456866 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047765 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 14:46:28.456866 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047767 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 14:46:28.456866 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047770 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 14:46:28.456866 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047774 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 14:46:28.456866 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047777 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 14:46:28.456866 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047780 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 14:46:28.456866 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047782 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 14:46:28.456866 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047785 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 14:46:28.456866 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047788 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 14:46:28.456866 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047790 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 14:46:28.456866 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047793 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 14:46:28.456866 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047796 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 14:46:28.456866 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047799 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 14:46:28.456866 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047802 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 14:46:28.456866 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047806 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 14:46:28.457458 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047809 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 14:46:28.457458 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047812 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 14:46:28.457458 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047814 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 14:46:28.457458 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047817 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 14:46:28.457458 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047820 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 14:46:28.457458 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047822 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 14:46:28.457458 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047825 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 14:46:28.457458 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047828 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 14:46:28.457458 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047830 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 14:46:28.457458 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047833 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 14:46:28.457458 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047836 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 14:46:28.457458 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047839 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 14:46:28.457458 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047842 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 14:46:28.457458 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047846 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 14:46:28.457458 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047849 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 14:46:28.457458 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047852 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 14:46:28.457458 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047854 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 14:46:28.457458 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047857 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 14:46:28.457458 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047860 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 14:46:28.457458 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047863 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 14:46:28.458089 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047865 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 14:46:28.458089 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047868 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 14:46:28.458089 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047871 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 14:46:28.458089 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047873 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 14:46:28.458089 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047876 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 14:46:28.458089 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047879 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 14:46:28.458089 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047881 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 14:46:28.458089 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047884 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 14:46:28.458089 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047887 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 13 14:46:28.458089 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047889 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 14:46:28.458089 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047892 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 14:46:28.458089 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047896 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 14:46:28.458089 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047900 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 14:46:28.458089 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047903 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 14:46:28.458089 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047905 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 14:46:28.458089 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047908 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 14:46:28.458089 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047911 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 14:46:28.458089 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047914 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 14:46:28.458089 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047916 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 14:46:28.458624 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047926 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 14:46:28.458624 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047929 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 14:46:28.458624 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047932 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 14:46:28.458624 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047935 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 14:46:28.458624 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047938 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 14:46:28.458624 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047941 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 14:46:28.458624 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047944 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 14:46:28.458624 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047947 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 14:46:28.458624 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047949 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 14:46:28.458624 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047952 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 14:46:28.458624 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047954 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 14:46:28.458624 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047957 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 14:46:28.458624 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047960 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 14:46:28.458624 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047962 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 14:46:28.458624 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047965 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 14:46:28.458624 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047968 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 14:46:28.458624 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047971 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 14:46:28.458624 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047973 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 14:46:28.458624 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047976 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 14:46:28.458624 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047978 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 14:46:28.459271 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047982 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 14:46:28.459271 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047986 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 14:46:28.459271 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047989 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 14:46:28.459271 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047992 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 14:46:28.459271 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047995 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 14:46:28.459271 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.047998 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 14:46:28.459271 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048001 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 14:46:28.459271 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048009 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 14:46:28.459271 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048011 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 14:46:28.459271 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048014 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 14:46:28.459271 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.048019 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 14:46:28.459271 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048134 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 14:46:28.459271 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048139 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 14:46:28.459271 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048142 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 14:46:28.459271 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048145 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 14:46:28.459757 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048148 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 14:46:28.459757 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048151 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 14:46:28.459757 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048154 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 14:46:28.459757 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048157 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 14:46:28.459757 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048159 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 14:46:28.459757 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048162 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 14:46:28.459757 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048164 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 14:46:28.459757 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048167 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 14:46:28.459757 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048170 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 14:46:28.459757 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048172 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 14:46:28.459757 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048175 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 14:46:28.459757 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048178 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 14:46:28.459757 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048180 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 14:46:28.459757 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048183 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 14:46:28.459757 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048187 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 14:46:28.459757 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048190 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 14:46:28.459757 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048193 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 14:46:28.459757 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048195 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 14:46:28.459757 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048198 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 14:46:28.459757 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048201 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 14:46:28.460354 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048203 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 13 14:46:28.460354 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048206 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 14:46:28.460354 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048209 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 14:46:28.460354 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048213 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 14:46:28.460354 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048217 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 14:46:28.460354 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048219 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 14:46:28.460354 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048222 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 14:46:28.460354 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048224 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 14:46:28.460354 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048227 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 14:46:28.460354 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048229 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 14:46:28.460354 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048232 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 14:46:28.460354 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048234 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 14:46:28.460354 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048237 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 14:46:28.460354 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048239 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 14:46:28.460354 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048242 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 14:46:28.460354 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048246 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 14:46:28.460354 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048249 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 14:46:28.460354 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048252 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 14:46:28.460354 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048255 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 14:46:28.460916 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048258 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 14:46:28.460916 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048261 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 14:46:28.460916 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048263 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 14:46:28.460916 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048266 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 14:46:28.460916 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048268 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 14:46:28.460916 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048271 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 14:46:28.460916 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048273 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 14:46:28.460916 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048277 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 14:46:28.460916 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048279 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 14:46:28.460916 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048282 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 14:46:28.460916 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048284 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 14:46:28.460916 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048287 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 14:46:28.460916 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048289 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 14:46:28.460916 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048292 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 14:46:28.460916 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048294 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 14:46:28.460916 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048297 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 14:46:28.460916 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048299 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 14:46:28.460916 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048304 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 14:46:28.460916 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048307 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 14:46:28.461499 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048310 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 14:46:28.461499 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048313 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 14:46:28.461499 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048316 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 14:46:28.461499 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048319 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 14:46:28.461499 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048322 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 14:46:28.461499 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048324 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 14:46:28.461499 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048327 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 14:46:28.461499 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048329 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 14:46:28.461499 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048332 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 14:46:28.461499 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048334 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 14:46:28.461499 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048337 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 14:46:28.461499 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048339 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 14:46:28.461499 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048342 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 14:46:28.461499 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048344 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 14:46:28.461499 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048347 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 14:46:28.461499 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048349 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 14:46:28.461499 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048352 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 14:46:28.461499 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048354 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 14:46:28.461499 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048357 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 14:46:28.462155 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048359 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 14:46:28.462155 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048363 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 14:46:28.462155 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048366 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 14:46:28.462155 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048368 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 14:46:28.462155 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048371 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 14:46:28.462155 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048373 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 14:46:28.462155 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048376 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 14:46:28.462155 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048379 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 14:46:28.462155 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048382 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 14:46:28.462155 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048385 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 14:46:28.462155 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048387 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 14:46:28.462155 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048391 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 14:46:28.462155 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048393 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 14:46:28.462155 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048396 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 14:46:28.462155 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048399 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 14:46:28.462155 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048402 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 14:46:28.462155 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048404 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 14:46:28.462155 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:28.048406 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 14:46:28.462743 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.048412 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 14:46:28.462743 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.048525 2568 server.go:962] "Client rotation is on, will bootstrap in background" Jul 13 14:46:28.462743 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.050471 2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 13 14:46:28.462743 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.051664 2568 server.go:1019] "Starting client certificate rotation" Jul 13 14:46:28.462743 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.051769 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 14:46:28.462743 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.051806 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 14:46:28.462743 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.055112 2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 14:46:28.462743 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.058517 2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 14:46:28.462743 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.070911 2568 log.go:25] "Validated CRI v1 runtime API" Jul 13 14:46:28.462743 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.072996 2568 log.go:25] "Validated CRI v1 image API" Jul 13 14:46:28.462743 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.073738 2568 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 13 14:46:28.462743 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.076059 2568 fs.go:135] Filesystem UUIDs: map[26545fd8-2876-4a5e-a230-3c6201ad308e:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 ac059dea-9fc6-405c-be25-b67b0cbb7417:/dev/nvme0n1p4] Jul 13 14:46:28.463124 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.076081 2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 13 14:46:28.463239 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.080494 2568 manager.go:216] Machine: {Timestamp:2026-07-13 14:46:28.079774771 +0000 UTC m=+0.320052623 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098588 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2052c4d34bfb67862e9c2f49a31574 SystemUUID:ec2052c4-d34b-fb67-862e-9c2f49a31574 BootID:cbb7d53c-a05c-420e-9694-7636001aa402 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:2e:96:ef:92:d1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:2e:96:ef:92:d1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9e:77:6b:b8:b2:cb Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 13 14:46:28.463239 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.080692 2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 13 14:46:28.463239 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.080884 2568 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 13 14:46:28.463239 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.084734 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 14:46:28.463239 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.084783 2568 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 13 14:46:28.463239 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.084818 2568 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-96.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 13 14:46:28.463239 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.085968 2568 topology_manager.go:138] "Creating topology manager with none policy" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.085979 2568 container_manager_linux.go:309] "Creating device plugin manager" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.085986 2568 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.086543 2568 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.088508 2568 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.091407 2568 state_mem.go:36] "Initialized new in-memory state store" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.091519 2568 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.091554 2568 kubelet.go:486] "Attempting to sync node with API server" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.091582 2568 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.091601 2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.091613 2568 kubelet.go:398] "Adding apiserver pod source" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.091625 2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.092003 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.092020 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.094057 2568 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.095885 2568 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.095913 2568 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.096101 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.096119 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.096124 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.096130 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.096138 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.096144 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.096529 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.096534 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.096543 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.096548 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.096561 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.096587 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.097567 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.097597 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.105739 2568 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-96.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:28.105770 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-96.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:28.105878 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.106158 2568 server.go:1268] "Started kubelet" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.106225 2568 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.106278 2568 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.106353 2568 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.107938 2568 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.109149 2568 server.go:310] "Adding debug handlers to kubelet server" Jul 13 14:46:28.463503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.111497 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6psmh" Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:28.111705 2568 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:28.111414 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-96.ec2.internal.18c1e0b34cb279c1 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-96.ec2.internal,UID:ip-10-0-136-96.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-96.ec2.internal,},FirstTimestamp:2026-07-13 14:46:28.105959873 +0000 UTC m=+0.346237711,LastTimestamp:2026-07-13 14:46:28.105959873 +0000 UTC m=+0.346237711,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-96.ec2.internal,}" Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.114594 2568 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:28.114672 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-96.ec2.internal\" not found" Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.114686 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.115302 2568 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.115318 2568 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.115322 2568 factory.go:55] Registering systemd factory Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.115340 2568 factory.go:223] Registration of the systemd container factory successfully Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.115302 2568 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.115434 2568 reconstruct.go:97] "Volume reconstruction finished" Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.115442 2568 reconciler.go:29] "Reconciler: start to sync state" Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.115562 2568 factory.go:153] Registering CRI-O factory Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.115590 2568 factory.go:223] Registration of the crio container factory successfully Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.115665 2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.115689 2568 factory.go:103] Registering Raw factory Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.115701 2568 manager.go:1196] Started watching for new ooms in manager Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.116733 2568 manager.go:349] Starting recovery of all containers Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.127853 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6psmh" Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.128614 2568 manager.go:354] Recovery completed Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.134299 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.136814 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-96.ec2.internal" event="NodeHasSufficientMemory" Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.136844 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-96.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.136854 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-96.ec2.internal" event="NodeHasSufficientPID" Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.137356 2568 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.137367 2568 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.137388 2568 state_mem.go:36] "Initialized new in-memory state store" Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.137418 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.138127 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-96.ec2.internal" event="NodeHasSufficientMemory" Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.138146 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-96.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.138158 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-96.ec2.internal" event="NodeHasSufficientPID" Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.138371 2568 policy_none.go:49] "None policy: Start" Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.138384 2568 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.138394 2568 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.139376 2568 policy_none.go:47] "Start" Jul 13 14:46:28.464817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.139999 2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:28.144107 2568 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-136-96.ec2.internal\" not found" node="ip-10-0-136-96.ec2.internal" Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.171271 2568 manager.go:337] "Starting Device Plugin manager" Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:28.171309 2568 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.171319 2568 server.go:87] "Starting device plugin registration server" Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.171591 2568 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.171604 2568 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.171699 2568 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.171946 2568 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.171957 2568 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:28.172324 2568 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:28.172377 2568 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-96.ec2.internal\" not found" Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.264528 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.265761 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.265793 2568 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.265819 2568 kubelet.go:2448] "Starting kubelet main sync loop" Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:28.265854 2568 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.268202 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.272022 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.272863 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-96.ec2.internal" event="NodeHasSufficientMemory" Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.272892 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-96.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.272903 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-96.ec2.internal" event="NodeHasSufficientPID" Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.272933 2568 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-96.ec2.internal" Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.279728 2568 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-96.ec2.internal" Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:28.279754 2568 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-96.ec2.internal\": node \"ip-10-0-136-96.ec2.internal\" not found" Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:28.295363 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-96.ec2.internal\" not found" Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.367275 2568 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-96.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-96.ec2.internal"] Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.367398 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.369064 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-96.ec2.internal" event="NodeHasSufficientMemory" Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.369094 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-96.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.369104 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-96.ec2.internal" event="NodeHasSufficientPID" Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.370608 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.370707 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-96.ec2.internal" Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.370739 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.371551 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-96.ec2.internal" event="NodeHasSufficientMemory" Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.371590 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-96.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 14:46:28.466698 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.371601 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-96.ec2.internal" event="NodeHasSufficientPID" Jul 13 14:46:28.808166 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.372069 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-96.ec2.internal" event="NodeHasSufficientMemory" Jul 13 14:46:28.808166 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.372096 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-96.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 14:46:28.808166 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.372107 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-96.ec2.internal" event="NodeHasSufficientPID" Jul 13 14:46:28.808166 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.373011 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-96.ec2.internal" Jul 13 14:46:28.808166 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.373053 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 14:46:28.808166 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.373898 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-96.ec2.internal" event="NodeHasSufficientMemory" Jul 13 14:46:28.808166 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.373923 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-96.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 14:46:28.808166 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.373934 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-96.ec2.internal" event="NodeHasSufficientPID" Jul 13 14:46:28.808166 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:28.395416 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-96.ec2.internal\" not found" Jul 13 14:46:28.808166 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:28.407743 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-96.ec2.internal\" not found" node="ip-10-0-136-96.ec2.internal" Jul 13 14:46:28.808166 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:28.414750 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-96.ec2.internal\" not found" node="ip-10-0-136-96.ec2.internal" Jul 13 14:46:28.808166 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.416821 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1ad2abc2bf15f3fd836f68bef031f771-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-96.ec2.internal\" (UID: \"1ad2abc2bf15f3fd836f68bef031f771\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-96.ec2.internal" Jul 13 14:46:28.808166 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.416847 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ad2abc2bf15f3fd836f68bef031f771-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-96.ec2.internal\" (UID: \"1ad2abc2bf15f3fd836f68bef031f771\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-96.ec2.internal" Jul 13 14:46:28.808166 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.416873 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4161d8f2b036cc0e991f7a67c69322ec-config\") pod \"kube-apiserver-proxy-ip-10-0-136-96.ec2.internal\" (UID: \"4161d8f2b036cc0e991f7a67c69322ec\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-96.ec2.internal" Jul 13 14:46:28.808166 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:28.496002 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-96.ec2.internal\" not found" Jul 13 14:46:28.808166 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.517998 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ad2abc2bf15f3fd836f68bef031f771-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-96.ec2.internal\" (UID: \"1ad2abc2bf15f3fd836f68bef031f771\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-96.ec2.internal" Jul 13 14:46:28.808166 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.518037 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4161d8f2b036cc0e991f7a67c69322ec-config\") pod \"kube-apiserver-proxy-ip-10-0-136-96.ec2.internal\" (UID: \"4161d8f2b036cc0e991f7a67c69322ec\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-96.ec2.internal" Jul 13 14:46:28.808166 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.518058 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ad2abc2bf15f3fd836f68bef031f771-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-96.ec2.internal\" (UID: \"1ad2abc2bf15f3fd836f68bef031f771\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-96.ec2.internal" Jul 13 14:46:28.808166 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.518068 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1ad2abc2bf15f3fd836f68bef031f771-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-96.ec2.internal\" (UID: \"1ad2abc2bf15f3fd836f68bef031f771\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-96.ec2.internal" Jul 13 14:46:28.810411 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.518098 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1ad2abc2bf15f3fd836f68bef031f771-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-96.ec2.internal\" (UID: \"1ad2abc2bf15f3fd836f68bef031f771\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-96.ec2.internal" Jul 13 14:46:28.810411 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.518142 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4161d8f2b036cc0e991f7a67c69322ec-config\") pod \"kube-apiserver-proxy-ip-10-0-136-96.ec2.internal\" (UID: \"4161d8f2b036cc0e991f7a67c69322ec\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-96.ec2.internal" Jul 13 14:46:28.810411 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:28.596546 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-96.ec2.internal\" not found" Jul 13 14:46:28.810411 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:28.703063 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-96.ec2.internal\" not found" Jul 13 14:46:28.810411 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.709681 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-96.ec2.internal" Jul 13 14:46:28.810411 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:28.720061 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-96.ec2.internal" Jul 13 14:46:28.810411 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:28.805041 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-96.ec2.internal\" not found" Jul 13 14:46:29.114672 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:28.910321 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-96.ec2.internal\" not found" Jul 13 14:46:29.114672 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:29.012386 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-96.ec2.internal\" not found" Jul 13 14:46:29.114672 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:29.050969 2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 13 14:46:29.114672 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:29.051127 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 13 14:46:29.114672 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:29.051138 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 13 14:46:29.114672 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:29.112464 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-96.ec2.internal\" not found" Jul 13 14:46:29.376493 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:29.133221 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-12 14:41:28 +0000 UTC" deadline="2028-04-04 05:15:38.711067014 +0000 UTC" Jul 13 14:46:29.376493 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:29.133251 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15134h29m9.577819478s" Jul 13 14:46:29.376493 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:29.215341 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-96.ec2.internal\" not found" Jul 13 14:46:29.376493 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:29.315724 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-96.ec2.internal\" not found" Jul 13 14:46:29.376493 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:29.352126 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ad2abc2bf15f3fd836f68bef031f771.slice/crio-f4811e026cfe1b4c8af8cc1383edd352bf4460610443927baf3b72036415d72c WatchSource:0}: Error finding container f4811e026cfe1b4c8af8cc1383edd352bf4460610443927baf3b72036415d72c: Status 404 returned error can't find the container with id f4811e026cfe1b4c8af8cc1383edd352bf4460610443927baf3b72036415d72c Jul 13 14:46:29.376493 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:29.359211 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 14:46:29.598919 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:29.416670 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-96.ec2.internal\" not found" Jul 13 14:46:29.598919 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:29.517698 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-96.ec2.internal\" not found" Jul 13 14:46:29.758940 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:29.618258 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-96.ec2.internal\" not found" Jul 13 14:46:29.758940 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:29.684192 2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 14:46:29.758940 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:29.713814 2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 14:46:29.758940 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:29.715148 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-96.ec2.internal" Jul 13 14:46:29.758940 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:29.736273 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 14:46:29.758940 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:29.737382 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-96.ec2.internal" Jul 13 14:46:29.758940 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:29.747290 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 14:46:29.993000 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:29.988800 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 14:46:30.109438 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.093366 2568 apiserver.go:52] "Watching apiserver" Jul 13 14:46:30.109438 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.100183 2568 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 13 14:46:30.109438 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.100849 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-7v6qt","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nq9c","openshift-dns/node-resolver-xz7wd","openshift-image-registry/node-ca-qfs4l","openshift-multus/multus-k8ltk","openshift-network-operator/iptables-alerter-d7nhj","openshift-ovn-kubernetes/ovnkube-node-xgqvx","kube-system/kube-apiserver-proxy-ip-10-0-136-96.ec2.internal","openshift-cluster-node-tuning-operator/tuned-5rmzg","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-96.ec2.internal","openshift-multus/multus-additional-cni-plugins-qg9jx","openshift-multus/network-metrics-daemon-9hk94","openshift-network-diagnostics/network-check-target-r82bx","kube-system/global-pull-secret-syncer-km6w9"] Jul 13 14:46:30.109438 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.103956 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.109438 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.107192 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r82bx" Jul 13 14:46:30.109438 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:30.107259 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r82bx" podUID="191cfe54-82c1-4694-92fd-3b93bb1783fd" Jul 13 14:46:30.113036 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.110672 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xz7wd" Jul 13 14:46:30.113036 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.111730 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 13 14:46:30.113036 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.111918 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 13 14:46:30.113036 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.111974 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 13 14:46:30.113036 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.112099 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 13 14:46:30.113036 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.112137 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 13 14:46:30.113036 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.112392 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-59qsf\"" Jul 13 14:46:30.113036 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.112492 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 13 14:46:30.116764 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.114181 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qfs4l" Jul 13 14:46:30.588110 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.588069 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 13 14:46:30.588320 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.588087 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-brlvl\"" Jul 13 14:46:30.589723 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.588501 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 13 14:46:30.589723 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.588604 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 13 14:46:30.589723 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.588778 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tj98\" (UniqueName: \"kubernetes.io/projected/df5174e4-503e-475b-ab01-20dd9f45124d-kube-api-access-8tj98\") pod \"node-ca-qfs4l\" (UID: \"df5174e4-503e-475b-ab01-20dd9f45124d\") " pod="openshift-image-registry/node-ca-qfs4l" Jul 13 14:46:30.589723 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.588818 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-system-cni-dir\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.589723 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.588848 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-cnibin\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.589723 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.588870 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-os-release\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.589723 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.588898 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-systemd-units\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.589723 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.588926 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-etc-openvswitch\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.589723 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.588968 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-run-ovn\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.589723 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.589025 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-ovn-node-metrics-cert\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.589723 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.589154 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5x7x\" (UniqueName: \"kubernetes.io/projected/d05cd493-5f17-4976-b2e5-404431d3389a-kube-api-access-h5x7x\") pod \"node-resolver-xz7wd\" (UID: \"d05cd493-5f17-4976-b2e5-404431d3389a\") " pod="openshift-dns/node-resolver-xz7wd" Jul 13 14:46:30.589723 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.589206 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-host-run-netns\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.589723 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.589249 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-env-overrides\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.589723 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.589289 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-host-kubelet\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.589723 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.589320 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-var-lib-openvswitch\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.590821 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.589346 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-host-cni-netd\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.590821 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.589404 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-host-run-k8s-cni-cncf-io\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.590821 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.589494 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqbsk\" (UniqueName: \"kubernetes.io/projected/191cfe54-82c1-4694-92fd-3b93bb1783fd-kube-api-access-pqbsk\") pod \"network-check-target-r82bx\" (UID: \"191cfe54-82c1-4694-92fd-3b93bb1783fd\") " pod="openshift-network-diagnostics/network-check-target-r82bx" Jul 13 14:46:30.590821 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.589530 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6a62a407-ab2f-44ca-896d-a1e642c6667d-cni-binary-copy\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.590821 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.589554 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/df5174e4-503e-475b-ab01-20dd9f45124d-host\") pod \"node-ca-qfs4l\" (UID: \"df5174e4-503e-475b-ab01-20dd9f45124d\") " pod="openshift-image-registry/node-ca-qfs4l" Jul 13 14:46:30.590821 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.589578 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.590821 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.589623 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-ovnkube-config\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.590821 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.589674 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d05cd493-5f17-4976-b2e5-404431d3389a-tmp-dir\") pod \"node-resolver-xz7wd\" (UID: \"d05cd493-5f17-4976-b2e5-404431d3389a\") " pod="openshift-dns/node-resolver-xz7wd" Jul 13 14:46:30.590821 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.589706 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-host-slash\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.590821 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.589729 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-host-run-netns\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.590821 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.589751 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-run-openvswitch\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.590821 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.589796 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-host-cni-bin\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.590821 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.589828 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-ovnkube-script-lib\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.590821 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.589854 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-multus-cni-dir\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.591780 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.589905 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-run-systemd\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.591780 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.589951 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d05cd493-5f17-4976-b2e5-404431d3389a-hosts-file\") pod \"node-resolver-xz7wd\" (UID: \"d05cd493-5f17-4976-b2e5-404431d3389a\") " pod="openshift-dns/node-resolver-xz7wd" Jul 13 14:46:30.591780 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.589984 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/df5174e4-503e-475b-ab01-20dd9f45124d-serviceca\") pod \"node-ca-qfs4l\" (UID: \"df5174e4-503e-475b-ab01-20dd9f45124d\") " pod="openshift-image-registry/node-ca-qfs4l" Jul 13 14:46:30.591780 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.590013 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-multus-socket-dir-parent\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.591780 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.590048 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-node-log\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.591780 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.590108 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-log-socket\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.591780 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.590150 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-host-run-ovn-kubernetes\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.591780 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.590168 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zt9h\" (UniqueName: \"kubernetes.io/projected/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-kube-api-access-7zt9h\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.591780 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.590613 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 13 14:46:30.591780 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.590620 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 13 14:46:30.591780 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.590893 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-6lkfj\"" Jul 13 14:46:30.591780 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.591238 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 13 14:46:30.594263 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.594239 2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 14:46:30.594612 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.594594 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.597202 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.597182 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 13 14:46:30.597360 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.597343 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 13 14:46:30.597555 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.597537 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 13 14:46:30.597619 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.597562 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 13 14:46:30.598342 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.597777 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ng9pb\"" Jul 13 14:46:30.598342 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.597961 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-d7nhj" Jul 13 14:46:30.601896 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.600780 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 13 14:46:30.601896 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.601083 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.601896 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.601303 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rb995\"" Jul 13 14:46:30.601896 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.601555 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 13 14:46:30.601896 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.601843 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 13 14:46:30.603837 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.603773 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 13 14:46:30.604027 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.603985 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hhkcf\"" Jul 13 14:46:30.604027 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.604007 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 13 14:46:30.604182 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.604042 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 14:46:30.604829 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.604368 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7v6qt" Jul 13 14:46:30.608624 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.606554 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 13 14:46:30.608624 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.607804 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nq9c" Jul 13 14:46:30.608785 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.608656 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cbl5z\"" Jul 13 14:46:30.608785 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.608722 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 13 14:46:30.609595 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.609560 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:46:30.609690 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:30.609663 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-km6w9" podUID="764568c4-93b4-4bf4-a627-a26e55610c81" Jul 13 14:46:30.610626 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.610278 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 13 14:46:30.610626 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.610288 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 13 14:46:30.610626 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.610502 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-86lq2\"" Jul 13 14:46:30.610895 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.610651 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 13 14:46:30.612315 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.612284 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qg9jx" Jul 13 14:46:30.616107 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.614900 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-96.ec2.internal" event={"ID":"4161d8f2b036cc0e991f7a67c69322ec","Type":"ContainerStarted","Data":"568d272332d9c36a4997c040a7254ecdb3ebb5c319735f84b3e18c0f7517b019"} Jul 13 14:46:30.616107 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.614957 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-96.ec2.internal" event={"ID":"1ad2abc2bf15f3fd836f68bef031f771","Type":"ContainerStarted","Data":"f4811e026cfe1b4c8af8cc1383edd352bf4460610443927baf3b72036415d72c"} Jul 13 14:46:30.616107 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.614985 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9hk94" Jul 13 14:46:30.616107 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:30.615056 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9hk94" podUID="30b24206-b58d-4409-b6e1-1dec743e754c" Jul 13 14:46:30.616107 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.615155 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 13 14:46:30.616107 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.615423 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-q7669\"" Jul 13 14:46:30.616107 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.615712 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 13 14:46:30.616505 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.616310 2568 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 13 14:46:30.646123 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.646090 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-g9gwb" Jul 13 14:46:30.657284 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.657259 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-g9gwb" Jul 13 14:46:30.690447 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.690408 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/36cb39be-505d-4a23-a395-4322f6301378-run\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.690447 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.690456 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822-konnectivity-ca\") pod \"konnectivity-agent-7v6qt\" (UID: \"5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822\") " pod="kube-system/konnectivity-agent-7v6qt" Jul 13 14:46:30.690746 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.690493 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-host-cni-netd\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.690746 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.690513 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-host-run-k8s-cni-cncf-io\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.690746 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.690537 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-multus-conf-dir\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.690746 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.690562 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6a62a407-ab2f-44ca-896d-a1e642c6667d-multus-daemon-config\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.690746 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.690653 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-host-cni-netd\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.690746 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.690685 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/36cb39be-505d-4a23-a395-4322f6301378-etc-modprobe-d\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.690746 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.690653 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-host-run-k8s-cni-cncf-io\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.690746 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.690739 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822-agent-certs\") pod \"konnectivity-agent-7v6qt\" (UID: \"5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822\") " pod="kube-system/konnectivity-agent-7v6qt" Jul 13 14:46:30.691225 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.690784 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqbsk\" (UniqueName: \"kubernetes.io/projected/191cfe54-82c1-4694-92fd-3b93bb1783fd-kube-api-access-pqbsk\") pod \"network-check-target-r82bx\" (UID: \"191cfe54-82c1-4694-92fd-3b93bb1783fd\") " pod="openshift-network-diagnostics/network-check-target-r82bx" Jul 13 14:46:30.691225 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.690816 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6a62a407-ab2f-44ca-896d-a1e642c6667d-cni-binary-copy\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.691225 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.690842 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/36cb39be-505d-4a23-a395-4322f6301378-lib-modules\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.691225 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.690868 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-hostroot\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.691225 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.690894 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/36cb39be-505d-4a23-a395-4322f6301378-sys\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.691225 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.690921 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fc45c5e7-30d9-4de3-ad6e-036706527893-etc-selinux\") pod \"aws-ebs-csi-driver-node-5nq9c\" (UID: \"fc45c5e7-30d9-4de3-ad6e-036706527893\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nq9c" Jul 13 14:46:30.691225 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.690946 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-host-cni-bin\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.691225 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.690969 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-multus-cni-dir\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.691225 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.690993 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwfz2\" (UniqueName: \"kubernetes.io/projected/959b52b7-090f-4cd0-95ab-68ed775d9093-kube-api-access-rwfz2\") pod \"iptables-alerter-d7nhj\" (UID: \"959b52b7-090f-4cd0-95ab-68ed775d9093\") " pod="openshift-network-operator/iptables-alerter-d7nhj" Jul 13 14:46:30.691225 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.691032 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/36cb39be-505d-4a23-a395-4322f6301378-etc-sysctl-conf\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.691225 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.691056 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30b24206-b58d-4409-b6e1-1dec743e754c-metrics-certs\") pod \"network-metrics-daemon-9hk94\" (UID: \"30b24206-b58d-4409-b6e1-1dec743e754c\") " pod="openshift-multus/network-metrics-daemon-9hk94" Jul 13 14:46:30.691225 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.691080 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fc45c5e7-30d9-4de3-ad6e-036706527893-socket-dir\") pod \"aws-ebs-csi-driver-node-5nq9c\" (UID: \"fc45c5e7-30d9-4de3-ad6e-036706527893\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nq9c" Jul 13 14:46:30.691225 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.691108 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fc45c5e7-30d9-4de3-ad6e-036706527893-sys-fs\") pod \"aws-ebs-csi-driver-node-5nq9c\" (UID: \"fc45c5e7-30d9-4de3-ad6e-036706527893\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nq9c" Jul 13 14:46:30.691225 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.691138 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-run-systemd\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.691953 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.691181 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/df5174e4-503e-475b-ab01-20dd9f45124d-serviceca\") pod \"node-ca-qfs4l\" (UID: \"df5174e4-503e-475b-ab01-20dd9f45124d\") " pod="openshift-image-registry/node-ca-qfs4l" Jul 13 14:46:30.691953 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.691210 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-multus-socket-dir-parent\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.691953 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.691236 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4cjl\" (UniqueName: \"kubernetes.io/projected/6a62a407-ab2f-44ca-896d-a1e642c6667d-kube-api-access-t4cjl\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.691953 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.691268 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/87cdb037-6b66-416c-95a7-08332490b5a3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qg9jx\" (UID: \"87cdb037-6b66-416c-95a7-08332490b5a3\") " pod="openshift-multus/multus-additional-cni-plugins-qg9jx" Jul 13 14:46:30.691953 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.691295 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36cb39be-505d-4a23-a395-4322f6301378-tmp\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.691953 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.691296 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-multus-cni-dir\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.691953 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.691325 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-host-run-ovn-kubernetes\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.691953 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.691364 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-system-cni-dir\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.691953 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.691393 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-cnibin\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.691953 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.691403 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-host-cni-bin\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.691953 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.691421 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5phv\" (UniqueName: \"kubernetes.io/projected/87cdb037-6b66-416c-95a7-08332490b5a3-kube-api-access-m5phv\") pod \"multus-additional-cni-plugins-qg9jx\" (UID: \"87cdb037-6b66-416c-95a7-08332490b5a3\") " pod="openshift-multus/multus-additional-cni-plugins-qg9jx" Jul 13 14:46:30.691953 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.691456 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-run-ovn\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.691953 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.691466 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-multus-socket-dir-parent\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.691953 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.691483 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5x7x\" (UniqueName: \"kubernetes.io/projected/d05cd493-5f17-4976-b2e5-404431d3389a-kube-api-access-h5x7x\") pod \"node-resolver-xz7wd\" (UID: \"d05cd493-5f17-4976-b2e5-404431d3389a\") " pod="openshift-dns/node-resolver-xz7wd" Jul 13 14:46:30.692678 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.691512 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87cdb037-6b66-416c-95a7-08332490b5a3-system-cni-dir\") pod \"multus-additional-cni-plugins-qg9jx\" (UID: \"87cdb037-6b66-416c-95a7-08332490b5a3\") " pod="openshift-multus/multus-additional-cni-plugins-qg9jx" Jul 13 14:46:30.692678 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.691535 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6a62a407-ab2f-44ca-896d-a1e642c6667d-cni-binary-copy\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.692678 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.691559 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-run-systemd\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.692678 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.691539 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-env-overrides\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.692678 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.691620 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-host-var-lib-kubelet\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.692678 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.691675 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-host-run-ovn-kubernetes\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.692678 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.691679 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-host-run-multus-certs\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.692678 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.691747 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-cnibin\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.692678 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.691795 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-system-cni-dir\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.692678 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.691826 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-etc-kubernetes\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.692678 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.691852 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/87cdb037-6b66-416c-95a7-08332490b5a3-os-release\") pod \"multus-additional-cni-plugins-qg9jx\" (UID: \"87cdb037-6b66-416c-95a7-08332490b5a3\") " pod="openshift-multus/multus-additional-cni-plugins-qg9jx" Jul 13 14:46:30.692678 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.691879 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/36cb39be-505d-4a23-a395-4322f6301378-etc-tuned\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.692678 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.691912 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fc45c5e7-30d9-4de3-ad6e-036706527893-device-dir\") pod \"aws-ebs-csi-driver-node-5nq9c\" (UID: \"fc45c5e7-30d9-4de3-ad6e-036706527893\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nq9c" Jul 13 14:46:30.692678 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.691940 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-host-kubelet\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.693313 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.691964 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-var-lib-openvswitch\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.693313 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.691993 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/87cdb037-6b66-416c-95a7-08332490b5a3-cni-binary-copy\") pod \"multus-additional-cni-plugins-qg9jx\" (UID: \"87cdb037-6b66-416c-95a7-08332490b5a3\") " pod="openshift-multus/multus-additional-cni-plugins-qg9jx" Jul 13 14:46:30.693313 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692022 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/959b52b7-090f-4cd0-95ab-68ed775d9093-iptables-alerter-script\") pod \"iptables-alerter-d7nhj\" (UID: \"959b52b7-090f-4cd0-95ab-68ed775d9093\") " pod="openshift-network-operator/iptables-alerter-d7nhj" Jul 13 14:46:30.693313 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692048 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/36cb39be-505d-4a23-a395-4322f6301378-etc-sysconfig\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.693313 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692107 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/df5174e4-503e-475b-ab01-20dd9f45124d-host\") pod \"node-ca-qfs4l\" (UID: \"df5174e4-503e-475b-ab01-20dd9f45124d\") " pod="openshift-image-registry/node-ca-qfs4l" Jul 13 14:46:30.693313 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692136 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.693313 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692149 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/df5174e4-503e-475b-ab01-20dd9f45124d-serviceca\") pod \"node-ca-qfs4l\" (UID: \"df5174e4-503e-475b-ab01-20dd9f45124d\") " pod="openshift-image-registry/node-ca-qfs4l" Jul 13 14:46:30.693313 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692166 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-ovnkube-config\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.693313 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692191 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-env-overrides\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.693313 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692199 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d05cd493-5f17-4976-b2e5-404431d3389a-tmp-dir\") pod \"node-resolver-xz7wd\" (UID: \"d05cd493-5f17-4976-b2e5-404431d3389a\") " pod="openshift-dns/node-resolver-xz7wd" Jul 13 14:46:30.693313 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692249 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-run-ovn\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.693313 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692283 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/959b52b7-090f-4cd0-95ab-68ed775d9093-host-slash\") pod \"iptables-alerter-d7nhj\" (UID: \"959b52b7-090f-4cd0-95ab-68ed775d9093\") " pod="openshift-network-operator/iptables-alerter-d7nhj" Jul 13 14:46:30.693313 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692311 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/36cb39be-505d-4a23-a395-4322f6301378-var-lib-kubelet\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.693313 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692338 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fc45c5e7-30d9-4de3-ad6e-036706527893-registration-dir\") pod \"aws-ebs-csi-driver-node-5nq9c\" (UID: \"fc45c5e7-30d9-4de3-ad6e-036706527893\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nq9c" Jul 13 14:46:30.694017 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692371 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-host-slash\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.694017 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692398 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-host-run-netns\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.694017 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692433 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-run-openvswitch\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.694017 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692478 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d05cd493-5f17-4976-b2e5-404431d3389a-tmp-dir\") pod \"node-resolver-xz7wd\" (UID: \"d05cd493-5f17-4976-b2e5-404431d3389a\") " pod="openshift-dns/node-resolver-xz7wd" Jul 13 14:46:30.694017 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692498 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-ovnkube-script-lib\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.694017 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692535 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/87cdb037-6b66-416c-95a7-08332490b5a3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qg9jx\" (UID: \"87cdb037-6b66-416c-95a7-08332490b5a3\") " pod="openshift-multus/multus-additional-cni-plugins-qg9jx" Jul 13 14:46:30.694017 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692566 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/36cb39be-505d-4a23-a395-4322f6301378-etc-kubernetes\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.694017 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692577 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-host-run-netns\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.694017 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692595 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncdd2\" (UniqueName: \"kubernetes.io/projected/fc45c5e7-30d9-4de3-ad6e-036706527893-kube-api-access-ncdd2\") pod \"aws-ebs-csi-driver-node-5nq9c\" (UID: \"fc45c5e7-30d9-4de3-ad6e-036706527893\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nq9c" Jul 13 14:46:30.694017 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692610 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-host-slash\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.694017 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692624 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d05cd493-5f17-4976-b2e5-404431d3389a-hosts-file\") pod \"node-resolver-xz7wd\" (UID: \"d05cd493-5f17-4976-b2e5-404431d3389a\") " pod="openshift-dns/node-resolver-xz7wd" Jul 13 14:46:30.694017 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692684 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/36cb39be-505d-4a23-a395-4322f6301378-host\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.694017 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692696 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-var-lib-openvswitch\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.694017 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692725 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/df5174e4-503e-475b-ab01-20dd9f45124d-host\") pod \"node-ca-qfs4l\" (UID: \"df5174e4-503e-475b-ab01-20dd9f45124d\") " pod="openshift-image-registry/node-ca-qfs4l" Jul 13 14:46:30.694613 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692735 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.694613 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692742 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d05cd493-5f17-4976-b2e5-404431d3389a-hosts-file\") pod \"node-resolver-xz7wd\" (UID: \"d05cd493-5f17-4976-b2e5-404431d3389a\") " pod="openshift-dns/node-resolver-xz7wd" Jul 13 14:46:30.694613 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692684 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-ovnkube-config\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.694613 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692746 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/764568c4-93b4-4bf4-a627-a26e55610c81-kubelet-config\") pod \"global-pull-secret-syncer-km6w9\" (UID: \"764568c4-93b4-4bf4-a627-a26e55610c81\") " pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:46:30.694613 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692769 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-run-openvswitch\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.694613 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692818 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-host-kubelet\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.694613 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692858 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/764568c4-93b4-4bf4-a627-a26e55610c81-dbus\") pod \"global-pull-secret-syncer-km6w9\" (UID: \"764568c4-93b4-4bf4-a627-a26e55610c81\") " pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:46:30.694613 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692885 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/764568c4-93b4-4bf4-a627-a26e55610c81-original-pull-secret\") pod \"global-pull-secret-syncer-km6w9\" (UID: \"764568c4-93b4-4bf4-a627-a26e55610c81\") " pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:46:30.694613 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692914 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/87cdb037-6b66-416c-95a7-08332490b5a3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qg9jx\" (UID: \"87cdb037-6b66-416c-95a7-08332490b5a3\") " pod="openshift-multus/multus-additional-cni-plugins-qg9jx" Jul 13 14:46:30.694613 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692959 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-node-log\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.694613 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.692994 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-log-socket\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.694613 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.693043 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-node-log\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.694613 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.693055 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zt9h\" (UniqueName: \"kubernetes.io/projected/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-kube-api-access-7zt9h\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.694613 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.693063 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-ovnkube-script-lib\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.695198 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.693082 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-log-socket\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.695198 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.693127 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tj98\" (UniqueName: \"kubernetes.io/projected/df5174e4-503e-475b-ab01-20dd9f45124d-kube-api-access-8tj98\") pod \"node-ca-qfs4l\" (UID: \"df5174e4-503e-475b-ab01-20dd9f45124d\") " pod="openshift-image-registry/node-ca-qfs4l" Jul 13 14:46:30.695198 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.693416 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-os-release\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.695198 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.693483 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/36cb39be-505d-4a23-a395-4322f6301378-etc-systemd\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.695198 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.693529 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85wr7\" (UniqueName: \"kubernetes.io/projected/36cb39be-505d-4a23-a395-4322f6301378-kube-api-access-85wr7\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.695198 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.693556 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/87cdb037-6b66-416c-95a7-08332490b5a3-cnibin\") pod \"multus-additional-cni-plugins-qg9jx\" (UID: \"87cdb037-6b66-416c-95a7-08332490b5a3\") " pod="openshift-multus/multus-additional-cni-plugins-qg9jx" Jul 13 14:46:30.695198 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.693557 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-os-release\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.695198 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.693590 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-systemd-units\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.695198 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.693640 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-etc-openvswitch\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.695198 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.693676 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-ovn-node-metrics-cert\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.695198 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.693732 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-etc-openvswitch\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.695198 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.693769 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-host-run-netns\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.695198 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.693771 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-systemd-units\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.695198 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.693865 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-host-var-lib-cni-bin\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.695924 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.693899 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pv67n\" (UniqueName: \"kubernetes.io/projected/30b24206-b58d-4409-b6e1-1dec743e754c-kube-api-access-pv67n\") pod \"network-metrics-daemon-9hk94\" (UID: \"30b24206-b58d-4409-b6e1-1dec743e754c\") " pod="openshift-multus/network-metrics-daemon-9hk94" Jul 13 14:46:30.695924 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.693946 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fc45c5e7-30d9-4de3-ad6e-036706527893-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5nq9c\" (UID: \"fc45c5e7-30d9-4de3-ad6e-036706527893\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nq9c" Jul 13 14:46:30.695924 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.693972 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-host-var-lib-cni-multus\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.695924 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.693997 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/36cb39be-505d-4a23-a395-4322f6301378-etc-sysctl-d\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.695924 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.694111 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-host-run-netns\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.695924 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.694110 2568 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 13 14:46:30.698522 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.698491 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-ovn-node-metrics-cert\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.699156 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:30.699133 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 14:46:30.699259 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:30.699163 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 14:46:30.699259 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:30.699184 2568 projected.go:196] Error preparing data for projected volume kube-api-access-pqbsk for pod openshift-network-diagnostics/network-check-target-r82bx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 14:46:30.699405 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:30.699271 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/191cfe54-82c1-4694-92fd-3b93bb1783fd-kube-api-access-pqbsk podName:191cfe54-82c1-4694-92fd-3b93bb1783fd nodeName:}" failed. No retries permitted until 2026-07-13 14:46:31.199248055 +0000 UTC m=+3.439525903 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-pqbsk" (UniqueName: "kubernetes.io/projected/191cfe54-82c1-4694-92fd-3b93bb1783fd-kube-api-access-pqbsk") pod "network-check-target-r82bx" (UID: "191cfe54-82c1-4694-92fd-3b93bb1783fd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 14:46:30.708116 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.708076 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zt9h\" (UniqueName: \"kubernetes.io/projected/0bd25f5c-138c-4c60-9d20-525c8b12ebb1-kube-api-access-7zt9h\") pod \"ovnkube-node-xgqvx\" (UID: \"0bd25f5c-138c-4c60-9d20-525c8b12ebb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.708253 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.708186 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5x7x\" (UniqueName: \"kubernetes.io/projected/d05cd493-5f17-4976-b2e5-404431d3389a-kube-api-access-h5x7x\") pod \"node-resolver-xz7wd\" (UID: \"d05cd493-5f17-4976-b2e5-404431d3389a\") " pod="openshift-dns/node-resolver-xz7wd" Jul 13 14:46:30.712064 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.712040 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tj98\" (UniqueName: \"kubernetes.io/projected/df5174e4-503e-475b-ab01-20dd9f45124d-kube-api-access-8tj98\") pod \"node-ca-qfs4l\" (UID: \"df5174e4-503e-475b-ab01-20dd9f45124d\") " pod="openshift-image-registry/node-ca-qfs4l" Jul 13 14:46:30.794329 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.794279 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-host-var-lib-cni-bin\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.794329 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.794331 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv67n\" (UniqueName: \"kubernetes.io/projected/30b24206-b58d-4409-b6e1-1dec743e754c-kube-api-access-pv67n\") pod \"network-metrics-daemon-9hk94\" (UID: \"30b24206-b58d-4409-b6e1-1dec743e754c\") " pod="openshift-multus/network-metrics-daemon-9hk94" Jul 13 14:46:30.794565 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.794359 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fc45c5e7-30d9-4de3-ad6e-036706527893-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5nq9c\" (UID: \"fc45c5e7-30d9-4de3-ad6e-036706527893\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nq9c" Jul 13 14:46:30.794565 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.794384 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-host-var-lib-cni-multus\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.794565 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.794468 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fc45c5e7-30d9-4de3-ad6e-036706527893-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5nq9c\" (UID: \"fc45c5e7-30d9-4de3-ad6e-036706527893\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nq9c" Jul 13 14:46:30.794565 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.794501 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-host-var-lib-cni-bin\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.794815 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.794568 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/36cb39be-505d-4a23-a395-4322f6301378-etc-sysctl-d\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.794815 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.794612 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-host-var-lib-cni-multus\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.794815 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.794622 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/36cb39be-505d-4a23-a395-4322f6301378-run\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.794815 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.794647 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822-konnectivity-ca\") pod \"konnectivity-agent-7v6qt\" (UID: \"5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822\") " pod="kube-system/konnectivity-agent-7v6qt" Jul 13 14:46:30.794815 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.794672 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-multus-conf-dir\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.794815 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.794694 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6a62a407-ab2f-44ca-896d-a1e642c6667d-multus-daemon-config\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.794815 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.794723 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/36cb39be-505d-4a23-a395-4322f6301378-etc-modprobe-d\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.794815 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.794741 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/36cb39be-505d-4a23-a395-4322f6301378-etc-sysctl-d\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.794815 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.794749 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822-agent-certs\") pod \"konnectivity-agent-7v6qt\" (UID: \"5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822\") " pod="kube-system/konnectivity-agent-7v6qt" Jul 13 14:46:30.794815 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.794740 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/36cb39be-505d-4a23-a395-4322f6301378-run\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.794815 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.794806 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/36cb39be-505d-4a23-a395-4322f6301378-lib-modules\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.795355 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.794833 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-hostroot\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.795355 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.794840 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/36cb39be-505d-4a23-a395-4322f6301378-etc-modprobe-d\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.795355 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.794857 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/36cb39be-505d-4a23-a395-4322f6301378-sys\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.795355 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.794884 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fc45c5e7-30d9-4de3-ad6e-036706527893-etc-selinux\") pod \"aws-ebs-csi-driver-node-5nq9c\" (UID: \"fc45c5e7-30d9-4de3-ad6e-036706527893\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nq9c" Jul 13 14:46:30.795355 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.794912 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwfz2\" (UniqueName: \"kubernetes.io/projected/959b52b7-090f-4cd0-95ab-68ed775d9093-kube-api-access-rwfz2\") pod \"iptables-alerter-d7nhj\" (UID: \"959b52b7-090f-4cd0-95ab-68ed775d9093\") " pod="openshift-network-operator/iptables-alerter-d7nhj" Jul 13 14:46:30.795355 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.794937 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/36cb39be-505d-4a23-a395-4322f6301378-etc-sysctl-conf\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.795355 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.794961 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30b24206-b58d-4409-b6e1-1dec743e754c-metrics-certs\") pod \"network-metrics-daemon-9hk94\" (UID: \"30b24206-b58d-4409-b6e1-1dec743e754c\") " pod="openshift-multus/network-metrics-daemon-9hk94" Jul 13 14:46:30.795355 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.794985 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fc45c5e7-30d9-4de3-ad6e-036706527893-socket-dir\") pod \"aws-ebs-csi-driver-node-5nq9c\" (UID: \"fc45c5e7-30d9-4de3-ad6e-036706527893\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nq9c" Jul 13 14:46:30.795355 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795023 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fc45c5e7-30d9-4de3-ad6e-036706527893-sys-fs\") pod \"aws-ebs-csi-driver-node-5nq9c\" (UID: \"fc45c5e7-30d9-4de3-ad6e-036706527893\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nq9c" Jul 13 14:46:30.795355 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795057 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4cjl\" (UniqueName: \"kubernetes.io/projected/6a62a407-ab2f-44ca-896d-a1e642c6667d-kube-api-access-t4cjl\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.795355 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795084 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/87cdb037-6b66-416c-95a7-08332490b5a3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qg9jx\" (UID: \"87cdb037-6b66-416c-95a7-08332490b5a3\") " pod="openshift-multus/multus-additional-cni-plugins-qg9jx" Jul 13 14:46:30.795355 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795112 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36cb39be-505d-4a23-a395-4322f6301378-tmp\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.795355 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795140 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5phv\" (UniqueName: \"kubernetes.io/projected/87cdb037-6b66-416c-95a7-08332490b5a3-kube-api-access-m5phv\") pod \"multus-additional-cni-plugins-qg9jx\" (UID: \"87cdb037-6b66-416c-95a7-08332490b5a3\") " pod="openshift-multus/multus-additional-cni-plugins-qg9jx" Jul 13 14:46:30.795355 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795168 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87cdb037-6b66-416c-95a7-08332490b5a3-system-cni-dir\") pod \"multus-additional-cni-plugins-qg9jx\" (UID: \"87cdb037-6b66-416c-95a7-08332490b5a3\") " pod="openshift-multus/multus-additional-cni-plugins-qg9jx" Jul 13 14:46:30.796040 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795198 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-host-var-lib-kubelet\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.796040 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795224 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-host-run-multus-certs\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.796040 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795239 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822-konnectivity-ca\") pod \"konnectivity-agent-7v6qt\" (UID: \"5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822\") " pod="kube-system/konnectivity-agent-7v6qt" Jul 13 14:46:30.796040 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795251 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-etc-kubernetes\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.796040 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795318 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/87cdb037-6b66-416c-95a7-08332490b5a3-os-release\") pod \"multus-additional-cni-plugins-qg9jx\" (UID: \"87cdb037-6b66-416c-95a7-08332490b5a3\") " pod="openshift-multus/multus-additional-cni-plugins-qg9jx" Jul 13 14:46:30.796040 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795341 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fc45c5e7-30d9-4de3-ad6e-036706527893-socket-dir\") pod \"aws-ebs-csi-driver-node-5nq9c\" (UID: \"fc45c5e7-30d9-4de3-ad6e-036706527893\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nq9c" Jul 13 14:46:30.796040 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795345 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/36cb39be-505d-4a23-a395-4322f6301378-etc-tuned\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.796040 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795381 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fc45c5e7-30d9-4de3-ad6e-036706527893-device-dir\") pod \"aws-ebs-csi-driver-node-5nq9c\" (UID: \"fc45c5e7-30d9-4de3-ad6e-036706527893\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nq9c" Jul 13 14:46:30.796040 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795408 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-multus-conf-dir\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.796040 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795413 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/87cdb037-6b66-416c-95a7-08332490b5a3-cni-binary-copy\") pod \"multus-additional-cni-plugins-qg9jx\" (UID: \"87cdb037-6b66-416c-95a7-08332490b5a3\") " pod="openshift-multus/multus-additional-cni-plugins-qg9jx" Jul 13 14:46:30.796040 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795459 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/959b52b7-090f-4cd0-95ab-68ed775d9093-iptables-alerter-script\") pod \"iptables-alerter-d7nhj\" (UID: \"959b52b7-090f-4cd0-95ab-68ed775d9093\") " pod="openshift-network-operator/iptables-alerter-d7nhj" Jul 13 14:46:30.796040 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795491 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/36cb39be-505d-4a23-a395-4322f6301378-etc-sysconfig\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.796040 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795548 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/959b52b7-090f-4cd0-95ab-68ed775d9093-host-slash\") pod \"iptables-alerter-d7nhj\" (UID: \"959b52b7-090f-4cd0-95ab-68ed775d9093\") " pod="openshift-network-operator/iptables-alerter-d7nhj" Jul 13 14:46:30.796040 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795558 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fc45c5e7-30d9-4de3-ad6e-036706527893-etc-selinux\") pod \"aws-ebs-csi-driver-node-5nq9c\" (UID: \"fc45c5e7-30d9-4de3-ad6e-036706527893\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nq9c" Jul 13 14:46:30.796765 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795595 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/36cb39be-505d-4a23-a395-4322f6301378-var-lib-kubelet\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.796765 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795627 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fc45c5e7-30d9-4de3-ad6e-036706527893-registration-dir\") pod \"aws-ebs-csi-driver-node-5nq9c\" (UID: \"fc45c5e7-30d9-4de3-ad6e-036706527893\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nq9c" Jul 13 14:46:30.796765 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795661 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/87cdb037-6b66-416c-95a7-08332490b5a3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qg9jx\" (UID: \"87cdb037-6b66-416c-95a7-08332490b5a3\") " pod="openshift-multus/multus-additional-cni-plugins-qg9jx" Jul 13 14:46:30.796765 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795714 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/36cb39be-505d-4a23-a395-4322f6301378-lib-modules\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.796765 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795726 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/36cb39be-505d-4a23-a395-4322f6301378-etc-kubernetes\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.796765 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795755 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncdd2\" (UniqueName: \"kubernetes.io/projected/fc45c5e7-30d9-4de3-ad6e-036706527893-kube-api-access-ncdd2\") pod \"aws-ebs-csi-driver-node-5nq9c\" (UID: \"fc45c5e7-30d9-4de3-ad6e-036706527893\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nq9c" Jul 13 14:46:30.796765 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795773 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-hostroot\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.796765 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795796 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/36cb39be-505d-4a23-a395-4322f6301378-host\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.796765 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795824 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/764568c4-93b4-4bf4-a627-a26e55610c81-kubelet-config\") pod \"global-pull-secret-syncer-km6w9\" (UID: \"764568c4-93b4-4bf4-a627-a26e55610c81\") " pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:46:30.796765 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795855 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/764568c4-93b4-4bf4-a627-a26e55610c81-dbus\") pod \"global-pull-secret-syncer-km6w9\" (UID: \"764568c4-93b4-4bf4-a627-a26e55610c81\") " pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:46:30.796765 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795888 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/764568c4-93b4-4bf4-a627-a26e55610c81-original-pull-secret\") pod \"global-pull-secret-syncer-km6w9\" (UID: \"764568c4-93b4-4bf4-a627-a26e55610c81\") " pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:46:30.796765 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795918 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/87cdb037-6b66-416c-95a7-08332490b5a3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qg9jx\" (UID: \"87cdb037-6b66-416c-95a7-08332490b5a3\") " pod="openshift-multus/multus-additional-cni-plugins-qg9jx" Jul 13 14:46:30.796765 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795955 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/36cb39be-505d-4a23-a395-4322f6301378-etc-systemd\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.796765 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795962 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-host-run-multus-certs\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.797549 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795979 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85wr7\" (UniqueName: \"kubernetes.io/projected/36cb39be-505d-4a23-a395-4322f6301378-kube-api-access-85wr7\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.797549 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.796026 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/87cdb037-6b66-416c-95a7-08332490b5a3-cnibin\") pod \"multus-additional-cni-plugins-qg9jx\" (UID: \"87cdb037-6b66-416c-95a7-08332490b5a3\") " pod="openshift-multus/multus-additional-cni-plugins-qg9jx" Jul 13 14:46:30.797549 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.796042 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6a62a407-ab2f-44ca-896d-a1e642c6667d-multus-daemon-config\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.797549 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.796111 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fc45c5e7-30d9-4de3-ad6e-036706527893-sys-fs\") pod \"aws-ebs-csi-driver-node-5nq9c\" (UID: \"fc45c5e7-30d9-4de3-ad6e-036706527893\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nq9c" Jul 13 14:46:30.797549 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.796161 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-host-var-lib-kubelet\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.797549 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.796161 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87cdb037-6b66-416c-95a7-08332490b5a3-system-cni-dir\") pod \"multus-additional-cni-plugins-qg9jx\" (UID: \"87cdb037-6b66-416c-95a7-08332490b5a3\") " pod="openshift-multus/multus-additional-cni-plugins-qg9jx" Jul 13 14:46:30.797549 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.796112 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/87cdb037-6b66-416c-95a7-08332490b5a3-cnibin\") pod \"multus-additional-cni-plugins-qg9jx\" (UID: \"87cdb037-6b66-416c-95a7-08332490b5a3\") " pod="openshift-multus/multus-additional-cni-plugins-qg9jx" Jul 13 14:46:30.797549 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.796239 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a62a407-ab2f-44ca-896d-a1e642c6667d-etc-kubernetes\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.797549 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.796306 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/87cdb037-6b66-416c-95a7-08332490b5a3-os-release\") pod \"multus-additional-cni-plugins-qg9jx\" (UID: \"87cdb037-6b66-416c-95a7-08332490b5a3\") " pod="openshift-multus/multus-additional-cni-plugins-qg9jx" Jul 13 14:46:30.797549 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.796446 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/36cb39be-505d-4a23-a395-4322f6301378-etc-sysctl-conf\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.797549 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.796549 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/87cdb037-6b66-416c-95a7-08332490b5a3-cni-binary-copy\") pod \"multus-additional-cni-plugins-qg9jx\" (UID: \"87cdb037-6b66-416c-95a7-08332490b5a3\") " pod="openshift-multus/multus-additional-cni-plugins-qg9jx" Jul 13 14:46:30.797549 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.796641 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/87cdb037-6b66-416c-95a7-08332490b5a3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qg9jx\" (UID: \"87cdb037-6b66-416c-95a7-08332490b5a3\") " pod="openshift-multus/multus-additional-cni-plugins-qg9jx" Jul 13 14:46:30.797549 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:30.796666 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 14:46:30.797549 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.796721 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/36cb39be-505d-4a23-a395-4322f6301378-var-lib-kubelet\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.798257 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:30.796738 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30b24206-b58d-4409-b6e1-1dec743e754c-metrics-certs podName:30b24206-b58d-4409-b6e1-1dec743e754c nodeName:}" failed. No retries permitted until 2026-07-13 14:46:31.296718885 +0000 UTC m=+3.536996730 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30b24206-b58d-4409-b6e1-1dec743e754c-metrics-certs") pod "network-metrics-daemon-9hk94" (UID: "30b24206-b58d-4409-b6e1-1dec743e754c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 14:46:30.798257 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.796761 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/36cb39be-505d-4a23-a395-4322f6301378-etc-sysconfig\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.798257 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.796766 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/959b52b7-090f-4cd0-95ab-68ed775d9093-host-slash\") pod \"iptables-alerter-d7nhj\" (UID: \"959b52b7-090f-4cd0-95ab-68ed775d9093\") " pod="openshift-network-operator/iptables-alerter-d7nhj" Jul 13 14:46:30.798257 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.796792 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fc45c5e7-30d9-4de3-ad6e-036706527893-device-dir\") pod \"aws-ebs-csi-driver-node-5nq9c\" (UID: \"fc45c5e7-30d9-4de3-ad6e-036706527893\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nq9c" Jul 13 14:46:30.798257 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.796829 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fc45c5e7-30d9-4de3-ad6e-036706527893-registration-dir\") pod \"aws-ebs-csi-driver-node-5nq9c\" (UID: \"fc45c5e7-30d9-4de3-ad6e-036706527893\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nq9c" Jul 13 14:46:30.798257 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.796829 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/959b52b7-090f-4cd0-95ab-68ed775d9093-iptables-alerter-script\") pod \"iptables-alerter-d7nhj\" (UID: \"959b52b7-090f-4cd0-95ab-68ed775d9093\") " pod="openshift-network-operator/iptables-alerter-d7nhj" Jul 13 14:46:30.798257 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.796852 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/764568c4-93b4-4bf4-a627-a26e55610c81-dbus\") pod \"global-pull-secret-syncer-km6w9\" (UID: \"764568c4-93b4-4bf4-a627-a26e55610c81\") " pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:46:30.798257 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.796897 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/36cb39be-505d-4a23-a395-4322f6301378-etc-systemd\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.798257 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.795825 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/36cb39be-505d-4a23-a395-4322f6301378-sys\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.798257 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.796903 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/36cb39be-505d-4a23-a395-4322f6301378-host\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.798257 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.796930 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/87cdb037-6b66-416c-95a7-08332490b5a3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qg9jx\" (UID: \"87cdb037-6b66-416c-95a7-08332490b5a3\") " pod="openshift-multus/multus-additional-cni-plugins-qg9jx" Jul 13 14:46:30.798257 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.796952 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/764568c4-93b4-4bf4-a627-a26e55610c81-kubelet-config\") pod \"global-pull-secret-syncer-km6w9\" (UID: \"764568c4-93b4-4bf4-a627-a26e55610c81\") " pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:46:30.798257 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.796985 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/36cb39be-505d-4a23-a395-4322f6301378-etc-kubernetes\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.798257 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:30.797000 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 14:46:30.798995 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:30.797069 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/764568c4-93b4-4bf4-a627-a26e55610c81-original-pull-secret podName:764568c4-93b4-4bf4-a627-a26e55610c81 nodeName:}" failed. No retries permitted until 2026-07-13 14:46:31.297049891 +0000 UTC m=+3.537327739 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/764568c4-93b4-4bf4-a627-a26e55610c81-original-pull-secret") pod "global-pull-secret-syncer-km6w9" (UID: "764568c4-93b4-4bf4-a627-a26e55610c81") : object "kube-system"/"original-pull-secret" not registered Jul 13 14:46:30.798995 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.797304 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/87cdb037-6b66-416c-95a7-08332490b5a3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qg9jx\" (UID: \"87cdb037-6b66-416c-95a7-08332490b5a3\") " pod="openshift-multus/multus-additional-cni-plugins-qg9jx" Jul 13 14:46:30.798995 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.798079 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822-agent-certs\") pod \"konnectivity-agent-7v6qt\" (UID: \"5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822\") " pod="kube-system/konnectivity-agent-7v6qt" Jul 13 14:46:30.799636 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.799586 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36cb39be-505d-4a23-a395-4322f6301378-tmp\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.799842 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.799816 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/36cb39be-505d-4a23-a395-4322f6301378-etc-tuned\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.812619 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.812076 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85wr7\" (UniqueName: \"kubernetes.io/projected/36cb39be-505d-4a23-a395-4322f6301378-kube-api-access-85wr7\") pod \"tuned-5rmzg\" (UID: \"36cb39be-505d-4a23-a395-4322f6301378\") " pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.812619 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.812390 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4cjl\" (UniqueName: \"kubernetes.io/projected/6a62a407-ab2f-44ca-896d-a1e642c6667d-kube-api-access-t4cjl\") pod \"multus-k8ltk\" (UID: \"6a62a407-ab2f-44ca-896d-a1e642c6667d\") " pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.812619 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.812590 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwfz2\" (UniqueName: \"kubernetes.io/projected/959b52b7-090f-4cd0-95ab-68ed775d9093-kube-api-access-rwfz2\") pod \"iptables-alerter-d7nhj\" (UID: \"959b52b7-090f-4cd0-95ab-68ed775d9093\") " pod="openshift-network-operator/iptables-alerter-d7nhj" Jul 13 14:46:30.814086 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.814058 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pv67n\" (UniqueName: \"kubernetes.io/projected/30b24206-b58d-4409-b6e1-1dec743e754c-kube-api-access-pv67n\") pod \"network-metrics-daemon-9hk94\" (UID: \"30b24206-b58d-4409-b6e1-1dec743e754c\") " pod="openshift-multus/network-metrics-daemon-9hk94" Jul 13 14:46:30.816207 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.816172 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5phv\" (UniqueName: \"kubernetes.io/projected/87cdb037-6b66-416c-95a7-08332490b5a3-kube-api-access-m5phv\") pod \"multus-additional-cni-plugins-qg9jx\" (UID: \"87cdb037-6b66-416c-95a7-08332490b5a3\") " pod="openshift-multus/multus-additional-cni-plugins-qg9jx" Jul 13 14:46:30.819098 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.819053 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncdd2\" (UniqueName: \"kubernetes.io/projected/fc45c5e7-30d9-4de3-ad6e-036706527893-kube-api-access-ncdd2\") pod \"aws-ebs-csi-driver-node-5nq9c\" (UID: \"fc45c5e7-30d9-4de3-ad6e-036706527893\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nq9c" Jul 13 14:46:30.895954 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.895854 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:30.903817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.903778 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xz7wd" Jul 13 14:46:30.915297 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.915255 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qfs4l" Jul 13 14:46:30.922750 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.922718 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k8ltk" Jul 13 14:46:30.933857 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.933824 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-d7nhj" Jul 13 14:46:30.940915 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.940887 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" Jul 13 14:46:30.949917 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.949890 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7v6qt" Jul 13 14:46:30.958922 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.958897 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nq9c" Jul 13 14:46:30.966945 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:30.966920 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qg9jx" Jul 13 14:46:31.200357 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:31.200265 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqbsk\" (UniqueName: \"kubernetes.io/projected/191cfe54-82c1-4694-92fd-3b93bb1783fd-kube-api-access-pqbsk\") pod \"network-check-target-r82bx\" (UID: \"191cfe54-82c1-4694-92fd-3b93bb1783fd\") " pod="openshift-network-diagnostics/network-check-target-r82bx" Jul 13 14:46:31.201012 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:31.200465 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 14:46:31.201012 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:31.200499 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 14:46:31.201012 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:31.200511 2568 projected.go:196] Error preparing data for projected volume kube-api-access-pqbsk for pod openshift-network-diagnostics/network-check-target-r82bx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 14:46:31.201012 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:31.200583 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/191cfe54-82c1-4694-92fd-3b93bb1783fd-kube-api-access-pqbsk podName:191cfe54-82c1-4694-92fd-3b93bb1783fd nodeName:}" failed. No retries permitted until 2026-07-13 14:46:32.200565506 +0000 UTC m=+4.440843343 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-pqbsk" (UniqueName: "kubernetes.io/projected/191cfe54-82c1-4694-92fd-3b93bb1783fd-kube-api-access-pqbsk") pod "network-check-target-r82bx" (UID: "191cfe54-82c1-4694-92fd-3b93bb1783fd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 14:46:31.301770 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:31.301728 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30b24206-b58d-4409-b6e1-1dec743e754c-metrics-certs\") pod \"network-metrics-daemon-9hk94\" (UID: \"30b24206-b58d-4409-b6e1-1dec743e754c\") " pod="openshift-multus/network-metrics-daemon-9hk94" Jul 13 14:46:31.301963 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:31.301810 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/764568c4-93b4-4bf4-a627-a26e55610c81-original-pull-secret\") pod \"global-pull-secret-syncer-km6w9\" (UID: \"764568c4-93b4-4bf4-a627-a26e55610c81\") " pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:46:31.301963 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:31.301898 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 14:46:31.301963 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:31.301930 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 14:46:31.302108 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:31.301991 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/764568c4-93b4-4bf4-a627-a26e55610c81-original-pull-secret podName:764568c4-93b4-4bf4-a627-a26e55610c81 nodeName:}" failed. No retries permitted until 2026-07-13 14:46:32.301974681 +0000 UTC m=+4.542252508 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/764568c4-93b4-4bf4-a627-a26e55610c81-original-pull-secret") pod "global-pull-secret-syncer-km6w9" (UID: "764568c4-93b4-4bf4-a627-a26e55610c81") : object "kube-system"/"original-pull-secret" not registered Jul 13 14:46:31.302108 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:31.302020 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30b24206-b58d-4409-b6e1-1dec743e754c-metrics-certs podName:30b24206-b58d-4409-b6e1-1dec743e754c nodeName:}" failed. No retries permitted until 2026-07-13 14:46:32.302011259 +0000 UTC m=+4.542289089 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30b24206-b58d-4409-b6e1-1dec743e754c-metrics-certs") pod "network-metrics-daemon-9hk94" (UID: "30b24206-b58d-4409-b6e1-1dec743e754c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 14:46:31.661285 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:31.661243 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 14:41:31 +0000 UTC" deadline="2028-01-16 14:40:37.595862597 +0000 UTC" Jul 13 14:46:31.661285 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:31.661282 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13247h54m5.934585164s" Jul 13 14:46:32.030937 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:32.030899 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d5298a8_b2b8_4f8f_80ea_e4e66ba7c822.slice/crio-af33cf31976020c73530594a701a439e01b6e46cf8a38c688162496f02db41cb WatchSource:0}: Error finding container af33cf31976020c73530594a701a439e01b6e46cf8a38c688162496f02db41cb: Status 404 returned error can't find the container with id af33cf31976020c73530594a701a439e01b6e46cf8a38c688162496f02db41cb Jul 13 14:46:32.032628 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:32.032599 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd05cd493_5f17_4976_b2e5_404431d3389a.slice/crio-a564bb895c3353cd5e40f3154d719920a56994d93bfe9c0f5d1c4657995be0e4 WatchSource:0}: Error finding container a564bb895c3353cd5e40f3154d719920a56994d93bfe9c0f5d1c4657995be0e4: Status 404 returned error can't find the container with id a564bb895c3353cd5e40f3154d719920a56994d93bfe9c0f5d1c4657995be0e4 Jul 13 14:46:32.038749 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:32.038726 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a62a407_ab2f_44ca_896d_a1e642c6667d.slice/crio-2b350388e5cc0b66e3fb920e86781bd44cc178dfb1a3feef4e5aef006567acd7 WatchSource:0}: Error finding container 2b350388e5cc0b66e3fb920e86781bd44cc178dfb1a3feef4e5aef006567acd7: Status 404 returned error can't find the container with id 2b350388e5cc0b66e3fb920e86781bd44cc178dfb1a3feef4e5aef006567acd7 Jul 13 14:46:32.039628 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:32.039609 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod959b52b7_090f_4cd0_95ab_68ed775d9093.slice/crio-8edf34659d771a98e27f50a1a8ddd94c28cb5489186a68bdeeb0cfbd26f01747 WatchSource:0}: Error finding container 8edf34659d771a98e27f50a1a8ddd94c28cb5489186a68bdeeb0cfbd26f01747: Status 404 returned error can't find the container with id 8edf34659d771a98e27f50a1a8ddd94c28cb5489186a68bdeeb0cfbd26f01747 Jul 13 14:46:32.040698 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:32.040641 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bd25f5c_138c_4c60_9d20_525c8b12ebb1.slice/crio-014c22ab22a773b8d88cafece145e80597aec8669c9794b93654bdc89ccac41a WatchSource:0}: Error finding container 014c22ab22a773b8d88cafece145e80597aec8669c9794b93654bdc89ccac41a: Status 404 returned error can't find the container with id 014c22ab22a773b8d88cafece145e80597aec8669c9794b93654bdc89ccac41a Jul 13 14:46:32.042565 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:32.042545 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc45c5e7_30d9_4de3_ad6e_036706527893.slice/crio-84dec31a78159e8a7b96067a9f40a5cf2b155e72d76638969aa61b902ae7267d WatchSource:0}: Error finding container 84dec31a78159e8a7b96067a9f40a5cf2b155e72d76638969aa61b902ae7267d: Status 404 returned error can't find the container with id 84dec31a78159e8a7b96067a9f40a5cf2b155e72d76638969aa61b902ae7267d Jul 13 14:46:32.043337 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:32.043112 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36cb39be_505d_4a23_a395_4322f6301378.slice/crio-5e31abe77bf974162220c8609efd76648ec8d6eece8671327656ff9630c3ada1 WatchSource:0}: Error finding container 5e31abe77bf974162220c8609efd76648ec8d6eece8671327656ff9630c3ada1: Status 404 returned error can't find the container with id 5e31abe77bf974162220c8609efd76648ec8d6eece8671327656ff9630c3ada1 Jul 13 14:46:32.045733 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:46:32.045124 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf5174e4_503e_475b_ab01_20dd9f45124d.slice/crio-b03da93c818a4e1de9c163fe19666399721a9d163c44adb4f5682331e88de6b9 WatchSource:0}: Error finding container b03da93c818a4e1de9c163fe19666399721a9d163c44adb4f5682331e88de6b9: Status 404 returned error can't find the container with id b03da93c818a4e1de9c163fe19666399721a9d163c44adb4f5682331e88de6b9 Jul 13 14:46:32.210303 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:32.210106 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqbsk\" (UniqueName: \"kubernetes.io/projected/191cfe54-82c1-4694-92fd-3b93bb1783fd-kube-api-access-pqbsk\") pod \"network-check-target-r82bx\" (UID: \"191cfe54-82c1-4694-92fd-3b93bb1783fd\") " pod="openshift-network-diagnostics/network-check-target-r82bx" Jul 13 14:46:32.210303 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:32.210285 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 14:46:32.210303 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:32.210312 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 14:46:32.210997 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:32.210325 2568 projected.go:196] Error preparing data for projected volume kube-api-access-pqbsk for pod openshift-network-diagnostics/network-check-target-r82bx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 14:46:32.210997 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:32.210398 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/191cfe54-82c1-4694-92fd-3b93bb1783fd-kube-api-access-pqbsk podName:191cfe54-82c1-4694-92fd-3b93bb1783fd nodeName:}" failed. No retries permitted until 2026-07-13 14:46:34.210377088 +0000 UTC m=+6.450654934 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-pqbsk" (UniqueName: "kubernetes.io/projected/191cfe54-82c1-4694-92fd-3b93bb1783fd-kube-api-access-pqbsk") pod "network-check-target-r82bx" (UID: "191cfe54-82c1-4694-92fd-3b93bb1783fd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 14:46:32.266993 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:32.266959 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9hk94" Jul 13 14:46:32.266993 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:32.266959 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r82bx" Jul 13 14:46:32.267160 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:32.267011 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:46:32.267160 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:32.267119 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9hk94" podUID="30b24206-b58d-4409-b6e1-1dec743e754c" Jul 13 14:46:32.267235 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:32.267202 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r82bx" podUID="191cfe54-82c1-4694-92fd-3b93bb1783fd" Jul 13 14:46:32.267332 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:32.267304 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-km6w9" podUID="764568c4-93b4-4bf4-a627-a26e55610c81" Jul 13 14:46:32.311017 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:32.310986 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/764568c4-93b4-4bf4-a627-a26e55610c81-original-pull-secret\") pod \"global-pull-secret-syncer-km6w9\" (UID: \"764568c4-93b4-4bf4-a627-a26e55610c81\") " pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:46:32.311154 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:32.311044 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30b24206-b58d-4409-b6e1-1dec743e754c-metrics-certs\") pod \"network-metrics-daemon-9hk94\" (UID: \"30b24206-b58d-4409-b6e1-1dec743e754c\") " pod="openshift-multus/network-metrics-daemon-9hk94" Jul 13 14:46:32.311154 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:32.311141 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 14:46:32.311241 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:32.311147 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 14:46:32.311241 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:32.311193 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30b24206-b58d-4409-b6e1-1dec743e754c-metrics-certs podName:30b24206-b58d-4409-b6e1-1dec743e754c nodeName:}" failed. No retries permitted until 2026-07-13 14:46:34.311178354 +0000 UTC m=+6.551456182 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30b24206-b58d-4409-b6e1-1dec743e754c-metrics-certs") pod "network-metrics-daemon-9hk94" (UID: "30b24206-b58d-4409-b6e1-1dec743e754c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 14:46:32.311241 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:32.311206 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/764568c4-93b4-4bf4-a627-a26e55610c81-original-pull-secret podName:764568c4-93b4-4bf4-a627-a26e55610c81 nodeName:}" failed. No retries permitted until 2026-07-13 14:46:34.311200068 +0000 UTC m=+6.551477897 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/764568c4-93b4-4bf4-a627-a26e55610c81-original-pull-secret") pod "global-pull-secret-syncer-km6w9" (UID: "764568c4-93b4-4bf4-a627-a26e55610c81") : object "kube-system"/"original-pull-secret" not registered Jul 13 14:46:32.608047 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:32.608001 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qfs4l" event={"ID":"df5174e4-503e-475b-ab01-20dd9f45124d","Type":"ContainerStarted","Data":"b03da93c818a4e1de9c163fe19666399721a9d163c44adb4f5682331e88de6b9"} Jul 13 14:46:32.616991 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:32.616948 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" event={"ID":"36cb39be-505d-4a23-a395-4322f6301378","Type":"ContainerStarted","Data":"5e31abe77bf974162220c8609efd76648ec8d6eece8671327656ff9630c3ada1"} Jul 13 14:46:32.621201 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:32.621151 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nq9c" event={"ID":"fc45c5e7-30d9-4de3-ad6e-036706527893","Type":"ContainerStarted","Data":"84dec31a78159e8a7b96067a9f40a5cf2b155e72d76638969aa61b902ae7267d"} Jul 13 14:46:32.630938 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:32.630878 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-d7nhj" event={"ID":"959b52b7-090f-4cd0-95ab-68ed775d9093","Type":"ContainerStarted","Data":"8edf34659d771a98e27f50a1a8ddd94c28cb5489186a68bdeeb0cfbd26f01747"} Jul 13 14:46:32.640211 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:32.640177 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k8ltk" event={"ID":"6a62a407-ab2f-44ca-896d-a1e642c6667d","Type":"ContainerStarted","Data":"2b350388e5cc0b66e3fb920e86781bd44cc178dfb1a3feef4e5aef006567acd7"} Jul 13 14:46:32.648696 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:32.648663 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7v6qt" event={"ID":"5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822","Type":"ContainerStarted","Data":"af33cf31976020c73530594a701a439e01b6e46cf8a38c688162496f02db41cb"} Jul 13 14:46:32.652702 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:32.652673 2568 generic.go:356] "Generic (PLEG): container finished" podID="1ad2abc2bf15f3fd836f68bef031f771" containerID="601f51eb8d3725227cc752b60ff94e16dd2709bc06454e6ef3efc8ebdfdfce39" exitCode=0 Jul 13 14:46:32.652825 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:32.652760 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-96.ec2.internal" event={"ID":"1ad2abc2bf15f3fd836f68bef031f771","Type":"ContainerDied","Data":"601f51eb8d3725227cc752b60ff94e16dd2709bc06454e6ef3efc8ebdfdfce39"} Jul 13 14:46:32.657035 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:32.657008 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" event={"ID":"0bd25f5c-138c-4c60-9d20-525c8b12ebb1","Type":"ContainerStarted","Data":"014c22ab22a773b8d88cafece145e80597aec8669c9794b93654bdc89ccac41a"} Jul 13 14:46:32.661555 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:32.661504 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 14:41:31 +0000 UTC" deadline="2028-01-23 22:17:37.273862682 +0000 UTC" Jul 13 14:46:32.661555 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:32.661556 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13423h31m4.612309981s" Jul 13 14:46:32.666246 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:32.666194 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qg9jx" event={"ID":"87cdb037-6b66-416c-95a7-08332490b5a3","Type":"ContainerStarted","Data":"d7cff08a95c7ff9d3be9c16fe63bfe71ca0450870ffc9fab008e1c7941b0fe11"} Jul 13 14:46:32.669708 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:32.669642 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xz7wd" event={"ID":"d05cd493-5f17-4976-b2e5-404431d3389a","Type":"ContainerStarted","Data":"a564bb895c3353cd5e40f3154d719920a56994d93bfe9c0f5d1c4657995be0e4"} Jul 13 14:46:32.675866 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:32.675217 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-96.ec2.internal" event={"ID":"4161d8f2b036cc0e991f7a67c69322ec","Type":"ContainerStarted","Data":"6719549868ba37b7fd644423619f212402f7ec0c4df03568f85912863afdf731"} Jul 13 14:46:32.693266 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:32.692760 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-96.ec2.internal" podStartSLOduration=2.692739316 podStartE2EDuration="2.692739316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 14:46:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:46:32.692366598 +0000 UTC m=+4.932644450" watchObservedRunningTime="2026-07-13 14:46:32.692739316 +0000 UTC m=+4.933017171" Jul 13 14:46:33.689206 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:33.689088 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-96.ec2.internal" event={"ID":"1ad2abc2bf15f3fd836f68bef031f771","Type":"ContainerStarted","Data":"c9e704f23c330012dc8b71989177aadff6a7b23204aa57c49cf14b53208e6d46"} Jul 13 14:46:34.227833 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:34.227143 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqbsk\" (UniqueName: \"kubernetes.io/projected/191cfe54-82c1-4694-92fd-3b93bb1783fd-kube-api-access-pqbsk\") pod \"network-check-target-r82bx\" (UID: \"191cfe54-82c1-4694-92fd-3b93bb1783fd\") " pod="openshift-network-diagnostics/network-check-target-r82bx" Jul 13 14:46:34.227833 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:34.227328 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 14:46:34.227833 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:34.227353 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 14:46:34.227833 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:34.227365 2568 projected.go:196] Error preparing data for projected volume kube-api-access-pqbsk for pod openshift-network-diagnostics/network-check-target-r82bx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 14:46:34.227833 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:34.227431 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/191cfe54-82c1-4694-92fd-3b93bb1783fd-kube-api-access-pqbsk podName:191cfe54-82c1-4694-92fd-3b93bb1783fd nodeName:}" failed. No retries permitted until 2026-07-13 14:46:38.227406779 +0000 UTC m=+10.467684611 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-pqbsk" (UniqueName: "kubernetes.io/projected/191cfe54-82c1-4694-92fd-3b93bb1783fd-kube-api-access-pqbsk") pod "network-check-target-r82bx" (UID: "191cfe54-82c1-4694-92fd-3b93bb1783fd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 14:46:34.267227 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:34.266352 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9hk94" Jul 13 14:46:34.267227 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:34.266508 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9hk94" podUID="30b24206-b58d-4409-b6e1-1dec743e754c" Jul 13 14:46:34.267227 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:34.266921 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r82bx" Jul 13 14:46:34.267227 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:34.267017 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r82bx" podUID="191cfe54-82c1-4694-92fd-3b93bb1783fd" Jul 13 14:46:34.267227 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:34.267093 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:46:34.267227 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:34.267189 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-km6w9" podUID="764568c4-93b4-4bf4-a627-a26e55610c81" Jul 13 14:46:34.327804 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:34.327764 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30b24206-b58d-4409-b6e1-1dec743e754c-metrics-certs\") pod \"network-metrics-daemon-9hk94\" (UID: \"30b24206-b58d-4409-b6e1-1dec743e754c\") " pod="openshift-multus/network-metrics-daemon-9hk94" Jul 13 14:46:34.328127 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:34.328104 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/764568c4-93b4-4bf4-a627-a26e55610c81-original-pull-secret\") pod \"global-pull-secret-syncer-km6w9\" (UID: \"764568c4-93b4-4bf4-a627-a26e55610c81\") " pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:46:34.338901 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:34.338847 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 14:46:34.339097 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:34.338951 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30b24206-b58d-4409-b6e1-1dec743e754c-metrics-certs podName:30b24206-b58d-4409-b6e1-1dec743e754c nodeName:}" failed. No retries permitted until 2026-07-13 14:46:38.338926701 +0000 UTC m=+10.579204544 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30b24206-b58d-4409-b6e1-1dec743e754c-metrics-certs") pod "network-metrics-daemon-9hk94" (UID: "30b24206-b58d-4409-b6e1-1dec743e754c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 14:46:34.343315 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:34.339642 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 14:46:34.343315 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:34.339741 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/764568c4-93b4-4bf4-a627-a26e55610c81-original-pull-secret podName:764568c4-93b4-4bf4-a627-a26e55610c81 nodeName:}" failed. No retries permitted until 2026-07-13 14:46:38.33972163 +0000 UTC m=+10.579999464 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/764568c4-93b4-4bf4-a627-a26e55610c81-original-pull-secret") pod "global-pull-secret-syncer-km6w9" (UID: "764568c4-93b4-4bf4-a627-a26e55610c81") : object "kube-system"/"original-pull-secret" not registered Jul 13 14:46:36.267894 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:36.267854 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9hk94" Jul 13 14:46:36.268405 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:36.268029 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9hk94" podUID="30b24206-b58d-4409-b6e1-1dec743e754c" Jul 13 14:46:36.268474 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:36.268461 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r82bx" Jul 13 14:46:36.268590 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:36.268562 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r82bx" podUID="191cfe54-82c1-4694-92fd-3b93bb1783fd" Jul 13 14:46:36.268765 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:36.268743 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:46:36.268879 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:36.268857 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-km6w9" podUID="764568c4-93b4-4bf4-a627-a26e55610c81" Jul 13 14:46:38.265548 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:38.265492 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqbsk\" (UniqueName: \"kubernetes.io/projected/191cfe54-82c1-4694-92fd-3b93bb1783fd-kube-api-access-pqbsk\") pod \"network-check-target-r82bx\" (UID: \"191cfe54-82c1-4694-92fd-3b93bb1783fd\") " pod="openshift-network-diagnostics/network-check-target-r82bx" Jul 13 14:46:38.266087 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:38.265706 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 14:46:38.266087 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:38.265730 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 14:46:38.266087 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:38.265746 2568 projected.go:196] Error preparing data for projected volume kube-api-access-pqbsk for pod openshift-network-diagnostics/network-check-target-r82bx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 14:46:38.266087 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:38.265821 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/191cfe54-82c1-4694-92fd-3b93bb1783fd-kube-api-access-pqbsk podName:191cfe54-82c1-4694-92fd-3b93bb1783fd nodeName:}" failed. No retries permitted until 2026-07-13 14:46:46.265800874 +0000 UTC m=+18.506078724 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-pqbsk" (UniqueName: "kubernetes.io/projected/191cfe54-82c1-4694-92fd-3b93bb1783fd-kube-api-access-pqbsk") pod "network-check-target-r82bx" (UID: "191cfe54-82c1-4694-92fd-3b93bb1783fd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 14:46:38.267494 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:38.267465 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:46:38.269655 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:38.268672 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r82bx" Jul 13 14:46:38.269655 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:38.268809 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-km6w9" podUID="764568c4-93b4-4bf4-a627-a26e55610c81" Jul 13 14:46:38.269655 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:38.268832 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r82bx" podUID="191cfe54-82c1-4694-92fd-3b93bb1783fd" Jul 13 14:46:38.269655 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:38.268897 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9hk94" Jul 13 14:46:38.269655 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:38.269039 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9hk94" podUID="30b24206-b58d-4409-b6e1-1dec743e754c" Jul 13 14:46:38.366414 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:38.366371 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/764568c4-93b4-4bf4-a627-a26e55610c81-original-pull-secret\") pod \"global-pull-secret-syncer-km6w9\" (UID: \"764568c4-93b4-4bf4-a627-a26e55610c81\") " pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:46:38.366669 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:38.366466 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30b24206-b58d-4409-b6e1-1dec743e754c-metrics-certs\") pod \"network-metrics-daemon-9hk94\" (UID: \"30b24206-b58d-4409-b6e1-1dec743e754c\") " pod="openshift-multus/network-metrics-daemon-9hk94" Jul 13 14:46:38.366669 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:38.366605 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 14:46:38.366793 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:38.366694 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30b24206-b58d-4409-b6e1-1dec743e754c-metrics-certs podName:30b24206-b58d-4409-b6e1-1dec743e754c nodeName:}" failed. No retries permitted until 2026-07-13 14:46:46.366671402 +0000 UTC m=+18.606949232 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30b24206-b58d-4409-b6e1-1dec743e754c-metrics-certs") pod "network-metrics-daemon-9hk94" (UID: "30b24206-b58d-4409-b6e1-1dec743e754c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 14:46:38.367127 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:38.367104 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 14:46:38.367224 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:38.367164 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/764568c4-93b4-4bf4-a627-a26e55610c81-original-pull-secret podName:764568c4-93b4-4bf4-a627-a26e55610c81 nodeName:}" failed. No retries permitted until 2026-07-13 14:46:46.367147637 +0000 UTC m=+18.607425470 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/764568c4-93b4-4bf4-a627-a26e55610c81-original-pull-secret") pod "global-pull-secret-syncer-km6w9" (UID: "764568c4-93b4-4bf4-a627-a26e55610c81") : object "kube-system"/"original-pull-secret" not registered Jul 13 14:46:40.266950 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:40.266119 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9hk94" Jul 13 14:46:40.266950 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:40.266264 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9hk94" podUID="30b24206-b58d-4409-b6e1-1dec743e754c" Jul 13 14:46:40.266950 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:40.266662 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r82bx" Jul 13 14:46:40.266950 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:40.266749 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r82bx" podUID="191cfe54-82c1-4694-92fd-3b93bb1783fd" Jul 13 14:46:40.266950 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:40.266846 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:46:40.266950 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:40.266918 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-km6w9" podUID="764568c4-93b4-4bf4-a627-a26e55610c81" Jul 13 14:46:42.266652 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:42.266605 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:46:42.267101 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:42.266746 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-km6w9" podUID="764568c4-93b4-4bf4-a627-a26e55610c81" Jul 13 14:46:42.267101 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:42.266605 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9hk94" Jul 13 14:46:42.267101 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:42.266917 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9hk94" podUID="30b24206-b58d-4409-b6e1-1dec743e754c" Jul 13 14:46:42.267101 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:42.267057 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r82bx" Jul 13 14:46:42.267367 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:42.267156 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r82bx" podUID="191cfe54-82c1-4694-92fd-3b93bb1783fd" Jul 13 14:46:44.266410 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:44.266364 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r82bx" Jul 13 14:46:44.266898 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:44.266384 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9hk94" Jul 13 14:46:44.266898 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:44.266502 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r82bx" podUID="191cfe54-82c1-4694-92fd-3b93bb1783fd" Jul 13 14:46:44.266898 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:44.266365 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:46:44.266898 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:44.266668 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9hk94" podUID="30b24206-b58d-4409-b6e1-1dec743e754c" Jul 13 14:46:44.266898 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:44.266776 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-km6w9" podUID="764568c4-93b4-4bf4-a627-a26e55610c81" Jul 13 14:46:46.266887 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:46.266855 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9hk94" Jul 13 14:46:46.266887 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:46.266887 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:46:46.267377 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:46.266855 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r82bx" Jul 13 14:46:46.267377 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:46.266997 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-km6w9" podUID="764568c4-93b4-4bf4-a627-a26e55610c81" Jul 13 14:46:46.267377 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:46.267080 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r82bx" podUID="191cfe54-82c1-4694-92fd-3b93bb1783fd" Jul 13 14:46:46.267377 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:46.267145 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9hk94" podUID="30b24206-b58d-4409-b6e1-1dec743e754c" Jul 13 14:46:46.324266 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:46.324227 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqbsk\" (UniqueName: \"kubernetes.io/projected/191cfe54-82c1-4694-92fd-3b93bb1783fd-kube-api-access-pqbsk\") pod \"network-check-target-r82bx\" (UID: \"191cfe54-82c1-4694-92fd-3b93bb1783fd\") " pod="openshift-network-diagnostics/network-check-target-r82bx" Jul 13 14:46:46.324455 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:46.324432 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 14:46:46.324541 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:46.324463 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 14:46:46.324541 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:46.324477 2568 projected.go:196] Error preparing data for projected volume kube-api-access-pqbsk for pod openshift-network-diagnostics/network-check-target-r82bx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 14:46:46.324663 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:46.324544 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/191cfe54-82c1-4694-92fd-3b93bb1783fd-kube-api-access-pqbsk podName:191cfe54-82c1-4694-92fd-3b93bb1783fd nodeName:}" failed. No retries permitted until 2026-07-13 14:47:02.324523578 +0000 UTC m=+34.564801408 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-pqbsk" (UniqueName: "kubernetes.io/projected/191cfe54-82c1-4694-92fd-3b93bb1783fd-kube-api-access-pqbsk") pod "network-check-target-r82bx" (UID: "191cfe54-82c1-4694-92fd-3b93bb1783fd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 14:46:46.425251 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:46.425216 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/764568c4-93b4-4bf4-a627-a26e55610c81-original-pull-secret\") pod \"global-pull-secret-syncer-km6w9\" (UID: \"764568c4-93b4-4bf4-a627-a26e55610c81\") " pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:46:46.425429 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:46.425273 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30b24206-b58d-4409-b6e1-1dec743e754c-metrics-certs\") pod \"network-metrics-daemon-9hk94\" (UID: \"30b24206-b58d-4409-b6e1-1dec743e754c\") " pod="openshift-multus/network-metrics-daemon-9hk94" Jul 13 14:46:46.425429 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:46.425362 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 14:46:46.425429 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:46.425367 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 14:46:46.425429 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:46.425427 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30b24206-b58d-4409-b6e1-1dec743e754c-metrics-certs podName:30b24206-b58d-4409-b6e1-1dec743e754c nodeName:}" failed. No retries permitted until 2026-07-13 14:47:02.425407867 +0000 UTC m=+34.665685695 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30b24206-b58d-4409-b6e1-1dec743e754c-metrics-certs") pod "network-metrics-daemon-9hk94" (UID: "30b24206-b58d-4409-b6e1-1dec743e754c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 14:46:46.425605 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:46.425446 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/764568c4-93b4-4bf4-a627-a26e55610c81-original-pull-secret podName:764568c4-93b4-4bf4-a627-a26e55610c81 nodeName:}" failed. No retries permitted until 2026-07-13 14:47:02.425436469 +0000 UTC m=+34.665714305 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/764568c4-93b4-4bf4-a627-a26e55610c81-original-pull-secret") pod "global-pull-secret-syncer-km6w9" (UID: "764568c4-93b4-4bf4-a627-a26e55610c81") : object "kube-system"/"original-pull-secret" not registered Jul 13 14:46:48.267246 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:48.267212 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9hk94" Jul 13 14:46:48.267246 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:48.267240 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r82bx" Jul 13 14:46:48.267786 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:48.267329 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9hk94" podUID="30b24206-b58d-4409-b6e1-1dec743e754c" Jul 13 14:46:48.267845 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:48.267833 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:46:48.267894 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:48.267848 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r82bx" podUID="191cfe54-82c1-4694-92fd-3b93bb1783fd" Jul 13 14:46:48.267948 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:48.267928 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-km6w9" podUID="764568c4-93b4-4bf4-a627-a26e55610c81" Jul 13 14:46:50.267154 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:50.267114 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9hk94" Jul 13 14:46:50.267154 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:50.267150 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:46:50.267753 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:50.267114 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r82bx" Jul 13 14:46:50.267753 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:50.267275 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9hk94" podUID="30b24206-b58d-4409-b6e1-1dec743e754c" Jul 13 14:46:50.267753 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:50.267400 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r82bx" podUID="191cfe54-82c1-4694-92fd-3b93bb1783fd" Jul 13 14:46:50.267753 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:50.267494 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-km6w9" podUID="764568c4-93b4-4bf4-a627-a26e55610c81" Jul 13 14:46:51.731454 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:51.731207 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k8ltk" event={"ID":"6a62a407-ab2f-44ca-896d-a1e642c6667d","Type":"ContainerStarted","Data":"f2b6e2403aebac36051ec12abe916edbcfefa0b0d601ff1a856b542972bd768f"} Jul 13 14:46:51.733256 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:51.733224 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7v6qt" event={"ID":"5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822","Type":"ContainerStarted","Data":"ea0c636deed4cc518439546ab66747b62ad841b6218434c574b64005803a1a90"} Jul 13 14:46:51.735273 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:51.735248 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" event={"ID":"0bd25f5c-138c-4c60-9d20-525c8b12ebb1","Type":"ContainerStarted","Data":"a653bc165b8006e25c00f8b4681a2f7d3079fdac17375df7deedeb3574e6eae8"} Jul 13 14:46:51.735384 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:51.735324 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" event={"ID":"0bd25f5c-138c-4c60-9d20-525c8b12ebb1","Type":"ContainerStarted","Data":"51353310cbfd1dc8d799ed033c3d5734180a7d52b10941ca074c6c1e94817300"} Jul 13 14:46:51.736830 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:51.736803 2568 generic.go:356] "Generic (PLEG): container finished" podID="87cdb037-6b66-416c-95a7-08332490b5a3" containerID="6190db8532ad5c7602437393d22fd9c4d72cf349fee6aaec60d555de0460f2cb" exitCode=0 Jul 13 14:46:51.737004 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:51.736884 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qg9jx" event={"ID":"87cdb037-6b66-416c-95a7-08332490b5a3","Type":"ContainerDied","Data":"6190db8532ad5c7602437393d22fd9c4d72cf349fee6aaec60d555de0460f2cb"} Jul 13 14:46:51.739292 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:51.739266 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xz7wd" event={"ID":"d05cd493-5f17-4976-b2e5-404431d3389a","Type":"ContainerStarted","Data":"e4f6e568988a78006c4f024ffb5d44b3f668c6567ddb266e67d8e243b7092717"} Jul 13 14:46:51.740911 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:51.740880 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qfs4l" event={"ID":"df5174e4-503e-475b-ab01-20dd9f45124d","Type":"ContainerStarted","Data":"e6b54cb7e66f691d9b3dffb4fee1c9d071b1d80868c3aa8e396470806c7ac22f"} Jul 13 14:46:51.742377 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:51.742354 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" event={"ID":"36cb39be-505d-4a23-a395-4322f6301378","Type":"ContainerStarted","Data":"b5500924f5caf3c16fd68f275ffaf90768ea3f2c9075e54bcc540a733ccfc1ec"} Jul 13 14:46:51.743970 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:51.743949 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nq9c" event={"ID":"fc45c5e7-30d9-4de3-ad6e-036706527893","Type":"ContainerStarted","Data":"b7237f368eef358f3ffda2c504d900e2ac980c5353680a5dad2bd9e2bd383693"} Jul 13 14:46:51.750677 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:51.750636 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-96.ec2.internal" podStartSLOduration=21.750624218 podStartE2EDuration="21.750624218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 14:46:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:46:33.708328038 +0000 UTC m=+5.948605892" watchObservedRunningTime="2026-07-13 14:46:51.750624218 +0000 UTC m=+23.990902066" Jul 13 14:46:51.767783 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:51.767739 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-k8ltk" podStartSLOduration=3.314700795 podStartE2EDuration="22.767723861s" totalImagesPullingTime="19.453023066s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 14:46:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:46:51.750918991 +0000 UTC m=+23.991196866" watchObservedRunningTime="2026-07-13 14:46:51.767723861 +0000 UTC m=+24.008001711" Jul 13 14:46:51.787839 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:51.787783 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-xz7wd" podStartSLOduration=3.388555216 podStartE2EDuration="22.7877597s" totalImagesPullingTime="19.399204484s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 14:46:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:46:51.767705297 +0000 UTC m=+24.007983145" watchObservedRunningTime="2026-07-13 14:46:51.7877597 +0000 UTC m=+24.028037611" Jul 13 14:46:51.814839 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:51.814632 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-7v6qt" podStartSLOduration=3.40887633 podStartE2EDuration="22.814614485s" totalImagesPullingTime="19.405738155s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 14:46:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:46:51.787637958 +0000 UTC m=+24.027915822" watchObservedRunningTime="2026-07-13 14:46:51.814614485 +0000 UTC m=+24.054892335" Jul 13 14:46:51.829884 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:51.829840 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qfs4l" podStartSLOduration=11.485103945 podStartE2EDuration="22.829825514s" totalImagesPullingTime="11.344721569s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 14:46:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:46:51.829585804 +0000 UTC m=+24.069863664" watchObservedRunningTime="2026-07-13 14:46:51.829825514 +0000 UTC m=+24.070103360" Jul 13 14:46:51.843854 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:51.843799 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:46:51.846097 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:51.846057 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-5rmzg" podStartSLOduration=3.439270915 podStartE2EDuration="22.846042759s" totalImagesPullingTime="19.406771844s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 14:46:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:46:51.845491581 +0000 UTC m=+24.085769429" watchObservedRunningTime="2026-07-13 14:46:51.846042759 +0000 UTC m=+24.086320643" Jul 13 14:46:51.913738 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:51.913669 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:46:52.015130 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:52.015092 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:46:52.266153 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:52.266075 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:46:52.266365 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:52.266219 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-km6w9" podUID="764568c4-93b4-4bf4-a627-a26e55610c81" Jul 13 14:46:52.266365 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:52.266245 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9hk94" Jul 13 14:46:52.266365 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:52.266259 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r82bx" Jul 13 14:46:52.266365 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:52.266344 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9hk94" podUID="30b24206-b58d-4409-b6e1-1dec743e754c" Jul 13 14:46:52.266533 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:52.266420 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r82bx" podUID="191cfe54-82c1-4694-92fd-3b93bb1783fd" Jul 13 14:46:52.748678 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:52.748426 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-d7nhj" event={"ID":"959b52b7-090f-4cd0-95ab-68ed775d9093","Type":"ContainerStarted","Data":"6800eb5e1da4f1e4840a5a14e7f9729e39d4a1b849be65f1c54352ad9ce76310"} Jul 13 14:46:52.752369 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:52.752342 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xgqvx_0bd25f5c-138c-4c60-9d20-525c8b12ebb1/ovn-acl-logging/0.log" Jul 13 14:46:52.752728 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:52.752697 2568 generic.go:356] "Generic (PLEG): container finished" podID="0bd25f5c-138c-4c60-9d20-525c8b12ebb1" containerID="a653bc165b8006e25c00f8b4681a2f7d3079fdac17375df7deedeb3574e6eae8" exitCode=1 Jul 13 14:46:52.752847 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:52.752725 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" event={"ID":"0bd25f5c-138c-4c60-9d20-525c8b12ebb1","Type":"ContainerDied","Data":"a653bc165b8006e25c00f8b4681a2f7d3079fdac17375df7deedeb3574e6eae8"} Jul 13 14:46:52.752847 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:52.752803 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" event={"ID":"0bd25f5c-138c-4c60-9d20-525c8b12ebb1","Type":"ContainerStarted","Data":"2c4367b3888d8f1df3eeca0a844da6f3a27dd23402e939608a4afac972951f6e"} Jul 13 14:46:52.752945 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:52.752850 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" event={"ID":"0bd25f5c-138c-4c60-9d20-525c8b12ebb1","Type":"ContainerStarted","Data":"94cbc7475d5b663994f2ada6a130063f2753085848af12013f62eebc55ee5c3d"} Jul 13 14:46:52.752945 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:52.752900 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" event={"ID":"0bd25f5c-138c-4c60-9d20-525c8b12ebb1","Type":"ContainerStarted","Data":"39e96991adb8a1c174f978373ff6c325036ce68fc12f5bf1787c46a59f8ffa12"} Jul 13 14:46:52.753084 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:52.752946 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" event={"ID":"0bd25f5c-138c-4c60-9d20-525c8b12ebb1","Type":"ContainerStarted","Data":"9dc2a03117d2e436952bce30dd58993150a76a34474af1fb78c00519a21b5170"} Jul 13 14:46:52.940703 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:52.940674 2568 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 13 14:46:53.184049 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:53.183949 2568 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-13T14:46:52.940695939Z","UUID":"22541dfa-3590-41ef-b289-85bab1184365","Handler":null,"Name":"","Endpoint":""} Jul 13 14:46:53.185629 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:53.185606 2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 13 14:46:53.185746 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:53.185647 2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 13 14:46:53.757774 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:53.757736 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nq9c" event={"ID":"fc45c5e7-30d9-4de3-ad6e-036706527893","Type":"ContainerStarted","Data":"7039bbd8d348089d93ff0a14fe191a9aab940c3013d3db86b6b27d11ebf594fe"} Jul 13 14:46:54.267188 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:54.266988 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r82bx" Jul 13 14:46:54.267369 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:54.266989 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9hk94" Jul 13 14:46:54.267369 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:54.267280 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r82bx" podUID="191cfe54-82c1-4694-92fd-3b93bb1783fd" Jul 13 14:46:54.267369 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:54.266988 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:46:54.267369 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:54.267335 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9hk94" podUID="30b24206-b58d-4409-b6e1-1dec743e754c" Jul 13 14:46:54.267601 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:54.267436 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-km6w9" podUID="764568c4-93b4-4bf4-a627-a26e55610c81" Jul 13 14:46:54.651074 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:54.651035 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:46:54.740290 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:54.740253 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:46:54.763534 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:54.763499 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xgqvx_0bd25f5c-138c-4c60-9d20-525c8b12ebb1/ovn-acl-logging/0.log" Jul 13 14:46:54.763982 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:54.763961 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" event={"ID":"0bd25f5c-138c-4c60-9d20-525c8b12ebb1","Type":"ContainerStarted","Data":"89c9235182567dbc73184df34e51e5a15a66f2c03593407030f1ade6c0c075b3"} Jul 13 14:46:55.292959 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:55.292922 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-7v6qt" Jul 13 14:46:55.293634 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:55.293608 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-7v6qt" Jul 13 14:46:55.315911 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:55.315852 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-d7nhj" podStartSLOduration=6.91845328 podStartE2EDuration="26.315830156s" totalImagesPullingTime="19.397376876s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 14:46:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:46:53.777628917 +0000 UTC m=+26.017906786" watchObservedRunningTime="2026-07-13 14:46:55.315830156 +0000 UTC m=+27.556108006" Jul 13 14:46:55.766391 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:55.766313 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-7v6qt" Jul 13 14:46:55.767104 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:55.766907 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-7v6qt" Jul 13 14:46:55.841134 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:55.841098 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:46:56.266924 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:56.266744 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:46:56.267099 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:56.266765 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9hk94" Jul 13 14:46:56.267099 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:56.267019 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-km6w9" podUID="764568c4-93b4-4bf4-a627-a26e55610c81" Jul 13 14:46:56.267207 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:56.266796 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r82bx" Jul 13 14:46:56.267207 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:56.267110 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9hk94" podUID="30b24206-b58d-4409-b6e1-1dec743e754c" Jul 13 14:46:56.267301 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:56.267245 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r82bx" podUID="191cfe54-82c1-4694-92fd-3b93bb1783fd" Jul 13 14:46:56.769763 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:56.769676 2568 generic.go:356] "Generic (PLEG): container finished" podID="87cdb037-6b66-416c-95a7-08332490b5a3" containerID="76f6a28f1862c6c5a148ab645f9c7f2f90eae394f9757b1e926de1129a903e34" exitCode=0 Jul 13 14:46:56.770401 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:56.769758 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qg9jx" event={"ID":"87cdb037-6b66-416c-95a7-08332490b5a3","Type":"ContainerDied","Data":"76f6a28f1862c6c5a148ab645f9c7f2f90eae394f9757b1e926de1129a903e34"} Jul 13 14:46:56.771823 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:56.771798 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nq9c" event={"ID":"fc45c5e7-30d9-4de3-ad6e-036706527893","Type":"ContainerStarted","Data":"4384c83f1dc2e2a6ef1a78c91715f21f41003d02540b2a52ddf0d5ca3792f341"} Jul 13 14:46:56.775154 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:56.775137 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xgqvx_0bd25f5c-138c-4c60-9d20-525c8b12ebb1/ovn-acl-logging/0.log" Jul 13 14:46:56.775468 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:56.775450 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" event={"ID":"0bd25f5c-138c-4c60-9d20-525c8b12ebb1","Type":"ContainerStarted","Data":"d4840b54001c45d488fef82be261934e10334cf3002180b6f95b52cead3a30c9"} Jul 13 14:46:56.775885 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:56.775732 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:56.775885 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:56.775804 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:56.775885 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:56.775852 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:56.775885 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:56.775883 2568 scope.go:117] "RemoveContainer" containerID="a653bc165b8006e25c00f8b4681a2f7d3079fdac17375df7deedeb3574e6eae8" Jul 13 14:46:56.791885 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:56.791862 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:56.792697 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:56.792679 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:46:56.814655 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:56.814609 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nq9c" podStartSLOduration=4.307922208 podStartE2EDuration="27.814593718s" totalImagesPullingTime="23.50667151s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 14:46:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:46:56.814288221 +0000 UTC m=+29.054566071" watchObservedRunningTime="2026-07-13 14:46:56.814593718 +0000 UTC m=+29.054871564" Jul 13 14:46:57.781689 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:57.781658 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xgqvx_0bd25f5c-138c-4c60-9d20-525c8b12ebb1/ovn-acl-logging/0.log" Jul 13 14:46:57.782171 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:57.782077 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" event={"ID":"0bd25f5c-138c-4c60-9d20-525c8b12ebb1","Type":"ContainerStarted","Data":"243eea42487e3790a7de62a68b2f07d6ac985a64eefe4fab68f7d14bb949f737"} Jul 13 14:46:57.816398 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:57.816303 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" podStartSLOduration=9.362763891 podStartE2EDuration="28.816263283s" totalImagesPullingTime="19.453499392s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 14:46:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:46:57.814518184 +0000 UTC m=+30.054796095" watchObservedRunningTime="2026-07-13 14:46:57.816263283 +0000 UTC m=+30.056541236" Jul 13 14:46:58.267762 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:58.267612 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9hk94" Jul 13 14:46:58.267762 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:58.267694 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:46:58.268086 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:58.267800 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-km6w9" podUID="764568c4-93b4-4bf4-a627-a26e55610c81" Jul 13 14:46:58.268086 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:58.267853 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r82bx" Jul 13 14:46:58.268086 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:58.267959 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r82bx" podUID="191cfe54-82c1-4694-92fd-3b93bb1783fd" Jul 13 14:46:58.268329 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:58.268268 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9hk94" podUID="30b24206-b58d-4409-b6e1-1dec743e754c" Jul 13 14:46:58.521210 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:58.520945 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r82bx"] Jul 13 14:46:58.521616 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:58.521548 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-km6w9"] Jul 13 14:46:58.523308 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:58.523284 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9hk94"] Jul 13 14:46:58.786655 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:58.786624 2568 generic.go:356] "Generic (PLEG): container finished" podID="87cdb037-6b66-416c-95a7-08332490b5a3" containerID="4c5e513f7b204a0bb7d1d4d0e8323da79d46f029cbb314527a6179422574dcf3" exitCode=0 Jul 13 14:46:58.787189 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:58.786694 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qg9jx" event={"ID":"87cdb037-6b66-416c-95a7-08332490b5a3","Type":"ContainerDied","Data":"4c5e513f7b204a0bb7d1d4d0e8323da79d46f029cbb314527a6179422574dcf3"} Jul 13 14:46:58.787189 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:58.786707 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9hk94" Jul 13 14:46:58.787189 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:58.786781 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r82bx" Jul 13 14:46:58.787189 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:46:58.786807 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:46:58.787189 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:58.786867 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r82bx" podUID="191cfe54-82c1-4694-92fd-3b93bb1783fd" Jul 13 14:46:58.787189 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:58.787035 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-km6w9" podUID="764568c4-93b4-4bf4-a627-a26e55610c81" Jul 13 14:46:58.787189 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:46:58.787155 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9hk94" podUID="30b24206-b58d-4409-b6e1-1dec743e754c" Jul 13 14:47:00.266882 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:00.266846 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9hk94" Jul 13 14:47:00.267356 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:00.266848 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:47:00.267356 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:00.266969 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9hk94" podUID="30b24206-b58d-4409-b6e1-1dec743e754c" Jul 13 14:47:00.267356 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:00.267068 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-km6w9" podUID="764568c4-93b4-4bf4-a627-a26e55610c81" Jul 13 14:47:00.267356 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:00.266846 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r82bx" Jul 13 14:47:00.267356 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:00.267174 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r82bx" podUID="191cfe54-82c1-4694-92fd-3b93bb1783fd" Jul 13 14:47:00.767582 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:00.767479 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:00.794062 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:00.794032 2568 generic.go:356] "Generic (PLEG): container finished" podID="87cdb037-6b66-416c-95a7-08332490b5a3" containerID="59729850effcf38a3b902e0d61a9ef3744fbd85fc4d8854a649636ba7dc90bf5" exitCode=0 Jul 13 14:47:00.794218 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:00.794069 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qg9jx" event={"ID":"87cdb037-6b66-416c-95a7-08332490b5a3","Type":"ContainerDied","Data":"59729850effcf38a3b902e0d61a9ef3744fbd85fc4d8854a649636ba7dc90bf5"} Jul 13 14:47:00.880585 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:00.880537 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:02.266531 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:02.266493 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9hk94" Jul 13 14:47:02.267297 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:02.266493 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:47:02.267297 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:02.266666 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9hk94" podUID="30b24206-b58d-4409-b6e1-1dec743e754c" Jul 13 14:47:02.267297 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:02.266678 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r82bx" Jul 13 14:47:02.267297 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:02.266742 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-km6w9" podUID="764568c4-93b4-4bf4-a627-a26e55610c81" Jul 13 14:47:02.267297 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:02.266808 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r82bx" podUID="191cfe54-82c1-4694-92fd-3b93bb1783fd" Jul 13 14:47:02.339266 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:02.339226 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqbsk\" (UniqueName: \"kubernetes.io/projected/191cfe54-82c1-4694-92fd-3b93bb1783fd-kube-api-access-pqbsk\") pod \"network-check-target-r82bx\" (UID: \"191cfe54-82c1-4694-92fd-3b93bb1783fd\") " pod="openshift-network-diagnostics/network-check-target-r82bx" Jul 13 14:47:02.339437 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:02.339345 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 14:47:02.339437 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:02.339363 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 14:47:02.339437 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:02.339371 2568 projected.go:196] Error preparing data for projected volume kube-api-access-pqbsk for pod openshift-network-diagnostics/network-check-target-r82bx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 14:47:02.339437 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:02.339433 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/191cfe54-82c1-4694-92fd-3b93bb1783fd-kube-api-access-pqbsk podName:191cfe54-82c1-4694-92fd-3b93bb1783fd nodeName:}" failed. No retries permitted until 2026-07-13 14:47:34.3394151 +0000 UTC m=+66.579692945 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-pqbsk" (UniqueName: "kubernetes.io/projected/191cfe54-82c1-4694-92fd-3b93bb1783fd-kube-api-access-pqbsk") pod "network-check-target-r82bx" (UID: "191cfe54-82c1-4694-92fd-3b93bb1783fd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 14:47:02.439851 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:02.439807 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30b24206-b58d-4409-b6e1-1dec743e754c-metrics-certs\") pod \"network-metrics-daemon-9hk94\" (UID: \"30b24206-b58d-4409-b6e1-1dec743e754c\") " pod="openshift-multus/network-metrics-daemon-9hk94" Jul 13 14:47:02.440020 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:02.439878 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/764568c4-93b4-4bf4-a627-a26e55610c81-original-pull-secret\") pod \"global-pull-secret-syncer-km6w9\" (UID: \"764568c4-93b4-4bf4-a627-a26e55610c81\") " pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:47:02.440020 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:02.439994 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 14:47:02.440148 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:02.440055 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 14:47:02.440148 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:02.440064 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30b24206-b58d-4409-b6e1-1dec743e754c-metrics-certs podName:30b24206-b58d-4409-b6e1-1dec743e754c nodeName:}" failed. No retries permitted until 2026-07-13 14:47:34.440048502 +0000 UTC m=+66.680326329 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30b24206-b58d-4409-b6e1-1dec743e754c-metrics-certs") pod "network-metrics-daemon-9hk94" (UID: "30b24206-b58d-4409-b6e1-1dec743e754c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 14:47:02.440148 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:02.440110 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/764568c4-93b4-4bf4-a627-a26e55610c81-original-pull-secret podName:764568c4-93b4-4bf4-a627-a26e55610c81 nodeName:}" failed. No retries permitted until 2026-07-13 14:47:34.440093919 +0000 UTC m=+66.680371747 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/764568c4-93b4-4bf4-a627-a26e55610c81-original-pull-secret") pod "global-pull-secret-syncer-km6w9" (UID: "764568c4-93b4-4bf4-a627-a26e55610c81") : object "kube-system"/"original-pull-secret" not registered Jul 13 14:47:03.259850 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:03.259813 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:03.339959 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:03.339913 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:04.041717 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.041687 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-96.ec2.internal" event="NodeReady" Jul 13 14:47:04.041908 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.041839 2568 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 13 14:47:04.092021 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.091986 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-b47lx"] Jul 13 14:47:04.094585 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.094555 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-mgkcj"] Jul 13 14:47:04.094708 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.094695 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b47lx" Jul 13 14:47:04.096447 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.096427 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mgkcj" Jul 13 14:47:04.097340 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.097316 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-k8vvm\"" Jul 13 14:47:04.097434 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.097323 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 13 14:47:04.097434 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.097322 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 13 14:47:04.098485 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.098460 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 13 14:47:04.098783 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.098732 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 13 14:47:04.098783 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.098734 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7xw2p\"" Jul 13 14:47:04.098935 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.098878 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 13 14:47:04.105100 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.105078 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b47lx"] Jul 13 14:47:04.107605 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.107585 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mgkcj"] Jul 13 14:47:04.156093 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.156054 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crvnc\" (UniqueName: \"kubernetes.io/projected/d8061d1f-15e7-4f20-a3c6-d3b9b865cb0e-kube-api-access-crvnc\") pod \"ingress-canary-mgkcj\" (UID: \"d8061d1f-15e7-4f20-a3c6-d3b9b865cb0e\") " pod="openshift-ingress-canary/ingress-canary-mgkcj" Jul 13 14:47:04.156259 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.156105 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c60d2d4-243b-4952-83b9-bb2c462d9e2c-metrics-tls\") pod \"dns-default-b47lx\" (UID: \"6c60d2d4-243b-4952-83b9-bb2c462d9e2c\") " pod="openshift-dns/dns-default-b47lx" Jul 13 14:47:04.156259 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.156134 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnq8c\" (UniqueName: \"kubernetes.io/projected/6c60d2d4-243b-4952-83b9-bb2c462d9e2c-kube-api-access-pnq8c\") pod \"dns-default-b47lx\" (UID: \"6c60d2d4-243b-4952-83b9-bb2c462d9e2c\") " pod="openshift-dns/dns-default-b47lx" Jul 13 14:47:04.156259 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.156231 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c60d2d4-243b-4952-83b9-bb2c462d9e2c-tmp-dir\") pod \"dns-default-b47lx\" (UID: \"6c60d2d4-243b-4952-83b9-bb2c462d9e2c\") " pod="openshift-dns/dns-default-b47lx" Jul 13 14:47:04.156378 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.156264 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d8061d1f-15e7-4f20-a3c6-d3b9b865cb0e-cert\") pod \"ingress-canary-mgkcj\" (UID: \"d8061d1f-15e7-4f20-a3c6-d3b9b865cb0e\") " pod="openshift-ingress-canary/ingress-canary-mgkcj" Jul 13 14:47:04.156378 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.156300 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6c60d2d4-243b-4952-83b9-bb2c462d9e2c-config-volume\") pod \"dns-default-b47lx\" (UID: \"6c60d2d4-243b-4952-83b9-bb2c462d9e2c\") " pod="openshift-dns/dns-default-b47lx" Jul 13 14:47:04.253382 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.253300 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:04.257282 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.257258 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c60d2d4-243b-4952-83b9-bb2c462d9e2c-tmp-dir\") pod \"dns-default-b47lx\" (UID: \"6c60d2d4-243b-4952-83b9-bb2c462d9e2c\") " pod="openshift-dns/dns-default-b47lx" Jul 13 14:47:04.257424 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.257304 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d8061d1f-15e7-4f20-a3c6-d3b9b865cb0e-cert\") pod \"ingress-canary-mgkcj\" (UID: \"d8061d1f-15e7-4f20-a3c6-d3b9b865cb0e\") " pod="openshift-ingress-canary/ingress-canary-mgkcj" Jul 13 14:47:04.257540 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.257524 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6c60d2d4-243b-4952-83b9-bb2c462d9e2c-config-volume\") pod \"dns-default-b47lx\" (UID: \"6c60d2d4-243b-4952-83b9-bb2c462d9e2c\") " pod="openshift-dns/dns-default-b47lx" Jul 13 14:47:04.257611 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.257567 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crvnc\" (UniqueName: \"kubernetes.io/projected/d8061d1f-15e7-4f20-a3c6-d3b9b865cb0e-kube-api-access-crvnc\") pod \"ingress-canary-mgkcj\" (UID: \"d8061d1f-15e7-4f20-a3c6-d3b9b865cb0e\") " pod="openshift-ingress-canary/ingress-canary-mgkcj" Jul 13 14:47:04.257665 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.257611 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c60d2d4-243b-4952-83b9-bb2c462d9e2c-metrics-tls\") pod \"dns-default-b47lx\" (UID: \"6c60d2d4-243b-4952-83b9-bb2c462d9e2c\") " pod="openshift-dns/dns-default-b47lx" Jul 13 14:47:04.257665 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.257642 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnq8c\" (UniqueName: \"kubernetes.io/projected/6c60d2d4-243b-4952-83b9-bb2c462d9e2c-kube-api-access-pnq8c\") pod \"dns-default-b47lx\" (UID: \"6c60d2d4-243b-4952-83b9-bb2c462d9e2c\") " pod="openshift-dns/dns-default-b47lx" Jul 13 14:47:04.257755 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.257697 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c60d2d4-243b-4952-83b9-bb2c462d9e2c-tmp-dir\") pod \"dns-default-b47lx\" (UID: \"6c60d2d4-243b-4952-83b9-bb2c462d9e2c\") " pod="openshift-dns/dns-default-b47lx" Jul 13 14:47:04.258010 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:04.257775 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 14:47:04.258135 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:04.258075 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c60d2d4-243b-4952-83b9-bb2c462d9e2c-metrics-tls podName:6c60d2d4-243b-4952-83b9-bb2c462d9e2c nodeName:}" failed. No retries permitted until 2026-07-13 14:47:04.758055171 +0000 UTC m=+36.998332999 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6c60d2d4-243b-4952-83b9-bb2c462d9e2c-metrics-tls") pod "dns-default-b47lx" (UID: "6c60d2d4-243b-4952-83b9-bb2c462d9e2c") : secret "dns-default-metrics-tls" not found Jul 13 14:47:04.258135 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.258073 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6c60d2d4-243b-4952-83b9-bb2c462d9e2c-config-volume\") pod \"dns-default-b47lx\" (UID: \"6c60d2d4-243b-4952-83b9-bb2c462d9e2c\") " pod="openshift-dns/dns-default-b47lx" Jul 13 14:47:04.262055 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.262035 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d8061d1f-15e7-4f20-a3c6-d3b9b865cb0e-cert\") pod \"ingress-canary-mgkcj\" (UID: \"d8061d1f-15e7-4f20-a3c6-d3b9b865cb0e\") " pod="openshift-ingress-canary/ingress-canary-mgkcj" Jul 13 14:47:04.266603 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.266535 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9hk94" Jul 13 14:47:04.266603 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.266552 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:47:04.266821 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.266805 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r82bx" Jul 13 14:47:04.267483 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.267461 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnq8c\" (UniqueName: \"kubernetes.io/projected/6c60d2d4-243b-4952-83b9-bb2c462d9e2c-kube-api-access-pnq8c\") pod \"dns-default-b47lx\" (UID: \"6c60d2d4-243b-4952-83b9-bb2c462d9e2c\") " pod="openshift-dns/dns-default-b47lx" Jul 13 14:47:04.267921 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.267898 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crvnc\" (UniqueName: \"kubernetes.io/projected/d8061d1f-15e7-4f20-a3c6-d3b9b865cb0e-kube-api-access-crvnc\") pod \"ingress-canary-mgkcj\" (UID: \"d8061d1f-15e7-4f20-a3c6-d3b9b865cb0e\") " pod="openshift-ingress-canary/ingress-canary-mgkcj" Jul 13 14:47:04.269309 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.269266 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jvqn2\"" Jul 13 14:47:04.269421 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.269356 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2gzcz\"" Jul 13 14:47:04.269421 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.269382 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 13 14:47:04.269530 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.269493 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 13 14:47:04.269631 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.269613 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 14:47:04.269697 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.269657 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 14:47:04.350560 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.350521 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:04.364532 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.364501 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6bfb696d64-gvxnc"] Jul 13 14:47:04.367225 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.367205 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:47:04.369735 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.369715 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7ljvr\"" Jul 13 14:47:04.370050 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.370028 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 13 14:47:04.370157 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.370137 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 13 14:47:04.370428 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.370410 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 13 14:47:04.375458 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.375404 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 13 14:47:04.382024 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.381997 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6bfb696d64-gvxnc"] Jul 13 14:47:04.414227 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.414194 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mgkcj" Jul 13 14:47:04.459182 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.459153 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-bound-sa-token\") pod \"image-registry-6bfb696d64-gvxnc\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:47:04.459182 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.459192 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2154d6d4-6c28-4077-8ce8-2830c1347168-registry-certificates\") pod \"image-registry-6bfb696d64-gvxnc\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:47:04.459431 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.459217 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2154d6d4-6c28-4077-8ce8-2830c1347168-installation-pull-secrets\") pod \"image-registry-6bfb696d64-gvxnc\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:47:04.459431 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.459267 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2154d6d4-6c28-4077-8ce8-2830c1347168-image-registry-private-configuration\") pod \"image-registry-6bfb696d64-gvxnc\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:47:04.459431 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.459326 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2154d6d4-6c28-4077-8ce8-2830c1347168-ca-trust-extracted\") pod \"image-registry-6bfb696d64-gvxnc\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:47:04.459431 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.459348 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2154d6d4-6c28-4077-8ce8-2830c1347168-trusted-ca\") pod \"image-registry-6bfb696d64-gvxnc\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:47:04.459431 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.459371 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfkkp\" (UniqueName: \"kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-kube-api-access-hfkkp\") pod \"image-registry-6bfb696d64-gvxnc\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:47:04.459679 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.459461 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-registry-tls\") pod \"image-registry-6bfb696d64-gvxnc\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:47:04.462218 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.462188 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-jmpvx"] Jul 13 14:47:04.464426 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.464404 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-jmpvx" Jul 13 14:47:04.468516 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.468491 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 13 14:47:04.468653 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.468616 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 13 14:47:04.468714 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.468665 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 13 14:47:04.469370 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.469350 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-ffs4n\"" Jul 13 14:47:04.469455 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.469417 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 13 14:47:04.479777 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.479751 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-jmpvx"] Jul 13 14:47:04.560305 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.560272 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2154d6d4-6c28-4077-8ce8-2830c1347168-trusted-ca\") pod \"image-registry-6bfb696d64-gvxnc\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:47:04.560305 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.560311 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfkkp\" (UniqueName: \"kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-kube-api-access-hfkkp\") pod \"image-registry-6bfb696d64-gvxnc\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:47:04.560556 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.560347 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94zdr\" (UniqueName: \"kubernetes.io/projected/a7789b1a-5fd7-4e52-93e4-0bf7e0927ab5-kube-api-access-94zdr\") pod \"kube-storage-version-migrator-operator-799cb7f958-jmpvx\" (UID: \"a7789b1a-5fd7-4e52-93e4-0bf7e0927ab5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-jmpvx" Jul 13 14:47:04.560556 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.560396 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a7789b1a-5fd7-4e52-93e4-0bf7e0927ab5-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-jmpvx\" (UID: \"a7789b1a-5fd7-4e52-93e4-0bf7e0927ab5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-jmpvx" Jul 13 14:47:04.560556 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.560425 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a7789b1a-5fd7-4e52-93e4-0bf7e0927ab5-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-jmpvx\" (UID: \"a7789b1a-5fd7-4e52-93e4-0bf7e0927ab5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-jmpvx" Jul 13 14:47:04.560556 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.560481 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-registry-tls\") pod \"image-registry-6bfb696d64-gvxnc\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:47:04.560556 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.560552 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-bound-sa-token\") pod \"image-registry-6bfb696d64-gvxnc\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:47:04.560840 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.560595 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2154d6d4-6c28-4077-8ce8-2830c1347168-registry-certificates\") pod \"image-registry-6bfb696d64-gvxnc\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:47:04.560840 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:04.560617 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 14:47:04.560840 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:04.560645 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bfb696d64-gvxnc: secret "image-registry-tls" not found Jul 13 14:47:04.560840 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:04.560700 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-registry-tls podName:2154d6d4-6c28-4077-8ce8-2830c1347168 nodeName:}" failed. No retries permitted until 2026-07-13 14:47:05.06068331 +0000 UTC m=+37.300961137 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-registry-tls") pod "image-registry-6bfb696d64-gvxnc" (UID: "2154d6d4-6c28-4077-8ce8-2830c1347168") : secret "image-registry-tls" not found Jul 13 14:47:04.560840 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.560619 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2154d6d4-6c28-4077-8ce8-2830c1347168-installation-pull-secrets\") pod \"image-registry-6bfb696d64-gvxnc\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:47:04.560840 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.560799 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2154d6d4-6c28-4077-8ce8-2830c1347168-image-registry-private-configuration\") pod \"image-registry-6bfb696d64-gvxnc\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:47:04.561133 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.560923 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2154d6d4-6c28-4077-8ce8-2830c1347168-ca-trust-extracted\") pod \"image-registry-6bfb696d64-gvxnc\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:47:04.561281 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.561258 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2154d6d4-6c28-4077-8ce8-2830c1347168-ca-trust-extracted\") pod \"image-registry-6bfb696d64-gvxnc\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:47:04.561356 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.561332 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2154d6d4-6c28-4077-8ce8-2830c1347168-registry-certificates\") pod \"image-registry-6bfb696d64-gvxnc\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:47:04.561401 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.561384 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2154d6d4-6c28-4077-8ce8-2830c1347168-trusted-ca\") pod \"image-registry-6bfb696d64-gvxnc\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:47:04.563830 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.563810 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2154d6d4-6c28-4077-8ce8-2830c1347168-installation-pull-secrets\") pod \"image-registry-6bfb696d64-gvxnc\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:47:04.563940 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.563836 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2154d6d4-6c28-4077-8ce8-2830c1347168-image-registry-private-configuration\") pod \"image-registry-6bfb696d64-gvxnc\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:47:04.574403 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.574376 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-bound-sa-token\") pod \"image-registry-6bfb696d64-gvxnc\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:47:04.574758 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.574731 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfkkp\" (UniqueName: \"kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-kube-api-access-hfkkp\") pod \"image-registry-6bfb696d64-gvxnc\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:47:04.661356 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.661316 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94zdr\" (UniqueName: \"kubernetes.io/projected/a7789b1a-5fd7-4e52-93e4-0bf7e0927ab5-kube-api-access-94zdr\") pod \"kube-storage-version-migrator-operator-799cb7f958-jmpvx\" (UID: \"a7789b1a-5fd7-4e52-93e4-0bf7e0927ab5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-jmpvx" Jul 13 14:47:04.661522 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.661379 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a7789b1a-5fd7-4e52-93e4-0bf7e0927ab5-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-jmpvx\" (UID: \"a7789b1a-5fd7-4e52-93e4-0bf7e0927ab5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-jmpvx" Jul 13 14:47:04.661522 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.661410 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a7789b1a-5fd7-4e52-93e4-0bf7e0927ab5-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-jmpvx\" (UID: \"a7789b1a-5fd7-4e52-93e4-0bf7e0927ab5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-jmpvx" Jul 13 14:47:04.662067 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.662043 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a7789b1a-5fd7-4e52-93e4-0bf7e0927ab5-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-jmpvx\" (UID: \"a7789b1a-5fd7-4e52-93e4-0bf7e0927ab5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-jmpvx" Jul 13 14:47:04.663877 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.663854 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a7789b1a-5fd7-4e52-93e4-0bf7e0927ab5-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-jmpvx\" (UID: \"a7789b1a-5fd7-4e52-93e4-0bf7e0927ab5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-jmpvx" Jul 13 14:47:04.675657 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.675625 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94zdr\" (UniqueName: \"kubernetes.io/projected/a7789b1a-5fd7-4e52-93e4-0bf7e0927ab5-kube-api-access-94zdr\") pod \"kube-storage-version-migrator-operator-799cb7f958-jmpvx\" (UID: \"a7789b1a-5fd7-4e52-93e4-0bf7e0927ab5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-jmpvx" Jul 13 14:47:04.762239 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.762205 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c60d2d4-243b-4952-83b9-bb2c462d9e2c-metrics-tls\") pod \"dns-default-b47lx\" (UID: \"6c60d2d4-243b-4952-83b9-bb2c462d9e2c\") " pod="openshift-dns/dns-default-b47lx" Jul 13 14:47:04.762407 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:04.762360 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 14:47:04.762454 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:04.762433 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c60d2d4-243b-4952-83b9-bb2c462d9e2c-metrics-tls podName:6c60d2d4-243b-4952-83b9-bb2c462d9e2c nodeName:}" failed. No retries permitted until 2026-07-13 14:47:05.762417212 +0000 UTC m=+38.002695038 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6c60d2d4-243b-4952-83b9-bb2c462d9e2c-metrics-tls") pod "dns-default-b47lx" (UID: "6c60d2d4-243b-4952-83b9-bb2c462d9e2c") : secret "dns-default-metrics-tls" not found Jul 13 14:47:04.775699 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:04.775669 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-jmpvx" Jul 13 14:47:05.074242 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.070262 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-registry-tls\") pod \"image-registry-6bfb696d64-gvxnc\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:47:05.074242 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:05.070466 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 14:47:05.074242 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:05.070481 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bfb696d64-gvxnc: secret "image-registry-tls" not found Jul 13 14:47:05.074242 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:05.070540 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-registry-tls podName:2154d6d4-6c28-4077-8ce8-2830c1347168 nodeName:}" failed. No retries permitted until 2026-07-13 14:47:06.07052195 +0000 UTC m=+38.310799990 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-registry-tls") pod "image-registry-6bfb696d64-gvxnc" (UID: "2154d6d4-6c28-4077-8ce8-2830c1347168") : secret "image-registry-tls" not found Jul 13 14:47:05.161065 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.161038 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mgkcj"] Jul 13 14:47:05.162092 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.162065 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-jmpvx"] Jul 13 14:47:05.163463 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:47:05.163252 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8061d1f_15e7_4f20_a3c6_d3b9b865cb0e.slice/crio-d9c390db2176154d66f816aec63fcf19f37c553c7b586dae1dc07475ceb8775b WatchSource:0}: Error finding container d9c390db2176154d66f816aec63fcf19f37c553c7b586dae1dc07475ceb8775b: Status 404 returned error can't find the container with id d9c390db2176154d66f816aec63fcf19f37c553c7b586dae1dc07475ceb8775b Jul 13 14:47:05.163635 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:47:05.163614 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7789b1a_5fd7_4e52_93e4_0bf7e0927ab5.slice/crio-9181809fa667a9eca2c0660908b97285ff9a4dcb8262400170e0df56a3622a52 WatchSource:0}: Error finding container 9181809fa667a9eca2c0660908b97285ff9a4dcb8262400170e0df56a3622a52: Status 404 returned error can't find the container with id 9181809fa667a9eca2c0660908b97285ff9a4dcb8262400170e0df56a3622a52 Jul 13 14:47:05.171711 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.171689 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fqm8q"] Jul 13 14:47:05.174307 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.174288 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fqm8q" Jul 13 14:47:05.176847 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.176828 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 13 14:47:05.176925 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.176828 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 13 14:47:05.176987 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.176968 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dhzjh\"" Jul 13 14:47:05.177246 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.177231 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 13 14:47:05.177400 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.177387 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 13 14:47:05.185809 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.185789 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fqm8q"] Jul 13 14:47:05.213536 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.213503 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:05.272016 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.271985 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2r9qp\" (UniqueName: \"kubernetes.io/projected/a3ae3942-f36d-405f-8c8c-124b903d6273-kube-api-access-2r9qp\") pod \"insights-runtime-extractor-fqm8q\" (UID: \"a3ae3942-f36d-405f-8c8c-124b903d6273\") " pod="openshift-insights/insights-runtime-extractor-fqm8q" Jul 13 14:47:05.272136 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.272046 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a3ae3942-f36d-405f-8c8c-124b903d6273-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fqm8q\" (UID: \"a3ae3942-f36d-405f-8c8c-124b903d6273\") " pod="openshift-insights/insights-runtime-extractor-fqm8q" Jul 13 14:47:05.272136 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.272110 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a3ae3942-f36d-405f-8c8c-124b903d6273-crio-socket\") pod \"insights-runtime-extractor-fqm8q\" (UID: \"a3ae3942-f36d-405f-8c8c-124b903d6273\") " pod="openshift-insights/insights-runtime-extractor-fqm8q" Jul 13 14:47:05.272202 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.272138 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3ae3942-f36d-405f-8c8c-124b903d6273-data-volume\") pod \"insights-runtime-extractor-fqm8q\" (UID: \"a3ae3942-f36d-405f-8c8c-124b903d6273\") " pod="openshift-insights/insights-runtime-extractor-fqm8q" Jul 13 14:47:05.272202 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.272168 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a3ae3942-f36d-405f-8c8c-124b903d6273-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fqm8q\" (UID: \"a3ae3942-f36d-405f-8c8c-124b903d6273\") " pod="openshift-insights/insights-runtime-extractor-fqm8q" Jul 13 14:47:05.282061 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.282031 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:05.372665 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.372615 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a3ae3942-f36d-405f-8c8c-124b903d6273-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fqm8q\" (UID: \"a3ae3942-f36d-405f-8c8c-124b903d6273\") " pod="openshift-insights/insights-runtime-extractor-fqm8q" Jul 13 14:47:05.373072 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.372711 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a3ae3942-f36d-405f-8c8c-124b903d6273-crio-socket\") pod \"insights-runtime-extractor-fqm8q\" (UID: \"a3ae3942-f36d-405f-8c8c-124b903d6273\") " pod="openshift-insights/insights-runtime-extractor-fqm8q" Jul 13 14:47:05.373072 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.372730 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3ae3942-f36d-405f-8c8c-124b903d6273-data-volume\") pod \"insights-runtime-extractor-fqm8q\" (UID: \"a3ae3942-f36d-405f-8c8c-124b903d6273\") " pod="openshift-insights/insights-runtime-extractor-fqm8q" Jul 13 14:47:05.373072 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.372760 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a3ae3942-f36d-405f-8c8c-124b903d6273-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fqm8q\" (UID: \"a3ae3942-f36d-405f-8c8c-124b903d6273\") " pod="openshift-insights/insights-runtime-extractor-fqm8q" Jul 13 14:47:05.373072 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:05.372767 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 14:47:05.373072 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.372802 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r9qp\" (UniqueName: \"kubernetes.io/projected/a3ae3942-f36d-405f-8c8c-124b903d6273-kube-api-access-2r9qp\") pod \"insights-runtime-extractor-fqm8q\" (UID: \"a3ae3942-f36d-405f-8c8c-124b903d6273\") " pod="openshift-insights/insights-runtime-extractor-fqm8q" Jul 13 14:47:05.373072 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:05.372859 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3ae3942-f36d-405f-8c8c-124b903d6273-insights-runtime-extractor-tls podName:a3ae3942-f36d-405f-8c8c-124b903d6273 nodeName:}" failed. No retries permitted until 2026-07-13 14:47:05.872833261 +0000 UTC m=+38.113111105 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a3ae3942-f36d-405f-8c8c-124b903d6273-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fqm8q" (UID: "a3ae3942-f36d-405f-8c8c-124b903d6273") : secret "insights-runtime-extractor-tls" not found Jul 13 14:47:05.373072 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:05.372905 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 14:47:05.373072 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.372933 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a3ae3942-f36d-405f-8c8c-124b903d6273-crio-socket\") pod \"insights-runtime-extractor-fqm8q\" (UID: \"a3ae3942-f36d-405f-8c8c-124b903d6273\") " pod="openshift-insights/insights-runtime-extractor-fqm8q" Jul 13 14:47:05.373072 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:05.372974 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a3ae3942-f36d-405f-8c8c-124b903d6273-kube-rbac-proxy-cm podName:a3ae3942-f36d-405f-8c8c-124b903d6273 nodeName:}" failed. No retries permitted until 2026-07-13 14:47:05.872957516 +0000 UTC m=+38.113235363 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a3ae3942-f36d-405f-8c8c-124b903d6273-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fqm8q" (UID: "a3ae3942-f36d-405f-8c8c-124b903d6273") : configmap "kube-rbac-proxy" not found Jul 13 14:47:05.373348 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.373109 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3ae3942-f36d-405f-8c8c-124b903d6273-data-volume\") pod \"insights-runtime-extractor-fqm8q\" (UID: \"a3ae3942-f36d-405f-8c8c-124b903d6273\") " pod="openshift-insights/insights-runtime-extractor-fqm8q" Jul 13 14:47:05.383343 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.383314 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2r9qp\" (UniqueName: \"kubernetes.io/projected/a3ae3942-f36d-405f-8c8c-124b903d6273-kube-api-access-2r9qp\") pod \"insights-runtime-extractor-fqm8q\" (UID: \"a3ae3942-f36d-405f-8c8c-124b903d6273\") " pod="openshift-insights/insights-runtime-extractor-fqm8q" Jul 13 14:47:05.440930 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.440899 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-q47dk"] Jul 13 14:47:05.443204 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.443188 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-q47dk" Jul 13 14:47:05.445403 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.445383 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-trtwm\"" Jul 13 14:47:05.454290 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.454263 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-q47dk"] Jul 13 14:47:05.473139 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.473100 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fbbw\" (UniqueName: \"kubernetes.io/projected/61695e51-b56c-4492-9fd2-5d80603d2eed-kube-api-access-5fbbw\") pod \"network-check-source-869dcfcdcd-q47dk\" (UID: \"61695e51-b56c-4492-9fd2-5d80603d2eed\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-q47dk" Jul 13 14:47:05.574210 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.574170 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fbbw\" (UniqueName: \"kubernetes.io/projected/61695e51-b56c-4492-9fd2-5d80603d2eed-kube-api-access-5fbbw\") pod \"network-check-source-869dcfcdcd-q47dk\" (UID: \"61695e51-b56c-4492-9fd2-5d80603d2eed\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-q47dk" Jul 13 14:47:05.584962 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.584931 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fbbw\" (UniqueName: \"kubernetes.io/projected/61695e51-b56c-4492-9fd2-5d80603d2eed-kube-api-access-5fbbw\") pod \"network-check-source-869dcfcdcd-q47dk\" (UID: \"61695e51-b56c-4492-9fd2-5d80603d2eed\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-q47dk" Jul 13 14:47:05.752017 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.751976 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-q47dk" Jul 13 14:47:05.776385 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.776165 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c60d2d4-243b-4952-83b9-bb2c462d9e2c-metrics-tls\") pod \"dns-default-b47lx\" (UID: \"6c60d2d4-243b-4952-83b9-bb2c462d9e2c\") " pod="openshift-dns/dns-default-b47lx" Jul 13 14:47:05.776518 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:05.776309 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 14:47:05.776609 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:05.776539 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c60d2d4-243b-4952-83b9-bb2c462d9e2c-metrics-tls podName:6c60d2d4-243b-4952-83b9-bb2c462d9e2c nodeName:}" failed. No retries permitted until 2026-07-13 14:47:07.776513054 +0000 UTC m=+40.016790897 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6c60d2d4-243b-4952-83b9-bb2c462d9e2c-metrics-tls") pod "dns-default-b47lx" (UID: "6c60d2d4-243b-4952-83b9-bb2c462d9e2c") : secret "dns-default-metrics-tls" not found Jul 13 14:47:05.811299 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.811199 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-jmpvx" event={"ID":"a7789b1a-5fd7-4e52-93e4-0bf7e0927ab5","Type":"ContainerStarted","Data":"9181809fa667a9eca2c0660908b97285ff9a4dcb8262400170e0df56a3622a52"} Jul 13 14:47:05.815604 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.815545 2568 generic.go:356] "Generic (PLEG): container finished" podID="87cdb037-6b66-416c-95a7-08332490b5a3" containerID="2dcd80ddd866aa8248213c64cda98f90f70ef6ed874e96e0a0f45a53ab085679" exitCode=0 Jul 13 14:47:05.815730 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.815650 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qg9jx" event={"ID":"87cdb037-6b66-416c-95a7-08332490b5a3","Type":"ContainerDied","Data":"2dcd80ddd866aa8248213c64cda98f90f70ef6ed874e96e0a0f45a53ab085679"} Jul 13 14:47:05.817560 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.817506 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mgkcj" event={"ID":"d8061d1f-15e7-4f20-a3c6-d3b9b865cb0e","Type":"ContainerStarted","Data":"d9c390db2176154d66f816aec63fcf19f37c553c7b586dae1dc07475ceb8775b"} Jul 13 14:47:05.878625 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.877853 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a3ae3942-f36d-405f-8c8c-124b903d6273-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fqm8q\" (UID: \"a3ae3942-f36d-405f-8c8c-124b903d6273\") " pod="openshift-insights/insights-runtime-extractor-fqm8q" Jul 13 14:47:05.878625 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.877970 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a3ae3942-f36d-405f-8c8c-124b903d6273-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fqm8q\" (UID: \"a3ae3942-f36d-405f-8c8c-124b903d6273\") " pod="openshift-insights/insights-runtime-extractor-fqm8q" Jul 13 14:47:05.878625 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:05.878007 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 14:47:05.878625 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:05.878075 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3ae3942-f36d-405f-8c8c-124b903d6273-insights-runtime-extractor-tls podName:a3ae3942-f36d-405f-8c8c-124b903d6273 nodeName:}" failed. No retries permitted until 2026-07-13 14:47:06.878054391 +0000 UTC m=+39.118332242 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a3ae3942-f36d-405f-8c8c-124b903d6273-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fqm8q" (UID: "a3ae3942-f36d-405f-8c8c-124b903d6273") : secret "insights-runtime-extractor-tls" not found Jul 13 14:47:05.878625 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:05.878438 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 14:47:05.878625 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:05.878503 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a3ae3942-f36d-405f-8c8c-124b903d6273-kube-rbac-proxy-cm podName:a3ae3942-f36d-405f-8c8c-124b903d6273 nodeName:}" failed. No retries permitted until 2026-07-13 14:47:06.878484683 +0000 UTC m=+39.118762512 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a3ae3942-f36d-405f-8c8c-124b903d6273-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fqm8q" (UID: "a3ae3942-f36d-405f-8c8c-124b903d6273") : configmap "kube-rbac-proxy" not found Jul 13 14:47:05.902115 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:05.902084 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-q47dk"] Jul 13 14:47:05.908046 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:47:05.908003 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61695e51_b56c_4492_9fd2_5d80603d2eed.slice/crio-5a6285cf468ae393471d4b8e0b001b3bd6a8f28e76ca1d7a45569ab948b230d6 WatchSource:0}: Error finding container 5a6285cf468ae393471d4b8e0b001b3bd6a8f28e76ca1d7a45569ab948b230d6: Status 404 returned error can't find the container with id 5a6285cf468ae393471d4b8e0b001b3bd6a8f28e76ca1d7a45569ab948b230d6 Jul 13 14:47:06.079493 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:06.079376 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-registry-tls\") pod \"image-registry-6bfb696d64-gvxnc\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:47:06.079719 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:06.079560 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 14:47:06.079719 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:06.079598 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bfb696d64-gvxnc: secret "image-registry-tls" not found Jul 13 14:47:06.079719 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:06.079666 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-registry-tls podName:2154d6d4-6c28-4077-8ce8-2830c1347168 nodeName:}" failed. No retries permitted until 2026-07-13 14:47:08.079645915 +0000 UTC m=+40.319923762 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-registry-tls") pod "image-registry-6bfb696d64-gvxnc" (UID: "2154d6d4-6c28-4077-8ce8-2830c1347168") : secret "image-registry-tls" not found Jul 13 14:47:06.362213 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:06.362124 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-k6pzr"] Jul 13 14:47:06.364775 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:06.364743 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-k6pzr" Jul 13 14:47:06.367241 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:06.367207 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 13 14:47:06.367374 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:06.367304 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 13 14:47:06.367443 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:06.367210 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 13 14:47:06.367529 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:06.367210 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 13 14:47:06.367670 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:06.367559 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-n98sh\"" Jul 13 14:47:06.375494 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:06.375466 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-k6pzr"] Jul 13 14:47:06.483589 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:06.483534 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9ccf4712-918a-4d3a-808e-d6162b7a0f51-config\") pod \"service-ca-operator-7695fc6c8c-k6pzr\" (UID: \"9ccf4712-918a-4d3a-808e-d6162b7a0f51\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-k6pzr" Jul 13 14:47:06.483769 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:06.483665 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnv2k\" (UniqueName: \"kubernetes.io/projected/9ccf4712-918a-4d3a-808e-d6162b7a0f51-kube-api-access-pnv2k\") pod \"service-ca-operator-7695fc6c8c-k6pzr\" (UID: \"9ccf4712-918a-4d3a-808e-d6162b7a0f51\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-k6pzr" Jul 13 14:47:06.483769 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:06.483757 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9ccf4712-918a-4d3a-808e-d6162b7a0f51-serving-cert\") pod \"service-ca-operator-7695fc6c8c-k6pzr\" (UID: \"9ccf4712-918a-4d3a-808e-d6162b7a0f51\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-k6pzr" Jul 13 14:47:06.584337 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:06.584178 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9ccf4712-918a-4d3a-808e-d6162b7a0f51-serving-cert\") pod \"service-ca-operator-7695fc6c8c-k6pzr\" (UID: \"9ccf4712-918a-4d3a-808e-d6162b7a0f51\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-k6pzr" Jul 13 14:47:06.584337 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:06.584275 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9ccf4712-918a-4d3a-808e-d6162b7a0f51-config\") pod \"service-ca-operator-7695fc6c8c-k6pzr\" (UID: \"9ccf4712-918a-4d3a-808e-d6162b7a0f51\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-k6pzr" Jul 13 14:47:06.584337 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:06.584334 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnv2k\" (UniqueName: \"kubernetes.io/projected/9ccf4712-918a-4d3a-808e-d6162b7a0f51-kube-api-access-pnv2k\") pod \"service-ca-operator-7695fc6c8c-k6pzr\" (UID: \"9ccf4712-918a-4d3a-808e-d6162b7a0f51\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-k6pzr" Jul 13 14:47:06.585272 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:06.585213 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9ccf4712-918a-4d3a-808e-d6162b7a0f51-config\") pod \"service-ca-operator-7695fc6c8c-k6pzr\" (UID: \"9ccf4712-918a-4d3a-808e-d6162b7a0f51\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-k6pzr" Jul 13 14:47:06.587697 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:06.587650 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9ccf4712-918a-4d3a-808e-d6162b7a0f51-serving-cert\") pod \"service-ca-operator-7695fc6c8c-k6pzr\" (UID: \"9ccf4712-918a-4d3a-808e-d6162b7a0f51\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-k6pzr" Jul 13 14:47:06.596508 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:06.596477 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnv2k\" (UniqueName: \"kubernetes.io/projected/9ccf4712-918a-4d3a-808e-d6162b7a0f51-kube-api-access-pnv2k\") pod \"service-ca-operator-7695fc6c8c-k6pzr\" (UID: \"9ccf4712-918a-4d3a-808e-d6162b7a0f51\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-k6pzr" Jul 13 14:47:06.676788 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:06.676705 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-k6pzr" Jul 13 14:47:06.822034 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:06.821987 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-q47dk" event={"ID":"61695e51-b56c-4492-9fd2-5d80603d2eed","Type":"ContainerStarted","Data":"5a6285cf468ae393471d4b8e0b001b3bd6a8f28e76ca1d7a45569ab948b230d6"} Jul 13 14:47:06.826879 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:06.826818 2568 generic.go:356] "Generic (PLEG): container finished" podID="87cdb037-6b66-416c-95a7-08332490b5a3" containerID="d886ae556f80052ab9ffc59f0d7c36aa675493ad46bb6911d18cf97d71bb874b" exitCode=0 Jul 13 14:47:06.826879 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:06.826871 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qg9jx" event={"ID":"87cdb037-6b66-416c-95a7-08332490b5a3","Type":"ContainerDied","Data":"d886ae556f80052ab9ffc59f0d7c36aa675493ad46bb6911d18cf97d71bb874b"} Jul 13 14:47:06.830792 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:06.830704 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-k6pzr"] Jul 13 14:47:06.887953 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:06.886972 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a3ae3942-f36d-405f-8c8c-124b903d6273-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fqm8q\" (UID: \"a3ae3942-f36d-405f-8c8c-124b903d6273\") " pod="openshift-insights/insights-runtime-extractor-fqm8q" Jul 13 14:47:06.887953 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:06.887130 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 14:47:06.887953 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:06.887151 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a3ae3942-f36d-405f-8c8c-124b903d6273-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fqm8q\" (UID: \"a3ae3942-f36d-405f-8c8c-124b903d6273\") " pod="openshift-insights/insights-runtime-extractor-fqm8q" Jul 13 14:47:06.887953 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:06.887203 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3ae3942-f36d-405f-8c8c-124b903d6273-insights-runtime-extractor-tls podName:a3ae3942-f36d-405f-8c8c-124b903d6273 nodeName:}" failed. No retries permitted until 2026-07-13 14:47:08.887181761 +0000 UTC m=+41.127459591 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a3ae3942-f36d-405f-8c8c-124b903d6273-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fqm8q" (UID: "a3ae3942-f36d-405f-8c8c-124b903d6273") : secret "insights-runtime-extractor-tls" not found Jul 13 14:47:06.887953 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:06.887230 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 14:47:06.887953 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:06.887285 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a3ae3942-f36d-405f-8c8c-124b903d6273-kube-rbac-proxy-cm podName:a3ae3942-f36d-405f-8c8c-124b903d6273 nodeName:}" failed. No retries permitted until 2026-07-13 14:47:08.887267068 +0000 UTC m=+41.127544897 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a3ae3942-f36d-405f-8c8c-124b903d6273-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fqm8q" (UID: "a3ae3942-f36d-405f-8c8c-124b903d6273") : configmap "kube-rbac-proxy" not found Jul 13 14:47:07.353767 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:07.353663 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xz7wd_d05cd493-5f17-4976-b2e5-404431d3389a/dns-node-resolver/0.log" Jul 13 14:47:07.737912 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:47:07.737831 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ccf4712_918a_4d3a_808e_d6162b7a0f51.slice/crio-fce132b998a4e3add4bae41192ea717222a24040a9f341b5adb3d95fb718db21 WatchSource:0}: Error finding container fce132b998a4e3add4bae41192ea717222a24040a9f341b5adb3d95fb718db21: Status 404 returned error can't find the container with id fce132b998a4e3add4bae41192ea717222a24040a9f341b5adb3d95fb718db21 Jul 13 14:47:07.795252 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:07.795036 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c60d2d4-243b-4952-83b9-bb2c462d9e2c-metrics-tls\") pod \"dns-default-b47lx\" (UID: \"6c60d2d4-243b-4952-83b9-bb2c462d9e2c\") " pod="openshift-dns/dns-default-b47lx" Jul 13 14:47:07.795403 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:07.795224 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 14:47:07.795403 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:07.795366 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c60d2d4-243b-4952-83b9-bb2c462d9e2c-metrics-tls podName:6c60d2d4-243b-4952-83b9-bb2c462d9e2c nodeName:}" failed. No retries permitted until 2026-07-13 14:47:11.795343906 +0000 UTC m=+44.035621736 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6c60d2d4-243b-4952-83b9-bb2c462d9e2c-metrics-tls") pod "dns-default-b47lx" (UID: "6c60d2d4-243b-4952-83b9-bb2c462d9e2c") : secret "dns-default-metrics-tls" not found Jul 13 14:47:07.831472 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:07.831435 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-k6pzr" event={"ID":"9ccf4712-918a-4d3a-808e-d6162b7a0f51","Type":"ContainerStarted","Data":"fce132b998a4e3add4bae41192ea717222a24040a9f341b5adb3d95fb718db21"} Jul 13 14:47:08.098746 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:08.098706 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-registry-tls\") pod \"image-registry-6bfb696d64-gvxnc\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:47:08.098915 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:08.098847 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 14:47:08.098915 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:08.098864 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bfb696d64-gvxnc: secret "image-registry-tls" not found Jul 13 14:47:08.099028 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:08.098927 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-registry-tls podName:2154d6d4-6c28-4077-8ce8-2830c1347168 nodeName:}" failed. No retries permitted until 2026-07-13 14:47:12.098907837 +0000 UTC m=+44.339185666 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-registry-tls") pod "image-registry-6bfb696d64-gvxnc" (UID: "2154d6d4-6c28-4077-8ce8-2830c1347168") : secret "image-registry-tls" not found Jul 13 14:47:08.156743 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:08.156711 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qfs4l_df5174e4-503e-475b-ab01-20dd9f45124d/node-ca/0.log" Jul 13 14:47:08.838432 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:08.838395 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qg9jx" event={"ID":"87cdb037-6b66-416c-95a7-08332490b5a3","Type":"ContainerStarted","Data":"d39af1bd333d0e6772be20a4a5d3926502824a067cc9084f236700949bd8e6a0"} Jul 13 14:47:08.840563 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:08.840535 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mgkcj" event={"ID":"d8061d1f-15e7-4f20-a3c6-d3b9b865cb0e","Type":"ContainerStarted","Data":"a1852925fb98ed04eb16ba416fc1cc0b9d59ff86c0bce683ac80d61670bc42cf"} Jul 13 14:47:08.864528 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:08.864463 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qg9jx" podStartSLOduration=8.767560114 podStartE2EDuration="39.864442265s" totalImagesPullingTime="31.096882151s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 14:46:29 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:47:08.86384592 +0000 UTC m=+41.104123798" watchObservedRunningTime="2026-07-13 14:47:08.864442265 +0000 UTC m=+41.104720115" Jul 13 14:47:08.882051 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:08.881992 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-mgkcj" podStartSLOduration=2.226673766 podStartE2EDuration="4.88196899s" totalImagesPullingTime="2.655295224s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 14:47:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:47:08.880731595 +0000 UTC m=+41.121009444" watchObservedRunningTime="2026-07-13 14:47:08.88196899 +0000 UTC m=+41.122246843" Jul 13 14:47:08.905379 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:08.905336 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a3ae3942-f36d-405f-8c8c-124b903d6273-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fqm8q\" (UID: \"a3ae3942-f36d-405f-8c8c-124b903d6273\") " pod="openshift-insights/insights-runtime-extractor-fqm8q" Jul 13 14:47:08.905542 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:08.905438 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 14:47:08.905542 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:08.905502 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a3ae3942-f36d-405f-8c8c-124b903d6273-kube-rbac-proxy-cm podName:a3ae3942-f36d-405f-8c8c-124b903d6273 nodeName:}" failed. No retries permitted until 2026-07-13 14:47:12.90548595 +0000 UTC m=+45.145763777 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a3ae3942-f36d-405f-8c8c-124b903d6273-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fqm8q" (UID: "a3ae3942-f36d-405f-8c8c-124b903d6273") : configmap "kube-rbac-proxy" not found Jul 13 14:47:08.905681 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:08.905636 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a3ae3942-f36d-405f-8c8c-124b903d6273-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fqm8q\" (UID: \"a3ae3942-f36d-405f-8c8c-124b903d6273\") " pod="openshift-insights/insights-runtime-extractor-fqm8q" Jul 13 14:47:08.905982 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:08.905960 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 14:47:08.906080 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:08.906019 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3ae3942-f36d-405f-8c8c-124b903d6273-insights-runtime-extractor-tls podName:a3ae3942-f36d-405f-8c8c-124b903d6273 nodeName:}" failed. No retries permitted until 2026-07-13 14:47:12.906004736 +0000 UTC m=+45.146282579 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a3ae3942-f36d-405f-8c8c-124b903d6273-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fqm8q" (UID: "a3ae3942-f36d-405f-8c8c-124b903d6273") : secret "insights-runtime-extractor-tls" not found Jul 13 14:47:09.064508 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:09.064462 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:09.157626 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:09.157552 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mgkcj_d8061d1f-15e7-4f20-a3c6-d3b9b865cb0e/serve-healthcheck-canary/0.log" Jul 13 14:47:09.177398 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:09.177356 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:10.221520 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:10.221455 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:10.288161 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:10.288126 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:10.847795 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:10.847756 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-jmpvx" event={"ID":"a7789b1a-5fd7-4e52-93e4-0bf7e0927ab5","Type":"ContainerStarted","Data":"37cf2f728c71c0aae8f17fa30f6821c1185fae690a4a0974921f3bd4f2505931"} Jul 13 14:47:10.849701 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:10.849670 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-q47dk" event={"ID":"61695e51-b56c-4492-9fd2-5d80603d2eed","Type":"ContainerStarted","Data":"0f6c8f3173a903888ba0baa0dbe70bbdadee389c6677e523398a2738b20f172e"} Jul 13 14:47:10.865398 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:10.865351 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-jmpvx" podStartSLOduration=2.274736285 podStartE2EDuration="6.865330996s" totalImagesPullingTime="4.590594711s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 14:47:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:47:10.864213045 +0000 UTC m=+43.104490898" watchObservedRunningTime="2026-07-13 14:47:10.865330996 +0000 UTC m=+43.105608846" Jul 13 14:47:10.880029 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:10.879976 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-q47dk" podStartSLOduration=2.03996785 podStartE2EDuration="5.879960569s" totalImagesPullingTime="3.839992719s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 14:47:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:47:10.878760366 +0000 UTC m=+43.119038217" watchObservedRunningTime="2026-07-13 14:47:10.879960569 +0000 UTC m=+43.120238417" Jul 13 14:47:11.830170 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:11.830136 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c60d2d4-243b-4952-83b9-bb2c462d9e2c-metrics-tls\") pod \"dns-default-b47lx\" (UID: \"6c60d2d4-243b-4952-83b9-bb2c462d9e2c\") " pod="openshift-dns/dns-default-b47lx" Jul 13 14:47:11.830654 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:11.830308 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 14:47:11.830654 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:11.830404 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c60d2d4-243b-4952-83b9-bb2c462d9e2c-metrics-tls podName:6c60d2d4-243b-4952-83b9-bb2c462d9e2c nodeName:}" failed. No retries permitted until 2026-07-13 14:47:19.83038152 +0000 UTC m=+52.070659370 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6c60d2d4-243b-4952-83b9-bb2c462d9e2c-metrics-tls") pod "dns-default-b47lx" (UID: "6c60d2d4-243b-4952-83b9-bb2c462d9e2c") : secret "dns-default-metrics-tls" not found Jul 13 14:47:11.894173 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:11.894138 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:11.986752 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:11.986722 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:12.087851 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:12.087759 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:12.132938 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:12.132903 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-registry-tls\") pod \"image-registry-6bfb696d64-gvxnc\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:47:12.133130 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:12.133054 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 14:47:12.133130 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:12.133070 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bfb696d64-gvxnc: secret "image-registry-tls" not found Jul 13 14:47:12.133130 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:12.133123 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-registry-tls podName:2154d6d4-6c28-4077-8ce8-2830c1347168 nodeName:}" failed. No retries permitted until 2026-07-13 14:47:20.133107618 +0000 UTC m=+52.373385450 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-registry-tls") pod "image-registry-6bfb696d64-gvxnc" (UID: "2154d6d4-6c28-4077-8ce8-2830c1347168") : secret "image-registry-tls" not found Jul 13 14:47:12.856990 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:12.856949 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-k6pzr" event={"ID":"9ccf4712-918a-4d3a-808e-d6162b7a0f51","Type":"ContainerStarted","Data":"cfbd7916e99a0dcc0d3f02fa00f31c0d8cf6c1a85c0a733df62e3f14f81ae39c"} Jul 13 14:47:12.874114 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:12.874062 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-k6pzr" podStartSLOduration=2.234650561 podStartE2EDuration="6.874047578s" totalImagesPullingTime="4.639397017s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 14:47:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:47:12.872926251 +0000 UTC m=+45.113204100" watchObservedRunningTime="2026-07-13 14:47:12.874047578 +0000 UTC m=+45.114325427" Jul 13 14:47:12.938893 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:12.938851 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a3ae3942-f36d-405f-8c8c-124b903d6273-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fqm8q\" (UID: \"a3ae3942-f36d-405f-8c8c-124b903d6273\") " pod="openshift-insights/insights-runtime-extractor-fqm8q" Jul 13 14:47:12.939055 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:12.939009 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 14:47:12.939130 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:12.939066 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a3ae3942-f36d-405f-8c8c-124b903d6273-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fqm8q\" (UID: \"a3ae3942-f36d-405f-8c8c-124b903d6273\") " pod="openshift-insights/insights-runtime-extractor-fqm8q" Jul 13 14:47:12.939130 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:12.939081 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3ae3942-f36d-405f-8c8c-124b903d6273-insights-runtime-extractor-tls podName:a3ae3942-f36d-405f-8c8c-124b903d6273 nodeName:}" failed. No retries permitted until 2026-07-13 14:47:20.939062461 +0000 UTC m=+53.179340305 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a3ae3942-f36d-405f-8c8c-124b903d6273-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fqm8q" (UID: "a3ae3942-f36d-405f-8c8c-124b903d6273") : secret "insights-runtime-extractor-tls" not found Jul 13 14:47:12.939130 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:12.939116 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 14:47:12.939285 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:12.939153 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a3ae3942-f36d-405f-8c8c-124b903d6273-kube-rbac-proxy-cm podName:a3ae3942-f36d-405f-8c8c-124b903d6273 nodeName:}" failed. No retries permitted until 2026-07-13 14:47:20.939142313 +0000 UTC m=+53.179420156 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a3ae3942-f36d-405f-8c8c-124b903d6273-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fqm8q" (UID: "a3ae3942-f36d-405f-8c8c-124b903d6273") : configmap "kube-rbac-proxy" not found Jul 13 14:47:13.772190 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:13.772154 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:13.873530 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:13.873488 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:14.210318 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:14.210283 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:15.157413 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:15.157385 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:15.238564 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:15.238488 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:15.711538 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:15.711503 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:15.773137 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:15.773103 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:16.111887 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:16.111849 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:16.177949 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:16.177915 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:17.063565 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:17.063531 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:18.343741 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:18.343704 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:19.893325 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:19.893290 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c60d2d4-243b-4952-83b9-bb2c462d9e2c-metrics-tls\") pod \"dns-default-b47lx\" (UID: \"6c60d2d4-243b-4952-83b9-bb2c462d9e2c\") " pod="openshift-dns/dns-default-b47lx" Jul 13 14:47:19.893732 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:19.893441 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 14:47:19.893732 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:19.893510 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c60d2d4-243b-4952-83b9-bb2c462d9e2c-metrics-tls podName:6c60d2d4-243b-4952-83b9-bb2c462d9e2c nodeName:}" failed. No retries permitted until 2026-07-13 14:47:35.893491625 +0000 UTC m=+68.133769452 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6c60d2d4-243b-4952-83b9-bb2c462d9e2c-metrics-tls") pod "dns-default-b47lx" (UID: "6c60d2d4-243b-4952-83b9-bb2c462d9e2c") : secret "dns-default-metrics-tls" not found Jul 13 14:47:20.006943 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:20.006907 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:20.195722 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:20.195624 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-registry-tls\") pod \"image-registry-6bfb696d64-gvxnc\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:47:20.195896 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:20.195764 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 14:47:20.195896 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:20.195789 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bfb696d64-gvxnc: secret "image-registry-tls" not found Jul 13 14:47:20.195896 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:20.195857 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-registry-tls podName:2154d6d4-6c28-4077-8ce8-2830c1347168 nodeName:}" failed. No retries permitted until 2026-07-13 14:47:36.195836452 +0000 UTC m=+68.436114297 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-registry-tls") pod "image-registry-6bfb696d64-gvxnc" (UID: "2154d6d4-6c28-4077-8ce8-2830c1347168") : secret "image-registry-tls" not found Jul 13 14:47:20.618873 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:20.618836 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:20.685026 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:20.684992 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:21.002059 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:21.001958 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a3ae3942-f36d-405f-8c8c-124b903d6273-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fqm8q\" (UID: \"a3ae3942-f36d-405f-8c8c-124b903d6273\") " pod="openshift-insights/insights-runtime-extractor-fqm8q" Jul 13 14:47:21.002510 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:21.002063 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a3ae3942-f36d-405f-8c8c-124b903d6273-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fqm8q\" (UID: \"a3ae3942-f36d-405f-8c8c-124b903d6273\") " pod="openshift-insights/insights-runtime-extractor-fqm8q" Jul 13 14:47:21.002510 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:21.002243 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 14:47:21.002510 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:21.002326 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3ae3942-f36d-405f-8c8c-124b903d6273-insights-runtime-extractor-tls podName:a3ae3942-f36d-405f-8c8c-124b903d6273 nodeName:}" failed. No retries permitted until 2026-07-13 14:47:37.002303849 +0000 UTC m=+69.242581692 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a3ae3942-f36d-405f-8c8c-124b903d6273-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fqm8q" (UID: "a3ae3942-f36d-405f-8c8c-124b903d6273") : secret "insights-runtime-extractor-tls" not found Jul 13 14:47:21.002725 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:21.002540 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a3ae3942-f36d-405f-8c8c-124b903d6273-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fqm8q\" (UID: \"a3ae3942-f36d-405f-8c8c-124b903d6273\") " pod="openshift-insights/insights-runtime-extractor-fqm8q" Jul 13 14:47:21.585057 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:21.585020 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:21.996245 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:21.996160 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:22.194290 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:22.194259 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:22.887645 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:22.887609 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:22.988891 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:22.988854 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:23.495408 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:23.495374 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:23.596676 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:23.596642 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:23.655807 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:23.655775 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:23.786508 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:23.786476 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:23.994746 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:23.994712 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:24.189007 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:24.188926 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:24.387927 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:24.387897 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:24.447727 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:24.447650 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:24.649184 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:24.649153 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:25.947944 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:25.947906 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:26.047891 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:26.047858 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:26.148518 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:26.148485 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:26.247068 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:26.246996 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:26.349798 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:26.349763 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:26.615097 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:26.615065 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:26.716237 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:26.716201 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:26.824442 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:26.824406 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:26.886613 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:26.886521 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:27.117948 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:27.117912 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:27.221218 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:27.221134 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:27.318290 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:27.318252 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:27.788830 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:27.788796 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:27.934859 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:27.934821 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:28.134752 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:28.134667 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:28.355988 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:28.355955 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:28.422605 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:28.422514 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:28.594780 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:28.594746 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:28.806589 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:28.806553 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xgqvx" Jul 13 14:47:28.998765 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:28.998705 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:29.193988 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:29.193899 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:29.785805 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:29.785750 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:30.191944 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:30.191847 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:30.387438 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:30.387403 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:30.593265 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:30.593232 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:30.990761 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:30.990673 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:32.364875 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:32.364836 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:32.611382 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:32.611346 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:34.417302 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:34.417268 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqbsk\" (UniqueName: \"kubernetes.io/projected/191cfe54-82c1-4694-92fd-3b93bb1783fd-kube-api-access-pqbsk\") pod \"network-check-target-r82bx\" (UID: \"191cfe54-82c1-4694-92fd-3b93bb1783fd\") " pod="openshift-network-diagnostics/network-check-target-r82bx" Jul 13 14:47:34.421316 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:34.421290 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqbsk\" (UniqueName: \"kubernetes.io/projected/191cfe54-82c1-4694-92fd-3b93bb1783fd-kube-api-access-pqbsk\") pod \"network-check-target-r82bx\" (UID: \"191cfe54-82c1-4694-92fd-3b93bb1783fd\") " pod="openshift-network-diagnostics/network-check-target-r82bx" Jul 13 14:47:34.518058 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:34.518017 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30b24206-b58d-4409-b6e1-1dec743e754c-metrics-certs\") pod \"network-metrics-daemon-9hk94\" (UID: \"30b24206-b58d-4409-b6e1-1dec743e754c\") " pod="openshift-multus/network-metrics-daemon-9hk94" Jul 13 14:47:34.518255 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:34.518086 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/764568c4-93b4-4bf4-a627-a26e55610c81-original-pull-secret\") pod \"global-pull-secret-syncer-km6w9\" (UID: \"764568c4-93b4-4bf4-a627-a26e55610c81\") " pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:47:34.520707 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:34.520687 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 14:47:34.521527 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:34.521510 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 14:47:34.530508 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:34.530479 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30b24206-b58d-4409-b6e1-1dec743e754c-metrics-certs\") pod \"network-metrics-daemon-9hk94\" (UID: \"30b24206-b58d-4409-b6e1-1dec743e754c\") " pod="openshift-multus/network-metrics-daemon-9hk94" Jul 13 14:47:34.530658 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:34.530597 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/764568c4-93b4-4bf4-a627-a26e55610c81-original-pull-secret\") pod \"global-pull-secret-syncer-km6w9\" (UID: \"764568c4-93b4-4bf4-a627-a26e55610c81\") " pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:47:34.587900 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:34.587868 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2gzcz\"" Jul 13 14:47:34.595151 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:34.595121 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9hk94" Jul 13 14:47:34.595291 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:34.595133 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jvqn2\"" Jul 13 14:47:34.598757 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:34.598740 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-km6w9" Jul 13 14:47:34.603278 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:34.603262 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r82bx" Jul 13 14:47:34.738007 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:34.737859 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9hk94"] Jul 13 14:47:34.910342 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:34.910306 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9hk94" event={"ID":"30b24206-b58d-4409-b6e1-1dec743e754c","Type":"ContainerStarted","Data":"c377aecaaf73a1fba2f445a7debc100697bf8d3de96f5957b24e6ef6221600ee"} Jul 13 14:47:34.968728 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:34.968657 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r82bx"] Jul 13 14:47:34.971667 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:47:34.971634 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod191cfe54_82c1_4694_92fd_3b93bb1783fd.slice/crio-f8cf0ca0b487d9905ae6629df790a562d1784ed27fa3971efa217346a169ad1c WatchSource:0}: Error finding container f8cf0ca0b487d9905ae6629df790a562d1784ed27fa3971efa217346a169ad1c: Status 404 returned error can't find the container with id f8cf0ca0b487d9905ae6629df790a562d1784ed27fa3971efa217346a169ad1c Jul 13 14:47:34.974148 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:34.974124 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-km6w9"] Jul 13 14:47:34.976822 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:47:34.976787 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod764568c4_93b4_4bf4_a627_a26e55610c81.slice/crio-b1a6592425dc2ff655ecc3853623af8c9c1047d9b7fedcfc9244ef8721103c19 WatchSource:0}: Error finding container b1a6592425dc2ff655ecc3853623af8c9c1047d9b7fedcfc9244ef8721103c19: Status 404 returned error can't find the container with id b1a6592425dc2ff655ecc3853623af8c9c1047d9b7fedcfc9244ef8721103c19 Jul 13 14:47:35.226963 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:35.226878 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:35.430128 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:35.430092 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:35.771656 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:35.771620 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:35.915824 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:35.915783 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9hk94" event={"ID":"30b24206-b58d-4409-b6e1-1dec743e754c","Type":"ContainerStarted","Data":"9330227b5ab71e460f4ba3ade650d4e3bf3d890a2d1c1ab090d3595a836824a6"} Jul 13 14:47:35.917327 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:35.917285 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-km6w9" event={"ID":"764568c4-93b4-4bf4-a627-a26e55610c81","Type":"ContainerStarted","Data":"b1a6592425dc2ff655ecc3853623af8c9c1047d9b7fedcfc9244ef8721103c19"} Jul 13 14:47:35.919321 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:35.919288 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r82bx" event={"ID":"191cfe54-82c1-4694-92fd-3b93bb1783fd","Type":"ContainerStarted","Data":"4b758d741d61fd28efb8e23ece5689dfb9a2901a0d9acf7159e2f0f44a84d0f0"} Jul 13 14:47:35.919439 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:35.919341 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r82bx" event={"ID":"191cfe54-82c1-4694-92fd-3b93bb1783fd","Type":"ContainerStarted","Data":"f8cf0ca0b487d9905ae6629df790a562d1784ed27fa3971efa217346a169ad1c"} Jul 13 14:47:35.919585 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:35.919549 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-r82bx" Jul 13 14:47:35.927279 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:35.927252 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c60d2d4-243b-4952-83b9-bb2c462d9e2c-metrics-tls\") pod \"dns-default-b47lx\" (UID: \"6c60d2d4-243b-4952-83b9-bb2c462d9e2c\") " pod="openshift-dns/dns-default-b47lx" Jul 13 14:47:35.929667 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:35.929646 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c60d2d4-243b-4952-83b9-bb2c462d9e2c-metrics-tls\") pod \"dns-default-b47lx\" (UID: \"6c60d2d4-243b-4952-83b9-bb2c462d9e2c\") " pod="openshift-dns/dns-default-b47lx" Jul 13 14:47:35.939187 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:35.939134 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-r82bx" podStartSLOduration=66.939115018 podStartE2EDuration="1m6.939115018s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 14:46:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:47:35.937984339 +0000 UTC m=+68.178262188" watchObservedRunningTime="2026-07-13 14:47:35.939115018 +0000 UTC m=+68.179392868" Jul 13 14:47:36.210416 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:36.210385 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-k8vvm\"" Jul 13 14:47:36.217581 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:36.217544 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b47lx" Jul 13 14:47:36.230125 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:36.230092 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-registry-tls\") pod \"image-registry-6bfb696d64-gvxnc\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:47:36.232659 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:36.232628 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-registry-tls\") pod \"image-registry-6bfb696d64-gvxnc\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:47:36.392156 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:36.392118 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b47lx"] Jul 13 14:47:36.395743 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:47:36.395716 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c60d2d4_243b_4952_83b9_bb2c462d9e2c.slice/crio-719ee42190610842c28b26112c7b315df4d098e3e1bf40f4139cfdd763c3869d WatchSource:0}: Error finding container 719ee42190610842c28b26112c7b315df4d098e3e1bf40f4139cfdd763c3869d: Status 404 returned error can't find the container with id 719ee42190610842c28b26112c7b315df4d098e3e1bf40f4139cfdd763c3869d Jul 13 14:47:36.481390 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:36.481353 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7ljvr\"" Jul 13 14:47:36.488595 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:36.488504 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:47:36.676436 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:47:36.676406 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2154d6d4_6c28_4077_8ce8_2830c1347168.slice/crio-af8f083935c2b59e3e9e50de52d81fbd3ac4743fe5c4dfbab4518bdb4609d480 WatchSource:0}: Error finding container af8f083935c2b59e3e9e50de52d81fbd3ac4743fe5c4dfbab4518bdb4609d480: Status 404 returned error can't find the container with id af8f083935c2b59e3e9e50de52d81fbd3ac4743fe5c4dfbab4518bdb4609d480 Jul 13 14:47:36.676882 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:36.676860 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6bfb696d64-gvxnc"] Jul 13 14:47:36.925255 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:36.925217 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" event={"ID":"2154d6d4-6c28-4077-8ce8-2830c1347168","Type":"ContainerStarted","Data":"f519ed4b3bbba97e0c9eed13590dd43f8ca46fc5e772156043b8e77a0d101bdd"} Jul 13 14:47:36.925443 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:36.925385 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:47:36.925443 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:36.925430 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" event={"ID":"2154d6d4-6c28-4077-8ce8-2830c1347168","Type":"ContainerStarted","Data":"af8f083935c2b59e3e9e50de52d81fbd3ac4743fe5c4dfbab4518bdb4609d480"} Jul 13 14:47:36.926633 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:36.926605 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b47lx" event={"ID":"6c60d2d4-243b-4952-83b9-bb2c462d9e2c","Type":"ContainerStarted","Data":"719ee42190610842c28b26112c7b315df4d098e3e1bf40f4139cfdd763c3869d"} Jul 13 14:47:36.928716 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:36.928691 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9hk94" event={"ID":"30b24206-b58d-4409-b6e1-1dec743e754c","Type":"ContainerStarted","Data":"2dca365eae87488ca3d8c522e312dac9f7ce4a9652bcfa285c592bd9ff8bebe1"} Jul 13 14:47:36.950148 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:36.950096 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" podStartSLOduration=32.950075994 podStartE2EDuration="32.950075994s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 14:47:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:47:36.949259087 +0000 UTC m=+69.189536967" watchObservedRunningTime="2026-07-13 14:47:36.950075994 +0000 UTC m=+69.190353822" Jul 13 14:47:36.966632 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:36.966561 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9hk94" podStartSLOduration=67.009296915 podStartE2EDuration="1m7.966544607s" totalImagesPullingTime="957.247692ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 14:46:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:47:36.965720304 +0000 UTC m=+69.205998154" watchObservedRunningTime="2026-07-13 14:47:36.966544607 +0000 UTC m=+69.206822455" Jul 13 14:47:37.037109 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:37.036417 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a3ae3942-f36d-405f-8c8c-124b903d6273-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fqm8q\" (UID: \"a3ae3942-f36d-405f-8c8c-124b903d6273\") " pod="openshift-insights/insights-runtime-extractor-fqm8q" Jul 13 14:47:37.039581 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:37.039549 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a3ae3942-f36d-405f-8c8c-124b903d6273-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fqm8q\" (UID: \"a3ae3942-f36d-405f-8c8c-124b903d6273\") " pod="openshift-insights/insights-runtime-extractor-fqm8q" Jul 13 14:47:37.285276 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:37.285240 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dhzjh\"" Jul 13 14:47:37.293457 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:37.293403 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fqm8q" Jul 13 14:47:37.467528 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:37.467498 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fqm8q"] Jul 13 14:47:37.471353 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:47:37.471326 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3ae3942_f36d_405f_8c8c_124b903d6273.slice/crio-f6f06671fdc14838a066fb25d00f97d6a2fb15f7a672df0723c2f6af45e06d0b WatchSource:0}: Error finding container f6f06671fdc14838a066fb25d00f97d6a2fb15f7a672df0723c2f6af45e06d0b: Status 404 returned error can't find the container with id f6f06671fdc14838a066fb25d00f97d6a2fb15f7a672df0723c2f6af45e06d0b Jul 13 14:47:37.875221 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:37.875179 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:37.935109 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:37.935060 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fqm8q" event={"ID":"a3ae3942-f36d-405f-8c8c-124b903d6273","Type":"ContainerStarted","Data":"f6f06671fdc14838a066fb25d00f97d6a2fb15f7a672df0723c2f6af45e06d0b"} Jul 13 14:47:37.976257 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:37.976218 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:38.356277 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.356247 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-4frwd"] Jul 13 14:47:38.372023 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.371990 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4845667c-xpxqr"] Jul 13 14:47:38.372418 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.372392 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-4frwd" Jul 13 14:47:38.375129 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.374810 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-jvs4c\"" Jul 13 14:47:38.376554 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.375768 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 13 14:47:38.376554 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.375775 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 13 14:47:38.384893 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.384688 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-4frwd"] Jul 13 14:47:38.384893 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.384737 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4845667c-xpxqr"] Jul 13 14:47:38.384893 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.384853 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4845667c-xpxqr" Jul 13 14:47:38.386181 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.386145 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6bfb696d64-gvxnc"] Jul 13 14:47:38.389173 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.388953 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 13 14:47:38.389173 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.388958 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 13 14:47:38.389173 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.389050 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 13 14:47:38.389399 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.389235 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 13 14:47:38.389399 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.389333 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 13 14:47:38.389503 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.389430 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 13 14:47:38.390433 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.390413 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 13 14:47:38.468411 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.468376 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:38.547247 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.547216 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vd69\" (UniqueName: \"kubernetes.io/projected/bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2-kube-api-access-9vd69\") pod \"cluster-proxy-proxy-agent-7b4845667c-xpxqr\" (UID: \"bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4845667c-xpxqr" Jul 13 14:47:38.547431 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.547253 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2-ca\") pod \"cluster-proxy-proxy-agent-7b4845667c-xpxqr\" (UID: \"bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4845667c-xpxqr" Jul 13 14:47:38.547431 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.547273 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6c8d99ef-1d66-4af6-bc92-e65658a50b36-nginx-conf\") pod \"networking-console-plugin-6cf856758c-4frwd\" (UID: \"6c8d99ef-1d66-4af6-bc92-e65658a50b36\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-4frwd" Jul 13 14:47:38.547431 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.547388 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7b4845667c-xpxqr\" (UID: \"bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4845667c-xpxqr" Jul 13 14:47:38.547587 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.547428 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2-hub\") pod \"cluster-proxy-proxy-agent-7b4845667c-xpxqr\" (UID: \"bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4845667c-xpxqr" Jul 13 14:47:38.547587 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.547464 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7b4845667c-xpxqr\" (UID: \"bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4845667c-xpxqr" Jul 13 14:47:38.547587 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.547499 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7b4845667c-xpxqr\" (UID: \"bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4845667c-xpxqr" Jul 13 14:47:38.547587 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.547561 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6c8d99ef-1d66-4af6-bc92-e65658a50b36-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-4frwd\" (UID: \"6c8d99ef-1d66-4af6-bc92-e65658a50b36\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-4frwd" Jul 13 14:47:38.562977 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.562944 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:38.649055 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.648970 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7b4845667c-xpxqr\" (UID: \"bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4845667c-xpxqr" Jul 13 14:47:38.649055 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.649030 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6c8d99ef-1d66-4af6-bc92-e65658a50b36-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-4frwd\" (UID: \"6c8d99ef-1d66-4af6-bc92-e65658a50b36\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-4frwd" Jul 13 14:47:38.649293 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.649076 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vd69\" (UniqueName: \"kubernetes.io/projected/bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2-kube-api-access-9vd69\") pod \"cluster-proxy-proxy-agent-7b4845667c-xpxqr\" (UID: \"bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4845667c-xpxqr" Jul 13 14:47:38.649293 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.649102 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2-ca\") pod \"cluster-proxy-proxy-agent-7b4845667c-xpxqr\" (UID: \"bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4845667c-xpxqr" Jul 13 14:47:38.649293 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.649126 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6c8d99ef-1d66-4af6-bc92-e65658a50b36-nginx-conf\") pod \"networking-console-plugin-6cf856758c-4frwd\" (UID: \"6c8d99ef-1d66-4af6-bc92-e65658a50b36\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-4frwd" Jul 13 14:47:38.649293 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.649151 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7b4845667c-xpxqr\" (UID: \"bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4845667c-xpxqr" Jul 13 14:47:38.649293 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.649176 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2-hub\") pod \"cluster-proxy-proxy-agent-7b4845667c-xpxqr\" (UID: \"bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4845667c-xpxqr" Jul 13 14:47:38.649293 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.649210 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7b4845667c-xpxqr\" (UID: \"bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4845667c-xpxqr" Jul 13 14:47:38.649952 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.649904 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7b4845667c-xpxqr\" (UID: \"bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4845667c-xpxqr" Jul 13 14:47:38.650329 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.650281 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6c8d99ef-1d66-4af6-bc92-e65658a50b36-nginx-conf\") pod \"networking-console-plugin-6cf856758c-4frwd\" (UID: \"6c8d99ef-1d66-4af6-bc92-e65658a50b36\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-4frwd" Jul 13 14:47:38.652651 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.652600 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2-ca\") pod \"cluster-proxy-proxy-agent-7b4845667c-xpxqr\" (UID: \"bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4845667c-xpxqr" Jul 13 14:47:38.652753 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.652685 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6c8d99ef-1d66-4af6-bc92-e65658a50b36-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-4frwd\" (UID: \"6c8d99ef-1d66-4af6-bc92-e65658a50b36\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-4frwd" Jul 13 14:47:38.652845 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.652819 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2-hub\") pod \"cluster-proxy-proxy-agent-7b4845667c-xpxqr\" (UID: \"bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4845667c-xpxqr" Jul 13 14:47:38.652902 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.652839 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7b4845667c-xpxqr\" (UID: \"bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4845667c-xpxqr" Jul 13 14:47:38.653406 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.653380 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7b4845667c-xpxqr\" (UID: \"bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4845667c-xpxqr" Jul 13 14:47:38.658085 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.658064 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vd69\" (UniqueName: \"kubernetes.io/projected/bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2-kube-api-access-9vd69\") pod \"cluster-proxy-proxy-agent-7b4845667c-xpxqr\" (UID: \"bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4845667c-xpxqr" Jul 13 14:47:38.664279 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.664251 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:38.688337 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.688302 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-4frwd" Jul 13 14:47:38.703779 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.703748 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4845667c-xpxqr" Jul 13 14:47:38.760533 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:38.760498 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:39.509737 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:39.509605 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-4frwd"] Jul 13 14:47:39.511758 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:47:39.511730 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c8d99ef_1d66_4af6_bc92_e65658a50b36.slice/crio-09968dbf8cb7ea5c396b50ecc867e98ba222d51c515a221d24465cddf118c297 WatchSource:0}: Error finding container 09968dbf8cb7ea5c396b50ecc867e98ba222d51c515a221d24465cddf118c297: Status 404 returned error can't find the container with id 09968dbf8cb7ea5c396b50ecc867e98ba222d51c515a221d24465cddf118c297 Jul 13 14:47:39.526074 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:39.526050 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4845667c-xpxqr"] Jul 13 14:47:39.529477 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:47:39.529455 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbab6f4a7_9a50_4543_8613_0b0e5d7fb4b2.slice/crio-d4b9cf4abee96164f5c287036a803267d748f525ac0474bb54084a6512dab719 WatchSource:0}: Error finding container d4b9cf4abee96164f5c287036a803267d748f525ac0474bb54084a6512dab719: Status 404 returned error can't find the container with id d4b9cf4abee96164f5c287036a803267d748f525ac0474bb54084a6512dab719 Jul 13 14:47:39.944415 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:39.944209 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-km6w9" event={"ID":"764568c4-93b4-4bf4-a627-a26e55610c81","Type":"ContainerStarted","Data":"53506b8e27549af650807ad112cdc7b47076442e6f20001419a7067af7603ae3"} Jul 13 14:47:39.945951 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:39.945919 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4845667c-xpxqr" event={"ID":"bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2","Type":"ContainerStarted","Data":"d4b9cf4abee96164f5c287036a803267d748f525ac0474bb54084a6512dab719"} Jul 13 14:47:39.947412 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:39.947383 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-4frwd" event={"ID":"6c8d99ef-1d66-4af6-bc92-e65658a50b36","Type":"ContainerStarted","Data":"09968dbf8cb7ea5c396b50ecc867e98ba222d51c515a221d24465cddf118c297"} Jul 13 14:47:39.962328 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:39.962264 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-km6w9" podStartSLOduration=66.202914126 podStartE2EDuration="1m10.962244012s" totalImagesPullingTime="4.759329886s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 14:46:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:47:39.961844231 +0000 UTC m=+72.202122106" watchObservedRunningTime="2026-07-13 14:47:39.962244012 +0000 UTC m=+72.202521862" Jul 13 14:47:43.352469 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:43.352420 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:43.412981 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:43.412932 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:43.495943 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:43.495911 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:43.578305 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:43.578273 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:43.675016 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:43.674929 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:46.978986 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:46.978941 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b47lx" event={"ID":"6c60d2d4-243b-4952-83b9-bb2c462d9e2c","Type":"ContainerStarted","Data":"a3783667f625611ba0e923de8761120e6523d4d7bf4658d5a7d77d55c9a3819b"} Jul 13 14:47:46.979451 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:46.979003 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b47lx" event={"ID":"6c60d2d4-243b-4952-83b9-bb2c462d9e2c","Type":"ContainerStarted","Data":"9082bb0fe79f14451397f023d78c2901bfff0842743a9492c17409063a181781"} Jul 13 14:47:46.979451 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:46.979062 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-b47lx" Jul 13 14:47:46.980430 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:46.980411 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fqm8q" event={"ID":"a3ae3942-f36d-405f-8c8c-124b903d6273","Type":"ContainerStarted","Data":"90be0675823037316f85d8ed108bb38cba9466f707aea229ae75a1bd9d7f0a2e"} Jul 13 14:47:46.981846 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:46.981823 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-4frwd" event={"ID":"6c8d99ef-1d66-4af6-bc92-e65658a50b36","Type":"ContainerStarted","Data":"5dde050785fd488381fdff86966d32cc787238b2d646a0421a45077a54a96753"} Jul 13 14:47:46.983724 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:46.983698 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4845667c-xpxqr" event={"ID":"bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2","Type":"ContainerStarted","Data":"ce45df581d6efa469df1e4247ebc451a33908a585227618f48cb0b9db3b909ec"} Jul 13 14:47:46.983819 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:46.983733 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4845667c-xpxqr" event={"ID":"bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2","Type":"ContainerStarted","Data":"ce57b056261178a981bf752aef3d2659ef879b0ce38869973e5e79d1a517cdd1"} Jul 13 14:47:46.983819 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:46.983752 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4845667c-xpxqr" event={"ID":"bab6f4a7-9a50-4543-8613-0b0e5d7fb4b2","Type":"ContainerStarted","Data":"e4de1429494599f48a2a58fd8ad60fcfee1a811e07aace35f83950bece2e72cd"} Jul 13 14:47:47.002459 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:47.002416 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-b47lx" podStartSLOduration=33.220100682 podStartE2EDuration="43.002405387s" totalImagesPullingTime="9.782304705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 14:47:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:47:47.001137526 +0000 UTC m=+79.241415493" watchObservedRunningTime="2026-07-13 14:47:47.002405387 +0000 UTC m=+79.242683236" Jul 13 14:47:47.022693 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:47.022654 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4845667c-xpxqr" podStartSLOduration=2.351957317 podStartE2EDuration="9.022640414s" totalImagesPullingTime="6.670683097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 14:47:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:47:47.021680109 +0000 UTC m=+79.261957959" watchObservedRunningTime="2026-07-13 14:47:47.022640414 +0000 UTC m=+79.262918260" Jul 13 14:47:47.048871 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:47.048830 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-4frwd" podStartSLOduration=2.722324446 podStartE2EDuration="9.048817163s" totalImagesPullingTime="6.326492717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 14:47:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:47:47.04801937 +0000 UTC m=+79.288297218" watchObservedRunningTime="2026-07-13 14:47:47.048817163 +0000 UTC m=+79.289095011" Jul 13 14:47:47.809405 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:47.809360 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:47.891623 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:47.891592 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:47.990269 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:47.990226 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fqm8q" event={"ID":"a3ae3942-f36d-405f-8c8c-124b903d6273","Type":"ContainerStarted","Data":"31a04365b0531be8e99c007b997989db14b5e99534f6fd9b5ab8bca815f31d15"} Jul 13 14:47:48.664866 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:48.664805 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:48.893756 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:48.893709 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:48.951347 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:48.951248 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-q748j"] Jul 13 14:47:48.954685 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:48.954664 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-q748j" Jul 13 14:47:48.957494 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:48.957471 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-qt79p\"" Jul 13 14:47:48.957798 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:48.957778 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 13 14:47:48.958656 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:48.958616 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 13 14:47:48.958768 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:48.958665 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 13 14:47:48.958768 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:48.958617 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 13 14:47:48.958768 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:48.958708 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 13 14:47:48.958768 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:48.958625 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 13 14:47:49.035559 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.035522 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a3feb036-72ac-407b-9847-5af7a59377f8-node-exporter-wtmp\") pod \"node-exporter-q748j\" (UID: \"a3feb036-72ac-407b-9847-5af7a59377f8\") " pod="openshift-monitoring/node-exporter-q748j" Jul 13 14:47:49.036028 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.035591 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x8rx\" (UniqueName: \"kubernetes.io/projected/a3feb036-72ac-407b-9847-5af7a59377f8-kube-api-access-4x8rx\") pod \"node-exporter-q748j\" (UID: \"a3feb036-72ac-407b-9847-5af7a59377f8\") " pod="openshift-monitoring/node-exporter-q748j" Jul 13 14:47:49.036028 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.035630 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a3feb036-72ac-407b-9847-5af7a59377f8-node-exporter-accelerators-collector-config\") pod \"node-exporter-q748j\" (UID: \"a3feb036-72ac-407b-9847-5af7a59377f8\") " pod="openshift-monitoring/node-exporter-q748j" Jul 13 14:47:49.036028 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.035665 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a3feb036-72ac-407b-9847-5af7a59377f8-node-exporter-textfile\") pod \"node-exporter-q748j\" (UID: \"a3feb036-72ac-407b-9847-5af7a59377f8\") " pod="openshift-monitoring/node-exporter-q748j" Jul 13 14:47:49.036028 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.035698 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a3feb036-72ac-407b-9847-5af7a59377f8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q748j\" (UID: \"a3feb036-72ac-407b-9847-5af7a59377f8\") " pod="openshift-monitoring/node-exporter-q748j" Jul 13 14:47:49.036028 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.035741 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a3feb036-72ac-407b-9847-5af7a59377f8-root\") pod \"node-exporter-q748j\" (UID: \"a3feb036-72ac-407b-9847-5af7a59377f8\") " pod="openshift-monitoring/node-exporter-q748j" Jul 13 14:47:49.036028 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.035763 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3feb036-72ac-407b-9847-5af7a59377f8-metrics-client-ca\") pod \"node-exporter-q748j\" (UID: \"a3feb036-72ac-407b-9847-5af7a59377f8\") " pod="openshift-monitoring/node-exporter-q748j" Jul 13 14:47:49.036028 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.035790 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a3feb036-72ac-407b-9847-5af7a59377f8-sys\") pod \"node-exporter-q748j\" (UID: \"a3feb036-72ac-407b-9847-5af7a59377f8\") " pod="openshift-monitoring/node-exporter-q748j" Jul 13 14:47:49.036028 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.035816 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a3feb036-72ac-407b-9847-5af7a59377f8-node-exporter-tls\") pod \"node-exporter-q748j\" (UID: \"a3feb036-72ac-407b-9847-5af7a59377f8\") " pod="openshift-monitoring/node-exporter-q748j" Jul 13 14:47:49.036555 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.036534 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:49.124089 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.124054 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:49.136434 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.136399 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a3feb036-72ac-407b-9847-5af7a59377f8-node-exporter-accelerators-collector-config\") pod \"node-exporter-q748j\" (UID: \"a3feb036-72ac-407b-9847-5af7a59377f8\") " pod="openshift-monitoring/node-exporter-q748j" Jul 13 14:47:49.136661 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.136451 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a3feb036-72ac-407b-9847-5af7a59377f8-node-exporter-textfile\") pod \"node-exporter-q748j\" (UID: \"a3feb036-72ac-407b-9847-5af7a59377f8\") " pod="openshift-monitoring/node-exporter-q748j" Jul 13 14:47:49.136661 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.136491 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a3feb036-72ac-407b-9847-5af7a59377f8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q748j\" (UID: \"a3feb036-72ac-407b-9847-5af7a59377f8\") " pod="openshift-monitoring/node-exporter-q748j" Jul 13 14:47:49.136661 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.136535 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a3feb036-72ac-407b-9847-5af7a59377f8-root\") pod \"node-exporter-q748j\" (UID: \"a3feb036-72ac-407b-9847-5af7a59377f8\") " pod="openshift-monitoring/node-exporter-q748j" Jul 13 14:47:49.136661 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.136563 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3feb036-72ac-407b-9847-5af7a59377f8-metrics-client-ca\") pod \"node-exporter-q748j\" (UID: \"a3feb036-72ac-407b-9847-5af7a59377f8\") " pod="openshift-monitoring/node-exporter-q748j" Jul 13 14:47:49.136661 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.136620 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a3feb036-72ac-407b-9847-5af7a59377f8-sys\") pod \"node-exporter-q748j\" (UID: \"a3feb036-72ac-407b-9847-5af7a59377f8\") " pod="openshift-monitoring/node-exporter-q748j" Jul 13 14:47:49.136661 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.136636 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a3feb036-72ac-407b-9847-5af7a59377f8-root\") pod \"node-exporter-q748j\" (UID: \"a3feb036-72ac-407b-9847-5af7a59377f8\") " pod="openshift-monitoring/node-exporter-q748j" Jul 13 14:47:49.136661 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.136662 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a3feb036-72ac-407b-9847-5af7a59377f8-node-exporter-tls\") pod \"node-exporter-q748j\" (UID: \"a3feb036-72ac-407b-9847-5af7a59377f8\") " pod="openshift-monitoring/node-exporter-q748j" Jul 13 14:47:49.137065 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.136751 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a3feb036-72ac-407b-9847-5af7a59377f8-node-exporter-wtmp\") pod \"node-exporter-q748j\" (UID: \"a3feb036-72ac-407b-9847-5af7a59377f8\") " pod="openshift-monitoring/node-exporter-q748j" Jul 13 14:47:49.137065 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:49.136774 2568 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 13 14:47:49.137065 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.136783 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x8rx\" (UniqueName: \"kubernetes.io/projected/a3feb036-72ac-407b-9847-5af7a59377f8-kube-api-access-4x8rx\") pod \"node-exporter-q748j\" (UID: \"a3feb036-72ac-407b-9847-5af7a59377f8\") " pod="openshift-monitoring/node-exporter-q748j" Jul 13 14:47:49.137065 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:49.136841 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3feb036-72ac-407b-9847-5af7a59377f8-node-exporter-tls podName:a3feb036-72ac-407b-9847-5af7a59377f8 nodeName:}" failed. No retries permitted until 2026-07-13 14:47:49.636819476 +0000 UTC m=+81.877097321 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a3feb036-72ac-407b-9847-5af7a59377f8-node-exporter-tls") pod "node-exporter-q748j" (UID: "a3feb036-72ac-407b-9847-5af7a59377f8") : secret "node-exporter-tls" not found Jul 13 14:47:49.137065 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.136975 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a3feb036-72ac-407b-9847-5af7a59377f8-node-exporter-textfile\") pod \"node-exporter-q748j\" (UID: \"a3feb036-72ac-407b-9847-5af7a59377f8\") " pod="openshift-monitoring/node-exporter-q748j" Jul 13 14:47:49.137327 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.137097 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a3feb036-72ac-407b-9847-5af7a59377f8-node-exporter-wtmp\") pod \"node-exporter-q748j\" (UID: \"a3feb036-72ac-407b-9847-5af7a59377f8\") " pod="openshift-monitoring/node-exporter-q748j" Jul 13 14:47:49.137327 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.137105 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a3feb036-72ac-407b-9847-5af7a59377f8-node-exporter-accelerators-collector-config\") pod \"node-exporter-q748j\" (UID: \"a3feb036-72ac-407b-9847-5af7a59377f8\") " pod="openshift-monitoring/node-exporter-q748j" Jul 13 14:47:49.137327 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.137139 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a3feb036-72ac-407b-9847-5af7a59377f8-sys\") pod \"node-exporter-q748j\" (UID: \"a3feb036-72ac-407b-9847-5af7a59377f8\") " pod="openshift-monitoring/node-exporter-q748j" Jul 13 14:47:49.137656 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.137609 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3feb036-72ac-407b-9847-5af7a59377f8-metrics-client-ca\") pod \"node-exporter-q748j\" (UID: \"a3feb036-72ac-407b-9847-5af7a59377f8\") " pod="openshift-monitoring/node-exporter-q748j" Jul 13 14:47:49.139948 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.139911 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a3feb036-72ac-407b-9847-5af7a59377f8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q748j\" (UID: \"a3feb036-72ac-407b-9847-5af7a59377f8\") " pod="openshift-monitoring/node-exporter-q748j" Jul 13 14:47:49.148340 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.148312 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x8rx\" (UniqueName: \"kubernetes.io/projected/a3feb036-72ac-407b-9847-5af7a59377f8-kube-api-access-4x8rx\") pod \"node-exporter-q748j\" (UID: \"a3feb036-72ac-407b-9847-5af7a59377f8\") " pod="openshift-monitoring/node-exporter-q748j" Jul 13 14:47:49.234068 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.233988 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:49.331214 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.331167 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:49.447492 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.447462 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:49.561419 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.561387 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:49.641202 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.641169 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a3feb036-72ac-407b-9847-5af7a59377f8-node-exporter-tls\") pod \"node-exporter-q748j\" (UID: \"a3feb036-72ac-407b-9847-5af7a59377f8\") " pod="openshift-monitoring/node-exporter-q748j" Jul 13 14:47:49.643684 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.643654 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a3feb036-72ac-407b-9847-5af7a59377f8-node-exporter-tls\") pod \"node-exporter-q748j\" (UID: \"a3feb036-72ac-407b-9847-5af7a59377f8\") " pod="openshift-monitoring/node-exporter-q748j" Jul 13 14:47:49.667994 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.667964 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:49.867157 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.867129 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-q748j" Jul 13 14:47:49.875992 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:47:49.875963 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3feb036_72ac_407b_9847_5af7a59377f8.slice/crio-f075f28be859571401bd3a90e76b233821a76b387f361f2f93458a961d91170f WatchSource:0}: Error finding container f075f28be859571401bd3a90e76b233821a76b387f361f2f93458a961d91170f: Status 404 returned error can't find the container with id f075f28be859571401bd3a90e76b233821a76b387f361f2f93458a961d91170f Jul 13 14:47:49.951015 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.950978 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:49.998796 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.998759 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fqm8q" event={"ID":"a3ae3942-f36d-405f-8c8c-124b903d6273","Type":"ContainerStarted","Data":"81d310dab87c6882fd5932d165e516d45609d384b8e72a3cec5510e539c0c3b3"} Jul 13 14:47:49.999894 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:49.999869 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q748j" event={"ID":"a3feb036-72ac-407b-9847-5af7a59377f8","Type":"ContainerStarted","Data":"f075f28be859571401bd3a90e76b233821a76b387f361f2f93458a961d91170f"} Jul 13 14:47:50.025598 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:50.025523 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-fqm8q" podStartSLOduration=32.829884676 podStartE2EDuration="45.025510096s" totalImagesPullingTime="12.19562542s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 14:47:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:47:50.02222442 +0000 UTC m=+82.262502336" watchObservedRunningTime="2026-07-13 14:47:50.025510096 +0000 UTC m=+82.265787945" Jul 13 14:47:50.064738 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:50.064699 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:50.416292 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:50.416202 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:51.005322 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:51.005287 2568 generic.go:356] "Generic (PLEG): container finished" podID="a3feb036-72ac-407b-9847-5af7a59377f8" containerID="084c86404fb28d9f5abea557c7e52996b2ff6e86a1d3effca0168109c389af2f" exitCode=0 Jul 13 14:47:51.005509 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:51.005368 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q748j" event={"ID":"a3feb036-72ac-407b-9847-5af7a59377f8","Type":"ContainerDied","Data":"084c86404fb28d9f5abea557c7e52996b2ff6e86a1d3effca0168109c389af2f"} Jul 13 14:47:51.895526 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:51.895491 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:52.010742 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:52.010706 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q748j" event={"ID":"a3feb036-72ac-407b-9847-5af7a59377f8","Type":"ContainerStarted","Data":"b7701e21542f05754af7389f9bad2209f3a1a308147f8b2e526447d7e3f64a25"} Jul 13 14:47:52.010892 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:52.010766 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q748j" event={"ID":"a3feb036-72ac-407b-9847-5af7a59377f8","Type":"ContainerStarted","Data":"24617ef16cfcc7569d5f95949e508698fb14bf848d446e865b017a766b52f7df"} Jul 13 14:47:52.039781 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:52.039726 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-q748j" podStartSLOduration=3.046771246 podStartE2EDuration="4.03971229s" totalImagesPullingTime="929.45486ms" totalInitContainerRuntime="63.486184ms" isStatefulPod=true podCreationTimestamp="2026-07-13 14:47:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:47:52.037962992 +0000 UTC m=+84.278240840" watchObservedRunningTime="2026-07-13 14:47:52.03971229 +0000 UTC m=+84.279990140" Jul 13 14:47:52.270911 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:52.270828 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:52.373831 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:52.373804 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:52.866067 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:52.866032 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:52.969439 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:52.969406 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:53.233740 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.233664 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6c65bfbc8f-kj8xv"] Jul 13 14:47:53.236715 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.236700 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6c65bfbc8f-kj8xv" Jul 13 14:47:53.240206 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.240182 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-q4nfw\"" Jul 13 14:47:53.240350 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.240219 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 13 14:47:53.240350 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.240257 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 13 14:47:53.240350 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.240277 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-6hjq1of8plbco\"" Jul 13 14:47:53.240350 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.240340 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 13 14:47:53.240526 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.240398 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 13 14:47:53.252855 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.252838 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6c65bfbc8f-kj8xv"] Jul 13 14:47:53.266027 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.266006 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/09033cc5-16e0-488e-bf10-a0e7a5838d94-secret-metrics-server-client-certs\") pod \"metrics-server-6c65bfbc8f-kj8xv\" (UID: \"09033cc5-16e0-488e-bf10-a0e7a5838d94\") " pod="openshift-monitoring/metrics-server-6c65bfbc8f-kj8xv" Jul 13 14:47:53.266130 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.266040 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09033cc5-16e0-488e-bf10-a0e7a5838d94-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6c65bfbc8f-kj8xv\" (UID: \"09033cc5-16e0-488e-bf10-a0e7a5838d94\") " pod="openshift-monitoring/metrics-server-6c65bfbc8f-kj8xv" Jul 13 14:47:53.266130 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.266066 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/09033cc5-16e0-488e-bf10-a0e7a5838d94-metrics-server-audit-profiles\") pod \"metrics-server-6c65bfbc8f-kj8xv\" (UID: \"09033cc5-16e0-488e-bf10-a0e7a5838d94\") " pod="openshift-monitoring/metrics-server-6c65bfbc8f-kj8xv" Jul 13 14:47:53.266130 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.266107 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/09033cc5-16e0-488e-bf10-a0e7a5838d94-secret-metrics-server-tls\") pod \"metrics-server-6c65bfbc8f-kj8xv\" (UID: \"09033cc5-16e0-488e-bf10-a0e7a5838d94\") " pod="openshift-monitoring/metrics-server-6c65bfbc8f-kj8xv" Jul 13 14:47:53.266130 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.266126 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/09033cc5-16e0-488e-bf10-a0e7a5838d94-client-ca-bundle\") pod \"metrics-server-6c65bfbc8f-kj8xv\" (UID: \"09033cc5-16e0-488e-bf10-a0e7a5838d94\") " pod="openshift-monitoring/metrics-server-6c65bfbc8f-kj8xv" Jul 13 14:47:53.266277 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.266144 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/09033cc5-16e0-488e-bf10-a0e7a5838d94-audit-log\") pod \"metrics-server-6c65bfbc8f-kj8xv\" (UID: \"09033cc5-16e0-488e-bf10-a0e7a5838d94\") " pod="openshift-monitoring/metrics-server-6c65bfbc8f-kj8xv" Jul 13 14:47:53.266277 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.266199 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqg8c\" (UniqueName: \"kubernetes.io/projected/09033cc5-16e0-488e-bf10-a0e7a5838d94-kube-api-access-qqg8c\") pod \"metrics-server-6c65bfbc8f-kj8xv\" (UID: \"09033cc5-16e0-488e-bf10-a0e7a5838d94\") " pod="openshift-monitoring/metrics-server-6c65bfbc8f-kj8xv" Jul 13 14:47:53.367356 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.367325 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/09033cc5-16e0-488e-bf10-a0e7a5838d94-secret-metrics-server-client-certs\") pod \"metrics-server-6c65bfbc8f-kj8xv\" (UID: \"09033cc5-16e0-488e-bf10-a0e7a5838d94\") " pod="openshift-monitoring/metrics-server-6c65bfbc8f-kj8xv" Jul 13 14:47:53.367513 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.367379 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09033cc5-16e0-488e-bf10-a0e7a5838d94-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6c65bfbc8f-kj8xv\" (UID: \"09033cc5-16e0-488e-bf10-a0e7a5838d94\") " pod="openshift-monitoring/metrics-server-6c65bfbc8f-kj8xv" Jul 13 14:47:53.367513 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.367406 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/09033cc5-16e0-488e-bf10-a0e7a5838d94-metrics-server-audit-profiles\") pod \"metrics-server-6c65bfbc8f-kj8xv\" (UID: \"09033cc5-16e0-488e-bf10-a0e7a5838d94\") " pod="openshift-monitoring/metrics-server-6c65bfbc8f-kj8xv" Jul 13 14:47:53.367513 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.367428 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/09033cc5-16e0-488e-bf10-a0e7a5838d94-secret-metrics-server-tls\") pod \"metrics-server-6c65bfbc8f-kj8xv\" (UID: \"09033cc5-16e0-488e-bf10-a0e7a5838d94\") " pod="openshift-monitoring/metrics-server-6c65bfbc8f-kj8xv" Jul 13 14:47:53.367513 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.367444 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/09033cc5-16e0-488e-bf10-a0e7a5838d94-client-ca-bundle\") pod \"metrics-server-6c65bfbc8f-kj8xv\" (UID: \"09033cc5-16e0-488e-bf10-a0e7a5838d94\") " pod="openshift-monitoring/metrics-server-6c65bfbc8f-kj8xv" Jul 13 14:47:53.367513 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.367471 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/09033cc5-16e0-488e-bf10-a0e7a5838d94-audit-log\") pod \"metrics-server-6c65bfbc8f-kj8xv\" (UID: \"09033cc5-16e0-488e-bf10-a0e7a5838d94\") " pod="openshift-monitoring/metrics-server-6c65bfbc8f-kj8xv" Jul 13 14:47:53.367513 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.367497 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqg8c\" (UniqueName: \"kubernetes.io/projected/09033cc5-16e0-488e-bf10-a0e7a5838d94-kube-api-access-qqg8c\") pod \"metrics-server-6c65bfbc8f-kj8xv\" (UID: \"09033cc5-16e0-488e-bf10-a0e7a5838d94\") " pod="openshift-monitoring/metrics-server-6c65bfbc8f-kj8xv" Jul 13 14:47:53.368004 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.367980 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/09033cc5-16e0-488e-bf10-a0e7a5838d94-audit-log\") pod \"metrics-server-6c65bfbc8f-kj8xv\" (UID: \"09033cc5-16e0-488e-bf10-a0e7a5838d94\") " pod="openshift-monitoring/metrics-server-6c65bfbc8f-kj8xv" Jul 13 14:47:53.368279 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.368258 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09033cc5-16e0-488e-bf10-a0e7a5838d94-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6c65bfbc8f-kj8xv\" (UID: \"09033cc5-16e0-488e-bf10-a0e7a5838d94\") " pod="openshift-monitoring/metrics-server-6c65bfbc8f-kj8xv" Jul 13 14:47:53.368469 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.368450 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/09033cc5-16e0-488e-bf10-a0e7a5838d94-metrics-server-audit-profiles\") pod \"metrics-server-6c65bfbc8f-kj8xv\" (UID: \"09033cc5-16e0-488e-bf10-a0e7a5838d94\") " pod="openshift-monitoring/metrics-server-6c65bfbc8f-kj8xv" Jul 13 14:47:53.369874 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.369857 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/09033cc5-16e0-488e-bf10-a0e7a5838d94-client-ca-bundle\") pod \"metrics-server-6c65bfbc8f-kj8xv\" (UID: \"09033cc5-16e0-488e-bf10-a0e7a5838d94\") " pod="openshift-monitoring/metrics-server-6c65bfbc8f-kj8xv" Jul 13 14:47:53.370322 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.370303 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/09033cc5-16e0-488e-bf10-a0e7a5838d94-secret-metrics-server-tls\") pod \"metrics-server-6c65bfbc8f-kj8xv\" (UID: \"09033cc5-16e0-488e-bf10-a0e7a5838d94\") " pod="openshift-monitoring/metrics-server-6c65bfbc8f-kj8xv" Jul 13 14:47:53.370360 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.370351 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/09033cc5-16e0-488e-bf10-a0e7a5838d94-secret-metrics-server-client-certs\") pod \"metrics-server-6c65bfbc8f-kj8xv\" (UID: \"09033cc5-16e0-488e-bf10-a0e7a5838d94\") " pod="openshift-monitoring/metrics-server-6c65bfbc8f-kj8xv" Jul 13 14:47:53.376414 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.376390 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqg8c\" (UniqueName: \"kubernetes.io/projected/09033cc5-16e0-488e-bf10-a0e7a5838d94-kube-api-access-qqg8c\") pod \"metrics-server-6c65bfbc8f-kj8xv\" (UID: \"09033cc5-16e0-488e-bf10-a0e7a5838d94\") " pod="openshift-monitoring/metrics-server-6c65bfbc8f-kj8xv" Jul 13 14:47:53.545886 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.545847 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6c65bfbc8f-kj8xv" Jul 13 14:47:53.696065 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.696042 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6c65bfbc8f-kj8xv"] Jul 13 14:47:53.698599 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:47:53.698556 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09033cc5_16e0_488e_bf10_a0e7a5838d94.slice/crio-cf8b2f0db3ead5d988cc8ed3906317bc8268952f57c98403ee7c82ac0b8aa479 WatchSource:0}: Error finding container cf8b2f0db3ead5d988cc8ed3906317bc8268952f57c98403ee7c82ac0b8aa479: Status 404 returned error can't find the container with id cf8b2f0db3ead5d988cc8ed3906317bc8268952f57c98403ee7c82ac0b8aa479 Jul 13 14:47:53.698861 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.698840 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:53.712957 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.712936 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-ghh7k"] Jul 13 14:47:53.715606 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.715560 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-ghh7k" Jul 13 14:47:53.718095 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.718074 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 13 14:47:53.718215 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.718199 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-2zrrj\"" Jul 13 14:47:53.725889 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.725865 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-ghh7k"] Jul 13 14:47:53.770278 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.770235 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88947007-7a57-4987-93e7-91531f043125-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-ghh7k\" (UID: \"88947007-7a57-4987-93e7-91531f043125\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-ghh7k" Jul 13 14:47:53.770278 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.770284 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76n7s\" (UniqueName: \"kubernetes.io/projected/88947007-7a57-4987-93e7-91531f043125-kube-api-access-76n7s\") pod \"monitoring-plugin-5f68df7f58-ghh7k\" (UID: \"88947007-7a57-4987-93e7-91531f043125\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-ghh7k" Jul 13 14:47:53.775990 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.775956 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:53.871633 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.871540 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88947007-7a57-4987-93e7-91531f043125-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-ghh7k\" (UID: \"88947007-7a57-4987-93e7-91531f043125\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-ghh7k" Jul 13 14:47:53.871633 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.871596 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76n7s\" (UniqueName: \"kubernetes.io/projected/88947007-7a57-4987-93e7-91531f043125-kube-api-access-76n7s\") pod \"monitoring-plugin-5f68df7f58-ghh7k\" (UID: \"88947007-7a57-4987-93e7-91531f043125\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-ghh7k" Jul 13 14:47:53.871838 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:53.871708 2568 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 13 14:47:53.871838 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:47:53.871832 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88947007-7a57-4987-93e7-91531f043125-monitoring-plugin-cert podName:88947007-7a57-4987-93e7-91531f043125 nodeName:}" failed. No retries permitted until 2026-07-13 14:47:54.371806526 +0000 UTC m=+86.612084369 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/88947007-7a57-4987-93e7-91531f043125-monitoring-plugin-cert") pod "monitoring-plugin-5f68df7f58-ghh7k" (UID: "88947007-7a57-4987-93e7-91531f043125") : secret "monitoring-plugin-cert" not found Jul 13 14:47:53.883040 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:53.883006 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76n7s\" (UniqueName: \"kubernetes.io/projected/88947007-7a57-4987-93e7-91531f043125-kube-api-access-76n7s\") pod \"monitoring-plugin-5f68df7f58-ghh7k\" (UID: \"88947007-7a57-4987-93e7-91531f043125\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-ghh7k" Jul 13 14:47:54.017928 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.017894 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6c65bfbc8f-kj8xv" event={"ID":"09033cc5-16e0-488e-bf10-a0e7a5838d94","Type":"ContainerStarted","Data":"cf8b2f0db3ead5d988cc8ed3906317bc8268952f57c98403ee7c82ac0b8aa479"} Jul 13 14:47:54.165523 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.165439 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-674b978496-25nld"] Jul 13 14:47:54.167861 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.167839 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-674b978496-25nld" Jul 13 14:47:54.170761 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.170739 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 13 14:47:54.171018 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.170997 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 13 14:47:54.171110 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.171068 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 13 14:47:54.171311 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.171283 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 13 14:47:54.171311 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.171307 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 13 14:47:54.171996 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.171969 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-89lsc\"" Jul 13 14:47:54.175157 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.175128 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8de88c3c-89f8-46bc-9486-d12804f06f6d-secret-telemeter-client\") pod \"telemeter-client-674b978496-25nld\" (UID: \"8de88c3c-89f8-46bc-9486-d12804f06f6d\") " pod="openshift-monitoring/telemeter-client-674b978496-25nld" Jul 13 14:47:54.175252 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.175195 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jk97w\" (UniqueName: \"kubernetes.io/projected/8de88c3c-89f8-46bc-9486-d12804f06f6d-kube-api-access-jk97w\") pod \"telemeter-client-674b978496-25nld\" (UID: \"8de88c3c-89f8-46bc-9486-d12804f06f6d\") " pod="openshift-monitoring/telemeter-client-674b978496-25nld" Jul 13 14:47:54.175309 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.175277 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8de88c3c-89f8-46bc-9486-d12804f06f6d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-674b978496-25nld\" (UID: \"8de88c3c-89f8-46bc-9486-d12804f06f6d\") " pod="openshift-monitoring/telemeter-client-674b978496-25nld" Jul 13 14:47:54.175360 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.175326 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8de88c3c-89f8-46bc-9486-d12804f06f6d-metrics-client-ca\") pod \"telemeter-client-674b978496-25nld\" (UID: \"8de88c3c-89f8-46bc-9486-d12804f06f6d\") " pod="openshift-monitoring/telemeter-client-674b978496-25nld" Jul 13 14:47:54.175630 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.175403 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8de88c3c-89f8-46bc-9486-d12804f06f6d-telemeter-client-tls\") pod \"telemeter-client-674b978496-25nld\" (UID: \"8de88c3c-89f8-46bc-9486-d12804f06f6d\") " pod="openshift-monitoring/telemeter-client-674b978496-25nld" Jul 13 14:47:54.175630 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.175556 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8de88c3c-89f8-46bc-9486-d12804f06f6d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-674b978496-25nld\" (UID: \"8de88c3c-89f8-46bc-9486-d12804f06f6d\") " pod="openshift-monitoring/telemeter-client-674b978496-25nld" Jul 13 14:47:54.175630 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.175612 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8de88c3c-89f8-46bc-9486-d12804f06f6d-serving-certs-ca-bundle\") pod \"telemeter-client-674b978496-25nld\" (UID: \"8de88c3c-89f8-46bc-9486-d12804f06f6d\") " pod="openshift-monitoring/telemeter-client-674b978496-25nld" Jul 13 14:47:54.175800 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.175653 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8de88c3c-89f8-46bc-9486-d12804f06f6d-federate-client-tls\") pod \"telemeter-client-674b978496-25nld\" (UID: \"8de88c3c-89f8-46bc-9486-d12804f06f6d\") " pod="openshift-monitoring/telemeter-client-674b978496-25nld" Jul 13 14:47:54.179474 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.179417 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 13 14:47:54.182831 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.182807 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-674b978496-25nld"] Jul 13 14:47:54.228943 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.228911 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:54.276646 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.276615 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jk97w\" (UniqueName: \"kubernetes.io/projected/8de88c3c-89f8-46bc-9486-d12804f06f6d-kube-api-access-jk97w\") pod \"telemeter-client-674b978496-25nld\" (UID: \"8de88c3c-89f8-46bc-9486-d12804f06f6d\") " pod="openshift-monitoring/telemeter-client-674b978496-25nld" Jul 13 14:47:54.276646 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.276651 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8de88c3c-89f8-46bc-9486-d12804f06f6d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-674b978496-25nld\" (UID: \"8de88c3c-89f8-46bc-9486-d12804f06f6d\") " pod="openshift-monitoring/telemeter-client-674b978496-25nld" Jul 13 14:47:54.276892 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.276830 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8de88c3c-89f8-46bc-9486-d12804f06f6d-metrics-client-ca\") pod \"telemeter-client-674b978496-25nld\" (UID: \"8de88c3c-89f8-46bc-9486-d12804f06f6d\") " pod="openshift-monitoring/telemeter-client-674b978496-25nld" Jul 13 14:47:54.276964 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.276901 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8de88c3c-89f8-46bc-9486-d12804f06f6d-telemeter-client-tls\") pod \"telemeter-client-674b978496-25nld\" (UID: \"8de88c3c-89f8-46bc-9486-d12804f06f6d\") " pod="openshift-monitoring/telemeter-client-674b978496-25nld" Jul 13 14:47:54.276964 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.276940 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8de88c3c-89f8-46bc-9486-d12804f06f6d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-674b978496-25nld\" (UID: \"8de88c3c-89f8-46bc-9486-d12804f06f6d\") " pod="openshift-monitoring/telemeter-client-674b978496-25nld" Jul 13 14:47:54.277071 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.276972 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8de88c3c-89f8-46bc-9486-d12804f06f6d-serving-certs-ca-bundle\") pod \"telemeter-client-674b978496-25nld\" (UID: \"8de88c3c-89f8-46bc-9486-d12804f06f6d\") " pod="openshift-monitoring/telemeter-client-674b978496-25nld" Jul 13 14:47:54.277071 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.277016 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8de88c3c-89f8-46bc-9486-d12804f06f6d-federate-client-tls\") pod \"telemeter-client-674b978496-25nld\" (UID: \"8de88c3c-89f8-46bc-9486-d12804f06f6d\") " pod="openshift-monitoring/telemeter-client-674b978496-25nld" Jul 13 14:47:54.277165 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.277093 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8de88c3c-89f8-46bc-9486-d12804f06f6d-secret-telemeter-client\") pod \"telemeter-client-674b978496-25nld\" (UID: \"8de88c3c-89f8-46bc-9486-d12804f06f6d\") " pod="openshift-monitoring/telemeter-client-674b978496-25nld" Jul 13 14:47:54.277761 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.277709 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8de88c3c-89f8-46bc-9486-d12804f06f6d-metrics-client-ca\") pod \"telemeter-client-674b978496-25nld\" (UID: \"8de88c3c-89f8-46bc-9486-d12804f06f6d\") " pod="openshift-monitoring/telemeter-client-674b978496-25nld" Jul 13 14:47:54.277761 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.277750 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8de88c3c-89f8-46bc-9486-d12804f06f6d-serving-certs-ca-bundle\") pod \"telemeter-client-674b978496-25nld\" (UID: \"8de88c3c-89f8-46bc-9486-d12804f06f6d\") " pod="openshift-monitoring/telemeter-client-674b978496-25nld" Jul 13 14:47:54.277976 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.277950 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8de88c3c-89f8-46bc-9486-d12804f06f6d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-674b978496-25nld\" (UID: \"8de88c3c-89f8-46bc-9486-d12804f06f6d\") " pod="openshift-monitoring/telemeter-client-674b978496-25nld" Jul 13 14:47:54.279873 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.279825 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8de88c3c-89f8-46bc-9486-d12804f06f6d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-674b978496-25nld\" (UID: \"8de88c3c-89f8-46bc-9486-d12804f06f6d\") " pod="openshift-monitoring/telemeter-client-674b978496-25nld" Jul 13 14:47:54.279984 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.279895 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8de88c3c-89f8-46bc-9486-d12804f06f6d-telemeter-client-tls\") pod \"telemeter-client-674b978496-25nld\" (UID: \"8de88c3c-89f8-46bc-9486-d12804f06f6d\") " pod="openshift-monitoring/telemeter-client-674b978496-25nld" Jul 13 14:47:54.280054 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.279999 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8de88c3c-89f8-46bc-9486-d12804f06f6d-secret-telemeter-client\") pod \"telemeter-client-674b978496-25nld\" (UID: \"8de88c3c-89f8-46bc-9486-d12804f06f6d\") " pod="openshift-monitoring/telemeter-client-674b978496-25nld" Jul 13 14:47:54.280508 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.280485 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8de88c3c-89f8-46bc-9486-d12804f06f6d-federate-client-tls\") pod \"telemeter-client-674b978496-25nld\" (UID: \"8de88c3c-89f8-46bc-9486-d12804f06f6d\") " pod="openshift-monitoring/telemeter-client-674b978496-25nld" Jul 13 14:47:54.287100 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.287079 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jk97w\" (UniqueName: \"kubernetes.io/projected/8de88c3c-89f8-46bc-9486-d12804f06f6d-kube-api-access-jk97w\") pod \"telemeter-client-674b978496-25nld\" (UID: \"8de88c3c-89f8-46bc-9486-d12804f06f6d\") " pod="openshift-monitoring/telemeter-client-674b978496-25nld" Jul 13 14:47:54.378493 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.378446 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88947007-7a57-4987-93e7-91531f043125-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-ghh7k\" (UID: \"88947007-7a57-4987-93e7-91531f043125\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-ghh7k" Jul 13 14:47:54.381143 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.381114 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88947007-7a57-4987-93e7-91531f043125-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-ghh7k\" (UID: \"88947007-7a57-4987-93e7-91531f043125\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-ghh7k" Jul 13 14:47:54.483223 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.483137 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-674b978496-25nld" Jul 13 14:47:54.523859 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.523823 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:54.620777 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.620737 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:54.625725 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.625698 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-ghh7k" Jul 13 14:47:54.627665 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:54.627639 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-674b978496-25nld"] Jul 13 14:47:54.904004 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:47:54.903973 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8de88c3c_89f8_46bc_9486_d12804f06f6d.slice/crio-545a59ea0332b91e1badfdce6138ec5a664bc48cdcf6d608d52f3a729a2bcabd WatchSource:0}: Error finding container 545a59ea0332b91e1badfdce6138ec5a664bc48cdcf6d608d52f3a729a2bcabd: Status 404 returned error can't find the container with id 545a59ea0332b91e1badfdce6138ec5a664bc48cdcf6d608d52f3a729a2bcabd Jul 13 14:47:55.024031 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:55.023734 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6c65bfbc8f-kj8xv" event={"ID":"09033cc5-16e0-488e-bf10-a0e7a5838d94","Type":"ContainerStarted","Data":"32f50efb86440b8d88e698ca3566d7db86284f88563a0c0d3f221ee297d9ff62"} Jul 13 14:47:55.025434 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:55.025403 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-674b978496-25nld" event={"ID":"8de88c3c-89f8-46bc-9486-d12804f06f6d","Type":"ContainerStarted","Data":"545a59ea0332b91e1badfdce6138ec5a664bc48cdcf6d608d52f3a729a2bcabd"} Jul 13 14:47:55.037875 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:55.037849 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-ghh7k"] Jul 13 14:47:55.040589 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:47:55.040551 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88947007_7a57_4987_93e7_91531f043125.slice/crio-e9a862016bb3ff303a2c111335dc96350411696ff0ea3f6c6144b0d9c1ed5d3a WatchSource:0}: Error finding container e9a862016bb3ff303a2c111335dc96350411696ff0ea3f6c6144b0d9c1ed5d3a: Status 404 returned error can't find the container with id e9a862016bb3ff303a2c111335dc96350411696ff0ea3f6c6144b0d9c1ed5d3a Jul 13 14:47:55.042932 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:55.042882 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6c65bfbc8f-kj8xv" podStartSLOduration=0.812232229 podStartE2EDuration="2.042862549s" totalImagesPullingTime="1.23063032s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 14:47:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:47:55.041893596 +0000 UTC m=+87.282171466" watchObservedRunningTime="2026-07-13 14:47:55.042862549 +0000 UTC m=+87.283140422" Jul 13 14:47:55.043600 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:55.043499 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:55.230491 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:55.230405 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:55.464122 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:55.464087 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:55.561267 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:55.561229 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:56.037262 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:56.037221 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-ghh7k" event={"ID":"88947007-7a57-4987-93e7-91531f043125","Type":"ContainerStarted","Data":"e9a862016bb3ff303a2c111335dc96350411696ff0ea3f6c6144b0d9c1ed5d3a"} Jul 13 14:47:56.992158 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:56.992131 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-b47lx" Jul 13 14:47:57.042792 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:57.042686 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-674b978496-25nld" event={"ID":"8de88c3c-89f8-46bc-9486-d12804f06f6d","Type":"ContainerStarted","Data":"30b527f2f16412913fd81157f34d978ce54c59f87c14db7337a8f46f58511716"} Jul 13 14:47:57.044233 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:57.044207 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-ghh7k" event={"ID":"88947007-7a57-4987-93e7-91531f043125","Type":"ContainerStarted","Data":"76b05b100f26d751781a13857e457a668d05001f75a049ed2ea866e2c61becb4"} Jul 13 14:47:57.044515 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:57.044495 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-ghh7k" Jul 13 14:47:57.048938 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:57.048917 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-ghh7k" Jul 13 14:47:57.063685 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:57.063635 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-ghh7k" podStartSLOduration=2.316684542 podStartE2EDuration="4.06361825s" totalImagesPullingTime="1.746933708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 14:47:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:47:57.062068436 +0000 UTC m=+89.302346302" watchObservedRunningTime="2026-07-13 14:47:57.06361825 +0000 UTC m=+89.303896098" Jul 13 14:47:58.492018 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:58.488659 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:58.608252 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:58.608212 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:47:58.944057 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:58.944032 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:47:59.052773 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:59.052735 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-674b978496-25nld" event={"ID":"8de88c3c-89f8-46bc-9486-d12804f06f6d","Type":"ContainerStarted","Data":"5096708c570a2ddee2190d2941b6dcb04187af77f39ed61b9ee8416c049d9652"} Jul 13 14:47:59.052927 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:59.052799 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-674b978496-25nld" event={"ID":"8de88c3c-89f8-46bc-9486-d12804f06f6d","Type":"ContainerStarted","Data":"8843bf0fc36cb88fdc9e71bb8f982c9b0eefdd581b936219a48850c51e4b2897"} Jul 13 14:47:59.077977 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:47:59.077920 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-674b978496-25nld" podStartSLOduration=1.9848955780000002 podStartE2EDuration="5.077902458s" totalImagesPullingTime="3.09300688s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 14:47:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:47:59.076235083 +0000 UTC m=+91.316512931" watchObservedRunningTime="2026-07-13 14:47:59.077902458 +0000 UTC m=+91.318180307" Jul 13 14:48:00.545737 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:00.545700 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:48:00.708342 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:00.708299 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:48:03.962537 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:03.962496 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" podUID="2154d6d4-6c28-4077-8ce8-2830c1347168" containerName="registry" containerID="cri-o://f519ed4b3bbba97e0c9eed13590dd43f8ca46fc5e772156043b8e77a0d101bdd" gracePeriod=30 Jul 13 14:48:04.026209 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:04.026170 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:48:04.125779 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:04.125748 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:48:04.199241 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:04.199215 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:48:04.248354 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:04.248268 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:48:04.266354 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:04.266324 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2154d6d4-6c28-4077-8ce8-2830c1347168-registry-certificates\") pod \"2154d6d4-6c28-4077-8ce8-2830c1347168\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " Jul 13 14:48:04.266484 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:04.266375 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-registry-tls\") pod \"2154d6d4-6c28-4077-8ce8-2830c1347168\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " Jul 13 14:48:04.266484 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:04.266410 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2154d6d4-6c28-4077-8ce8-2830c1347168-image-registry-private-configuration\") pod \"2154d6d4-6c28-4077-8ce8-2830c1347168\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " Jul 13 14:48:04.266484 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:04.266429 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-bound-sa-token\") pod \"2154d6d4-6c28-4077-8ce8-2830c1347168\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " Jul 13 14:48:04.266484 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:04.266453 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2154d6d4-6c28-4077-8ce8-2830c1347168-installation-pull-secrets\") pod \"2154d6d4-6c28-4077-8ce8-2830c1347168\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " Jul 13 14:48:04.266821 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:04.266489 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2154d6d4-6c28-4077-8ce8-2830c1347168-ca-trust-extracted\") pod \"2154d6d4-6c28-4077-8ce8-2830c1347168\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " Jul 13 14:48:04.266821 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:04.266528 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2154d6d4-6c28-4077-8ce8-2830c1347168-trusted-ca\") pod \"2154d6d4-6c28-4077-8ce8-2830c1347168\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " Jul 13 14:48:04.266821 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:04.266556 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfkkp\" (UniqueName: \"kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-kube-api-access-hfkkp\") pod \"2154d6d4-6c28-4077-8ce8-2830c1347168\" (UID: \"2154d6d4-6c28-4077-8ce8-2830c1347168\") " Jul 13 14:48:04.266821 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:04.266797 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2154d6d4-6c28-4077-8ce8-2830c1347168-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "2154d6d4-6c28-4077-8ce8-2830c1347168" (UID: "2154d6d4-6c28-4077-8ce8-2830c1347168"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 14:48:04.267254 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:04.267215 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2154d6d4-6c28-4077-8ce8-2830c1347168-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "2154d6d4-6c28-4077-8ce8-2830c1347168" (UID: "2154d6d4-6c28-4077-8ce8-2830c1347168"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 14:48:04.269451 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:04.269422 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "2154d6d4-6c28-4077-8ce8-2830c1347168" (UID: "2154d6d4-6c28-4077-8ce8-2830c1347168"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 14:48:04.269555 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:04.269486 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2154d6d4-6c28-4077-8ce8-2830c1347168-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "2154d6d4-6c28-4077-8ce8-2830c1347168" (UID: "2154d6d4-6c28-4077-8ce8-2830c1347168"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 14:48:04.269555 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:04.269529 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-kube-api-access-hfkkp" (OuterVolumeSpecName: "kube-api-access-hfkkp") pod "2154d6d4-6c28-4077-8ce8-2830c1347168" (UID: "2154d6d4-6c28-4077-8ce8-2830c1347168"). InnerVolumeSpecName "kube-api-access-hfkkp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 14:48:04.269555 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:04.269546 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2154d6d4-6c28-4077-8ce8-2830c1347168-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "2154d6d4-6c28-4077-8ce8-2830c1347168" (UID: "2154d6d4-6c28-4077-8ce8-2830c1347168"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 14:48:04.269746 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:04.269671 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "2154d6d4-6c28-4077-8ce8-2830c1347168" (UID: "2154d6d4-6c28-4077-8ce8-2830c1347168"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 14:48:04.278716 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:04.278688 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2154d6d4-6c28-4077-8ce8-2830c1347168-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "2154d6d4-6c28-4077-8ce8-2830c1347168" (UID: "2154d6d4-6c28-4077-8ce8-2830c1347168"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 14:48:04.348818 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:04.348780 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:48:04.367911 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:04.367877 2568 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2154d6d4-6c28-4077-8ce8-2830c1347168-ca-trust-extracted\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 14:48:04.367911 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:04.367907 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2154d6d4-6c28-4077-8ce8-2830c1347168-trusted-ca\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 14:48:04.368044 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:04.367917 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfkkp\" (UniqueName: \"kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-kube-api-access-hfkkp\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 14:48:04.368044 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:04.367928 2568 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2154d6d4-6c28-4077-8ce8-2830c1347168-registry-certificates\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 14:48:04.368044 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:04.367936 2568 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-registry-tls\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 14:48:04.368044 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:04.367948 2568 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2154d6d4-6c28-4077-8ce8-2830c1347168-image-registry-private-configuration\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 14:48:04.368044 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:04.367957 2568 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2154d6d4-6c28-4077-8ce8-2830c1347168-bound-sa-token\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 14:48:04.368044 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:04.367964 2568 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2154d6d4-6c28-4077-8ce8-2830c1347168-installation-pull-secrets\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 14:48:05.077011 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:05.076979 2568 generic.go:356] "Generic (PLEG): container finished" podID="2154d6d4-6c28-4077-8ce8-2830c1347168" containerID="f519ed4b3bbba97e0c9eed13590dd43f8ca46fc5e772156043b8e77a0d101bdd" exitCode=0 Jul 13 14:48:05.077411 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:05.077024 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" Jul 13 14:48:05.077411 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:05.077029 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" event={"ID":"2154d6d4-6c28-4077-8ce8-2830c1347168","Type":"ContainerDied","Data":"f519ed4b3bbba97e0c9eed13590dd43f8ca46fc5e772156043b8e77a0d101bdd"} Jul 13 14:48:05.077411 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:05.077126 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6bfb696d64-gvxnc" event={"ID":"2154d6d4-6c28-4077-8ce8-2830c1347168","Type":"ContainerDied","Data":"af8f083935c2b59e3e9e50de52d81fbd3ac4743fe5c4dfbab4518bdb4609d480"} Jul 13 14:48:05.077411 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:05.077159 2568 scope.go:117] "RemoveContainer" containerID="f519ed4b3bbba97e0c9eed13590dd43f8ca46fc5e772156043b8e77a0d101bdd" Jul 13 14:48:05.086228 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:05.086207 2568 scope.go:117] "RemoveContainer" containerID="f519ed4b3bbba97e0c9eed13590dd43f8ca46fc5e772156043b8e77a0d101bdd" Jul 13 14:48:05.086508 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:48:05.086487 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f519ed4b3bbba97e0c9eed13590dd43f8ca46fc5e772156043b8e77a0d101bdd\": container with ID starting with f519ed4b3bbba97e0c9eed13590dd43f8ca46fc5e772156043b8e77a0d101bdd not found: ID does not exist" containerID="f519ed4b3bbba97e0c9eed13590dd43f8ca46fc5e772156043b8e77a0d101bdd" Jul 13 14:48:05.086588 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:05.086519 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f519ed4b3bbba97e0c9eed13590dd43f8ca46fc5e772156043b8e77a0d101bdd"} err="failed to get container status \"f519ed4b3bbba97e0c9eed13590dd43f8ca46fc5e772156043b8e77a0d101bdd\": rpc error: code = NotFound desc = could not find container \"f519ed4b3bbba97e0c9eed13590dd43f8ca46fc5e772156043b8e77a0d101bdd\": container with ID starting with f519ed4b3bbba97e0c9eed13590dd43f8ca46fc5e772156043b8e77a0d101bdd not found: ID does not exist" Jul 13 14:48:05.099645 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:05.099605 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6bfb696d64-gvxnc"] Jul 13 14:48:05.106892 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:05.106869 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6bfb696d64-gvxnc"] Jul 13 14:48:06.143955 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:06.143926 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:48:06.245766 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:06.245736 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:48:06.271064 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:06.271033 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2154d6d4-6c28-4077-8ce8-2830c1347168" path="/var/lib/kubelet/pods/2154d6d4-6c28-4077-8ce8-2830c1347168/volumes" Jul 13 14:48:06.930741 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:06.930708 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-r82bx" Jul 13 14:48:07.587788 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:07.587745 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:48:07.689159 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:07.689123 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:48:07.882188 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:07.882110 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:48:07.983824 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:07.983783 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:48:11.159902 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:11.159865 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:48:11.261568 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:11.261538 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:48:13.546236 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:13.546182 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6c65bfbc8f-kj8xv" Jul 13 14:48:13.546722 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:13.546262 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6c65bfbc8f-kj8xv" Jul 13 14:48:14.773383 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:14.773351 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:48:15.668855 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:15.668818 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:48:15.764285 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:15.764250 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:48:15.901355 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:15.901330 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mgkcj_d8061d1f-15e7-4f20-a3c6-d3b9b865cb0e/serve-healthcheck-canary/0.log" Jul 13 14:48:19.538052 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:19.538017 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:48:19.637742 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:19.637704 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:48:19.712329 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:19.712289 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:48:22.891741 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:22.891705 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:48:22.991061 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:22.991029 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:48:33.551976 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:33.551944 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6c65bfbc8f-kj8xv" Jul 13 14:48:33.555678 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:33.555654 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6c65bfbc8f-kj8xv" Jul 13 14:48:37.874247 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:37.874207 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:48:37.975368 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:37.975336 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:48:41.209070 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:41.209034 2568 generic.go:356] "Generic (PLEG): container finished" podID="a7789b1a-5fd7-4e52-93e4-0bf7e0927ab5" containerID="37cf2f728c71c0aae8f17fa30f6821c1185fae690a4a0974921f3bd4f2505931" exitCode=0 Jul 13 14:48:41.209464 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:41.209097 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-jmpvx" event={"ID":"a7789b1a-5fd7-4e52-93e4-0bf7e0927ab5","Type":"ContainerDied","Data":"37cf2f728c71c0aae8f17fa30f6821c1185fae690a4a0974921f3bd4f2505931"} Jul 13 14:48:41.209464 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:41.209451 2568 scope.go:117] "RemoveContainer" containerID="37cf2f728c71c0aae8f17fa30f6821c1185fae690a4a0974921f3bd4f2505931" Jul 13 14:48:42.214778 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:42.214740 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-jmpvx" event={"ID":"a7789b1a-5fd7-4e52-93e4-0bf7e0927ab5","Type":"ContainerStarted","Data":"f60916de0042e7c59f9b832f78cbbe42e562d3df68dee59f44f580b897771f36"} Jul 13 14:48:48.237069 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:48.236973 2568 generic.go:356] "Generic (PLEG): container finished" podID="9ccf4712-918a-4d3a-808e-d6162b7a0f51" containerID="cfbd7916e99a0dcc0d3f02fa00f31c0d8cf6c1a85c0a733df62e3f14f81ae39c" exitCode=0 Jul 13 14:48:48.237069 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:48.237033 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-k6pzr" event={"ID":"9ccf4712-918a-4d3a-808e-d6162b7a0f51","Type":"ContainerDied","Data":"cfbd7916e99a0dcc0d3f02fa00f31c0d8cf6c1a85c0a733df62e3f14f81ae39c"} Jul 13 14:48:48.237512 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:48.237353 2568 scope.go:117] "RemoveContainer" containerID="cfbd7916e99a0dcc0d3f02fa00f31c0d8cf6c1a85c0a733df62e3f14f81ae39c" Jul 13 14:48:49.242586 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:48:49.242545 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-k6pzr" event={"ID":"9ccf4712-918a-4d3a-808e-d6162b7a0f51","Type":"ContainerStarted","Data":"58a2da317c60a0e437b4c52b657a90cd5d7501a29c7b218e48c2c6d8d9ca7caa"} Jul 13 14:49:04.946521 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:49:04.946459 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:49:05.090215 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:49:05.090187 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:49:05.189741 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:49:05.189709 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:49:14.787692 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:49:14.787652 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:49:14.907390 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:49:14.907353 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:49:16.912081 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:49:16.912045 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:49:16.973500 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:49:16.973471 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:49:22.891625 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:49:22.891519 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:49:22.993487 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:49:22.993455 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:49:25.895477 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:49:25.895408 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:49:25.995605 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:49:25.995554 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:49:26.063787 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:49:26.063750 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:49:45.999747 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:49:45.999710 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:49:46.059565 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:49:46.059507 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:50:09.209002 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:09.208968 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:50:09.306806 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:09.306772 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:50:09.743847 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:09.743816 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xhrj2"] Jul 13 14:50:09.744074 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:09.744063 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2154d6d4-6c28-4077-8ce8-2830c1347168" containerName="registry" Jul 13 14:50:09.744115 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:09.744079 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2154d6d4-6c28-4077-8ce8-2830c1347168" containerName="registry" Jul 13 14:50:09.744152 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:09.744127 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2154d6d4-6c28-4077-8ce8-2830c1347168" containerName="registry" Jul 13 14:50:09.746990 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:09.746973 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xhrj2" Jul 13 14:50:09.751804 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:09.751778 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 13 14:50:09.752323 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:09.752252 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7zqjs\"" Jul 13 14:50:09.752323 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:09.752251 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 13 14:50:09.755559 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:09.755537 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xhrj2"] Jul 13 14:50:09.925731 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:09.925687 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bf080da8-645f-418e-ac66-2992830ff3b1-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xhrj2\" (UID: \"bf080da8-645f-418e-ac66-2992830ff3b1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xhrj2" Jul 13 14:50:09.925731 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:09.925740 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bf080da8-645f-418e-ac66-2992830ff3b1-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xhrj2\" (UID: \"bf080da8-645f-418e-ac66-2992830ff3b1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xhrj2" Jul 13 14:50:09.925956 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:09.925770 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzh5b\" (UniqueName: \"kubernetes.io/projected/bf080da8-645f-418e-ac66-2992830ff3b1-kube-api-access-lzh5b\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xhrj2\" (UID: \"bf080da8-645f-418e-ac66-2992830ff3b1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xhrj2" Jul 13 14:50:10.026966 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:10.026933 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bf080da8-645f-418e-ac66-2992830ff3b1-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xhrj2\" (UID: \"bf080da8-645f-418e-ac66-2992830ff3b1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xhrj2" Jul 13 14:50:10.027106 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:10.026979 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bf080da8-645f-418e-ac66-2992830ff3b1-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xhrj2\" (UID: \"bf080da8-645f-418e-ac66-2992830ff3b1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xhrj2" Jul 13 14:50:10.027106 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:10.027046 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzh5b\" (UniqueName: \"kubernetes.io/projected/bf080da8-645f-418e-ac66-2992830ff3b1-kube-api-access-lzh5b\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xhrj2\" (UID: \"bf080da8-645f-418e-ac66-2992830ff3b1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xhrj2" Jul 13 14:50:10.027393 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:10.027374 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bf080da8-645f-418e-ac66-2992830ff3b1-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xhrj2\" (UID: \"bf080da8-645f-418e-ac66-2992830ff3b1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xhrj2" Jul 13 14:50:10.027429 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:10.027389 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bf080da8-645f-418e-ac66-2992830ff3b1-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xhrj2\" (UID: \"bf080da8-645f-418e-ac66-2992830ff3b1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xhrj2" Jul 13 14:50:10.036620 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:10.036588 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzh5b\" (UniqueName: \"kubernetes.io/projected/bf080da8-645f-418e-ac66-2992830ff3b1-kube-api-access-lzh5b\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xhrj2\" (UID: \"bf080da8-645f-418e-ac66-2992830ff3b1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xhrj2" Jul 13 14:50:10.058697 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:10.058664 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xhrj2" Jul 13 14:50:10.188050 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:10.187998 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xhrj2"] Jul 13 14:50:10.190505 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:50:10.190481 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf080da8_645f_418e_ac66_2992830ff3b1.slice/crio-c887df86794db1ca128724b6f4f4b06f755d2388f77b046581e1f398a515aabf WatchSource:0}: Error finding container c887df86794db1ca128724b6f4f4b06f755d2388f77b046581e1f398a515aabf: Status 404 returned error can't find the container with id c887df86794db1ca128724b6f4f4b06f755d2388f77b046581e1f398a515aabf Jul 13 14:50:10.528284 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:10.528250 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xhrj2" event={"ID":"bf080da8-645f-418e-ac66-2992830ff3b1","Type":"ContainerStarted","Data":"c887df86794db1ca128724b6f4f4b06f755d2388f77b046581e1f398a515aabf"} Jul 13 14:50:12.856511 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:12.856473 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:50:12.910229 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:12.910197 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:50:16.554209 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:16.554173 2568 generic.go:356] "Generic (PLEG): container finished" podID="bf080da8-645f-418e-ac66-2992830ff3b1" containerID="a557aa5cc52986bf39ff1dccb9fd16b1f148bf2bf5a99a9f254023d5f2fe30a5" exitCode=0 Jul 13 14:50:16.554617 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:16.554237 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xhrj2" event={"ID":"bf080da8-645f-418e-ac66-2992830ff3b1","Type":"ContainerDied","Data":"a557aa5cc52986bf39ff1dccb9fd16b1f148bf2bf5a99a9f254023d5f2fe30a5"} Jul 13 14:50:19.566582 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:19.566548 2568 generic.go:356] "Generic (PLEG): container finished" podID="bf080da8-645f-418e-ac66-2992830ff3b1" containerID="880e1e16e098e7681fe8e1e45a8d34d5603b508b37f6540d81264141d3d4f6fd" exitCode=0 Jul 13 14:50:19.566974 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:19.566619 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xhrj2" event={"ID":"bf080da8-645f-418e-ac66-2992830ff3b1","Type":"ContainerDied","Data":"880e1e16e098e7681fe8e1e45a8d34d5603b508b37f6540d81264141d3d4f6fd"} Jul 13 14:50:27.601102 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:27.601064 2568 generic.go:356] "Generic (PLEG): container finished" podID="bf080da8-645f-418e-ac66-2992830ff3b1" containerID="942b41adc106e7811b0c339557ab29d93693047a54d4a1c8f0b3b559cba1f96f" exitCode=0 Jul 13 14:50:27.601518 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:27.601125 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xhrj2" event={"ID":"bf080da8-645f-418e-ac66-2992830ff3b1","Type":"ContainerDied","Data":"942b41adc106e7811b0c339557ab29d93693047a54d4a1c8f0b3b559cba1f96f"} Jul 13 14:50:28.730540 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:28.730515 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xhrj2" Jul 13 14:50:28.785010 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:28.784973 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bf080da8-645f-418e-ac66-2992830ff3b1-util\") pod \"bf080da8-645f-418e-ac66-2992830ff3b1\" (UID: \"bf080da8-645f-418e-ac66-2992830ff3b1\") " Jul 13 14:50:28.785197 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:28.785028 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lzh5b\" (UniqueName: \"kubernetes.io/projected/bf080da8-645f-418e-ac66-2992830ff3b1-kube-api-access-lzh5b\") pod \"bf080da8-645f-418e-ac66-2992830ff3b1\" (UID: \"bf080da8-645f-418e-ac66-2992830ff3b1\") " Jul 13 14:50:28.785197 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:28.785054 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bf080da8-645f-418e-ac66-2992830ff3b1-bundle\") pod \"bf080da8-645f-418e-ac66-2992830ff3b1\" (UID: \"bf080da8-645f-418e-ac66-2992830ff3b1\") " Jul 13 14:50:28.785708 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:28.785682 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf080da8-645f-418e-ac66-2992830ff3b1-bundle" (OuterVolumeSpecName: "bundle") pod "bf080da8-645f-418e-ac66-2992830ff3b1" (UID: "bf080da8-645f-418e-ac66-2992830ff3b1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 14:50:28.787259 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:28.787239 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf080da8-645f-418e-ac66-2992830ff3b1-kube-api-access-lzh5b" (OuterVolumeSpecName: "kube-api-access-lzh5b") pod "bf080da8-645f-418e-ac66-2992830ff3b1" (UID: "bf080da8-645f-418e-ac66-2992830ff3b1"). InnerVolumeSpecName "kube-api-access-lzh5b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 14:50:28.789495 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:28.789472 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf080da8-645f-418e-ac66-2992830ff3b1-util" (OuterVolumeSpecName: "util") pod "bf080da8-645f-418e-ac66-2992830ff3b1" (UID: "bf080da8-645f-418e-ac66-2992830ff3b1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 14:50:28.886469 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:28.886374 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bf080da8-645f-418e-ac66-2992830ff3b1-bundle\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 14:50:28.886469 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:28.886408 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bf080da8-645f-418e-ac66-2992830ff3b1-util\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 14:50:28.886469 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:28.886417 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzh5b\" (UniqueName: \"kubernetes.io/projected/bf080da8-645f-418e-ac66-2992830ff3b1-kube-api-access-lzh5b\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 14:50:29.610014 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:29.609977 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xhrj2" event={"ID":"bf080da8-645f-418e-ac66-2992830ff3b1","Type":"ContainerDied","Data":"c887df86794db1ca128724b6f4f4b06f755d2388f77b046581e1f398a515aabf"} Jul 13 14:50:29.610217 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:29.610024 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c887df86794db1ca128724b6f4f4b06f755d2388f77b046581e1f398a515aabf" Jul 13 14:50:29.610217 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:29.610024 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xhrj2" Jul 13 14:50:30.535272 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:30.535231 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:50:31.981955 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:31.981923 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:50:32.512447 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:32.512418 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q9z7w"] Jul 13 14:50:32.512717 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:32.512704 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf080da8-645f-418e-ac66-2992830ff3b1" containerName="util" Jul 13 14:50:32.512765 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:32.512729 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf080da8-645f-418e-ac66-2992830ff3b1" containerName="util" Jul 13 14:50:32.512765 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:32.512745 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf080da8-645f-418e-ac66-2992830ff3b1" containerName="pull" Jul 13 14:50:32.512765 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:32.512750 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf080da8-645f-418e-ac66-2992830ff3b1" containerName="pull" Jul 13 14:50:32.512765 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:32.512758 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf080da8-645f-418e-ac66-2992830ff3b1" containerName="extract" Jul 13 14:50:32.512765 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:32.512764 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf080da8-645f-418e-ac66-2992830ff3b1" containerName="extract" Jul 13 14:50:32.512907 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:32.512801 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf080da8-645f-418e-ac66-2992830ff3b1" containerName="extract" Jul 13 14:50:32.543274 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:32.543246 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q9z7w"] Jul 13 14:50:32.543435 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:32.543335 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q9z7w" Jul 13 14:50:32.545992 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:32.545969 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 13 14:50:32.545992 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:32.545969 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-4q9l7\"" Jul 13 14:50:32.546196 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:32.545999 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 13 14:50:32.613622 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:32.613562 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8846831f-96eb-449c-aa92-a7f64d38c1f5-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-q9z7w\" (UID: \"8846831f-96eb-449c-aa92-a7f64d38c1f5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q9z7w" Jul 13 14:50:32.613785 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:32.613631 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb9nr\" (UniqueName: \"kubernetes.io/projected/8846831f-96eb-449c-aa92-a7f64d38c1f5-kube-api-access-bb9nr\") pod \"cert-manager-operator-controller-manager-55c444bf86-q9z7w\" (UID: \"8846831f-96eb-449c-aa92-a7f64d38c1f5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q9z7w" Jul 13 14:50:32.714220 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:32.714181 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8846831f-96eb-449c-aa92-a7f64d38c1f5-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-q9z7w\" (UID: \"8846831f-96eb-449c-aa92-a7f64d38c1f5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q9z7w" Jul 13 14:50:32.714390 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:32.714233 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb9nr\" (UniqueName: \"kubernetes.io/projected/8846831f-96eb-449c-aa92-a7f64d38c1f5-kube-api-access-bb9nr\") pod \"cert-manager-operator-controller-manager-55c444bf86-q9z7w\" (UID: \"8846831f-96eb-449c-aa92-a7f64d38c1f5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q9z7w" Jul 13 14:50:32.714593 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:32.714550 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8846831f-96eb-449c-aa92-a7f64d38c1f5-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-q9z7w\" (UID: \"8846831f-96eb-449c-aa92-a7f64d38c1f5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q9z7w" Jul 13 14:50:32.723151 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:32.723127 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb9nr\" (UniqueName: \"kubernetes.io/projected/8846831f-96eb-449c-aa92-a7f64d38c1f5-kube-api-access-bb9nr\") pod \"cert-manager-operator-controller-manager-55c444bf86-q9z7w\" (UID: \"8846831f-96eb-449c-aa92-a7f64d38c1f5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q9z7w" Jul 13 14:50:32.856706 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:32.856667 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q9z7w" Jul 13 14:50:33.007188 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:33.007104 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q9z7w"] Jul 13 14:50:33.009478 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:50:33.009450 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8846831f_96eb_449c_aa92_a7f64d38c1f5.slice/crio-5a51081ccc4e411e04a9117745181fd369ecbac61f6a6b65d7c5d7fa70c71c5e WatchSource:0}: Error finding container 5a51081ccc4e411e04a9117745181fd369ecbac61f6a6b65d7c5d7fa70c71c5e: Status 404 returned error can't find the container with id 5a51081ccc4e411e04a9117745181fd369ecbac61f6a6b65d7c5d7fa70c71c5e Jul 13 14:50:33.626237 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:33.626205 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q9z7w" event={"ID":"8846831f-96eb-449c-aa92-a7f64d38c1f5","Type":"ContainerStarted","Data":"5a51081ccc4e411e04a9117745181fd369ecbac61f6a6b65d7c5d7fa70c71c5e"} Jul 13 14:50:34.147636 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:34.147592 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:50:34.205242 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:34.205186 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:50:36.639219 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:36.639181 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q9z7w" event={"ID":"8846831f-96eb-449c-aa92-a7f64d38c1f5","Type":"ContainerStarted","Data":"a8407dc0cc87009a509b6895b334de079ce827ecc32724396e6630522553efc6"} Jul 13 14:50:36.643477 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:36.643449 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:50:36.660927 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:36.660880 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q9z7w" podStartSLOduration=1.583306852 podStartE2EDuration="4.660863881s" totalImagesPullingTime="3.077557029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 14:50:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:50:36.659814143 +0000 UTC m=+248.900091992" watchObservedRunningTime="2026-07-13 14:50:36.660863881 +0000 UTC m=+248.901141731" Jul 13 14:50:36.748420 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:36.748382 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:50:37.051903 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:37.051859 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:50:37.162471 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:37.162439 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:50:37.411775 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:37.411692 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7rh9v"] Jul 13 14:50:37.414116 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:37.414097 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7rh9v" Jul 13 14:50:37.416558 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:37.416532 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 13 14:50:37.416715 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:37.416586 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7zqjs\"" Jul 13 14:50:37.417442 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:37.417423 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 13 14:50:37.424973 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:37.424952 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7rh9v"] Jul 13 14:50:37.450438 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:37.450408 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt5rh\" (UniqueName: \"kubernetes.io/projected/881c0b54-817e-4d32-b6e3-01eb092827b0-kube-api-access-pt5rh\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7rh9v\" (UID: \"881c0b54-817e-4d32-b6e3-01eb092827b0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7rh9v" Jul 13 14:50:37.450606 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:37.450478 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/881c0b54-817e-4d32-b6e3-01eb092827b0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7rh9v\" (UID: \"881c0b54-817e-4d32-b6e3-01eb092827b0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7rh9v" Jul 13 14:50:37.450606 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:37.450497 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/881c0b54-817e-4d32-b6e3-01eb092827b0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7rh9v\" (UID: \"881c0b54-817e-4d32-b6e3-01eb092827b0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7rh9v" Jul 13 14:50:37.551813 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:37.551778 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/881c0b54-817e-4d32-b6e3-01eb092827b0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7rh9v\" (UID: \"881c0b54-817e-4d32-b6e3-01eb092827b0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7rh9v" Jul 13 14:50:37.551993 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:37.551815 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/881c0b54-817e-4d32-b6e3-01eb092827b0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7rh9v\" (UID: \"881c0b54-817e-4d32-b6e3-01eb092827b0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7rh9v" Jul 13 14:50:37.551993 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:37.551863 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt5rh\" (UniqueName: \"kubernetes.io/projected/881c0b54-817e-4d32-b6e3-01eb092827b0-kube-api-access-pt5rh\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7rh9v\" (UID: \"881c0b54-817e-4d32-b6e3-01eb092827b0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7rh9v" Jul 13 14:50:37.552692 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:37.552648 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/881c0b54-817e-4d32-b6e3-01eb092827b0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7rh9v\" (UID: \"881c0b54-817e-4d32-b6e3-01eb092827b0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7rh9v" Jul 13 14:50:37.552853 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:37.552833 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/881c0b54-817e-4d32-b6e3-01eb092827b0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7rh9v\" (UID: \"881c0b54-817e-4d32-b6e3-01eb092827b0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7rh9v" Jul 13 14:50:37.561221 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:37.561200 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt5rh\" (UniqueName: \"kubernetes.io/projected/881c0b54-817e-4d32-b6e3-01eb092827b0-kube-api-access-pt5rh\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7rh9v\" (UID: \"881c0b54-817e-4d32-b6e3-01eb092827b0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7rh9v" Jul 13 14:50:37.723416 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:37.723314 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7rh9v" Jul 13 14:50:37.848149 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:37.848054 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7rh9v"] Jul 13 14:50:37.850389 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:50:37.850362 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod881c0b54_817e_4d32_b6e3_01eb092827b0.slice/crio-a37f42fa9bc36096dd1c9d9af391e27b79c2d34e25c3b285feb87447cbbc7198 WatchSource:0}: Error finding container a37f42fa9bc36096dd1c9d9af391e27b79c2d34e25c3b285feb87447cbbc7198: Status 404 returned error can't find the container with id a37f42fa9bc36096dd1c9d9af391e27b79c2d34e25c3b285feb87447cbbc7198 Jul 13 14:50:38.648130 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:38.648096 2568 generic.go:356] "Generic (PLEG): container finished" podID="881c0b54-817e-4d32-b6e3-01eb092827b0" containerID="0ebfb11e799289290915972c4cf48eefbda7043b2947e321747e5fd1aa6e622f" exitCode=0 Jul 13 14:50:38.648300 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:38.648184 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7rh9v" event={"ID":"881c0b54-817e-4d32-b6e3-01eb092827b0","Type":"ContainerDied","Data":"0ebfb11e799289290915972c4cf48eefbda7043b2947e321747e5fd1aa6e622f"} Jul 13 14:50:38.648300 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:38.648236 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7rh9v" event={"ID":"881c0b54-817e-4d32-b6e3-01eb092827b0","Type":"ContainerStarted","Data":"a37f42fa9bc36096dd1c9d9af391e27b79c2d34e25c3b285feb87447cbbc7198"} Jul 13 14:50:38.864874 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:38.864784 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:50:39.057931 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:39.057887 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:50:41.663804 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:41.663767 2568 generic.go:356] "Generic (PLEG): container finished" podID="881c0b54-817e-4d32-b6e3-01eb092827b0" containerID="35b47a0617e617c56259855736e8d5f11475f39163f357aac2af25206ddff3ea" exitCode=0 Jul 13 14:50:41.664181 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:41.663808 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7rh9v" event={"ID":"881c0b54-817e-4d32-b6e3-01eb092827b0","Type":"ContainerDied","Data":"35b47a0617e617c56259855736e8d5f11475f39163f357aac2af25206ddff3ea"} Jul 13 14:50:42.428052 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:42.427991 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:50:42.483802 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:42.483762 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:50:42.670179 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:42.670143 2568 generic.go:356] "Generic (PLEG): container finished" podID="881c0b54-817e-4d32-b6e3-01eb092827b0" containerID="5de0cea6a88a05ac9e6a2f0a78bd2a9a9c9e6a4d0d51f2043ac937c3f2fd6f6c" exitCode=0 Jul 13 14:50:42.670531 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:42.670205 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7rh9v" event={"ID":"881c0b54-817e-4d32-b6e3-01eb092827b0","Type":"ContainerDied","Data":"5de0cea6a88a05ac9e6a2f0a78bd2a9a9c9e6a4d0d51f2043ac937c3f2fd6f6c"} Jul 13 14:50:43.796081 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:43.796057 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7rh9v" Jul 13 14:50:43.902768 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:43.902731 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/881c0b54-817e-4d32-b6e3-01eb092827b0-util\") pod \"881c0b54-817e-4d32-b6e3-01eb092827b0\" (UID: \"881c0b54-817e-4d32-b6e3-01eb092827b0\") " Jul 13 14:50:43.902768 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:43.902772 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/881c0b54-817e-4d32-b6e3-01eb092827b0-bundle\") pod \"881c0b54-817e-4d32-b6e3-01eb092827b0\" (UID: \"881c0b54-817e-4d32-b6e3-01eb092827b0\") " Jul 13 14:50:43.903005 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:43.902804 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pt5rh\" (UniqueName: \"kubernetes.io/projected/881c0b54-817e-4d32-b6e3-01eb092827b0-kube-api-access-pt5rh\") pod \"881c0b54-817e-4d32-b6e3-01eb092827b0\" (UID: \"881c0b54-817e-4d32-b6e3-01eb092827b0\") " Jul 13 14:50:43.903170 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:43.903149 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/881c0b54-817e-4d32-b6e3-01eb092827b0-bundle" (OuterVolumeSpecName: "bundle") pod "881c0b54-817e-4d32-b6e3-01eb092827b0" (UID: "881c0b54-817e-4d32-b6e3-01eb092827b0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 14:50:43.904901 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:43.904877 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/881c0b54-817e-4d32-b6e3-01eb092827b0-kube-api-access-pt5rh" (OuterVolumeSpecName: "kube-api-access-pt5rh") pod "881c0b54-817e-4d32-b6e3-01eb092827b0" (UID: "881c0b54-817e-4d32-b6e3-01eb092827b0"). InnerVolumeSpecName "kube-api-access-pt5rh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 14:50:43.907478 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:43.907450 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/881c0b54-817e-4d32-b6e3-01eb092827b0-util" (OuterVolumeSpecName: "util") pod "881c0b54-817e-4d32-b6e3-01eb092827b0" (UID: "881c0b54-817e-4d32-b6e3-01eb092827b0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 14:50:44.004000 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:44.003918 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pt5rh\" (UniqueName: \"kubernetes.io/projected/881c0b54-817e-4d32-b6e3-01eb092827b0-kube-api-access-pt5rh\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 14:50:44.004000 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:44.003946 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/881c0b54-817e-4d32-b6e3-01eb092827b0-util\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 14:50:44.004000 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:44.003958 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/881c0b54-817e-4d32-b6e3-01eb092827b0-bundle\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 14:50:44.679933 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:44.679841 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7rh9v" event={"ID":"881c0b54-817e-4d32-b6e3-01eb092827b0","Type":"ContainerDied","Data":"a37f42fa9bc36096dd1c9d9af391e27b79c2d34e25c3b285feb87447cbbc7198"} Jul 13 14:50:44.679933 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:44.679863 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7rh9v" Jul 13 14:50:44.679933 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:44.679888 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a37f42fa9bc36096dd1c9d9af391e27b79c2d34e25c3b285feb87447cbbc7198" Jul 13 14:50:46.873842 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:46.873806 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:50:47.134700 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:47.134619 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-cncfx"] Jul 13 14:50:47.134892 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:47.134879 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="881c0b54-817e-4d32-b6e3-01eb092827b0" containerName="extract" Jul 13 14:50:47.134939 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:47.134894 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="881c0b54-817e-4d32-b6e3-01eb092827b0" containerName="extract" Jul 13 14:50:47.134939 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:47.134904 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="881c0b54-817e-4d32-b6e3-01eb092827b0" containerName="util" Jul 13 14:50:47.134939 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:47.134909 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="881c0b54-817e-4d32-b6e3-01eb092827b0" containerName="util" Jul 13 14:50:47.134939 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:47.134921 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="881c0b54-817e-4d32-b6e3-01eb092827b0" containerName="pull" Jul 13 14:50:47.134939 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:47.134926 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="881c0b54-817e-4d32-b6e3-01eb092827b0" containerName="pull" Jul 13 14:50:47.135101 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:47.134968 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="881c0b54-817e-4d32-b6e3-01eb092827b0" containerName="extract" Jul 13 14:50:47.138184 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:47.138167 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-cncfx" Jul 13 14:50:47.140669 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:47.140644 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 13 14:50:47.140817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:47.140711 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 13 14:50:47.141422 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:47.141408 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-s6cwv\"" Jul 13 14:50:47.148909 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:47.148883 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-cncfx"] Jul 13 14:50:47.230633 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:47.230567 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zf4r8\" (UniqueName: \"kubernetes.io/projected/94e4b45c-408a-4b07-a3d9-259249a9bd1d-kube-api-access-zf4r8\") pod \"cert-manager-55b4d954d5-cncfx\" (UID: \"94e4b45c-408a-4b07-a3d9-259249a9bd1d\") " pod="cert-manager/cert-manager-55b4d954d5-cncfx" Jul 13 14:50:47.230785 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:47.230711 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/94e4b45c-408a-4b07-a3d9-259249a9bd1d-bound-sa-token\") pod \"cert-manager-55b4d954d5-cncfx\" (UID: \"94e4b45c-408a-4b07-a3d9-259249a9bd1d\") " pod="cert-manager/cert-manager-55b4d954d5-cncfx" Jul 13 14:50:47.331176 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:47.331144 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/94e4b45c-408a-4b07-a3d9-259249a9bd1d-bound-sa-token\") pod \"cert-manager-55b4d954d5-cncfx\" (UID: \"94e4b45c-408a-4b07-a3d9-259249a9bd1d\") " pod="cert-manager/cert-manager-55b4d954d5-cncfx" Jul 13 14:50:47.331340 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:47.331183 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zf4r8\" (UniqueName: \"kubernetes.io/projected/94e4b45c-408a-4b07-a3d9-259249a9bd1d-kube-api-access-zf4r8\") pod \"cert-manager-55b4d954d5-cncfx\" (UID: \"94e4b45c-408a-4b07-a3d9-259249a9bd1d\") " pod="cert-manager/cert-manager-55b4d954d5-cncfx" Jul 13 14:50:47.340136 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:47.340107 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/94e4b45c-408a-4b07-a3d9-259249a9bd1d-bound-sa-token\") pod \"cert-manager-55b4d954d5-cncfx\" (UID: \"94e4b45c-408a-4b07-a3d9-259249a9bd1d\") " pod="cert-manager/cert-manager-55b4d954d5-cncfx" Jul 13 14:50:47.340239 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:47.340212 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zf4r8\" (UniqueName: \"kubernetes.io/projected/94e4b45c-408a-4b07-a3d9-259249a9bd1d-kube-api-access-zf4r8\") pod \"cert-manager-55b4d954d5-cncfx\" (UID: \"94e4b45c-408a-4b07-a3d9-259249a9bd1d\") " pod="cert-manager/cert-manager-55b4d954d5-cncfx" Jul 13 14:50:47.448437 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:47.448341 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-cncfx" Jul 13 14:50:47.576377 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:47.576345 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-cncfx"] Jul 13 14:50:47.579380 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:50:47.579351 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod94e4b45c_408a_4b07_a3d9_259249a9bd1d.slice/crio-e7a8bb12ee2f4bdeb6f7ab774911b00547cafafb3444a42271d46ad2da738485 WatchSource:0}: Error finding container e7a8bb12ee2f4bdeb6f7ab774911b00547cafafb3444a42271d46ad2da738485: Status 404 returned error can't find the container with id e7a8bb12ee2f4bdeb6f7ab774911b00547cafafb3444a42271d46ad2da738485 Jul 13 14:50:47.693522 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:47.693489 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-cncfx" event={"ID":"94e4b45c-408a-4b07-a3d9-259249a9bd1d","Type":"ContainerStarted","Data":"e7a8bb12ee2f4bdeb6f7ab774911b00547cafafb3444a42271d46ad2da738485"} Jul 13 14:50:48.359448 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:48.359408 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:50:48.469787 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:48.469752 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:50:50.592767 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:50.592727 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n9fx8"] Jul 13 14:50:50.595752 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:50.595735 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n9fx8" Jul 13 14:50:50.598200 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:50.598175 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 13 14:50:50.599118 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:50.599096 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-ddxcj\"" Jul 13 14:50:50.599234 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:50.599127 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 13 14:50:50.605599 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:50.605560 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n9fx8"] Jul 13 14:50:50.661670 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:50.661628 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e643ce43-0221-45f9-b421-c9320445f31b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-n9fx8\" (UID: \"e643ce43-0221-45f9-b421-c9320445f31b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n9fx8" Jul 13 14:50:50.661848 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:50.661679 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5g8j5\" (UniqueName: \"kubernetes.io/projected/e643ce43-0221-45f9-b421-c9320445f31b-kube-api-access-5g8j5\") pod \"openshift-lws-operator-fd8ccff4c-n9fx8\" (UID: \"e643ce43-0221-45f9-b421-c9320445f31b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n9fx8" Jul 13 14:50:50.708041 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:50.708005 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-cncfx" event={"ID":"94e4b45c-408a-4b07-a3d9-259249a9bd1d","Type":"ContainerStarted","Data":"a65f55d3cd23ae577aabe11bd5fdbc28a1b9ba9a3f00842747ff50618bfb3fbd"} Jul 13 14:50:50.725732 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:50.725684 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-cncfx" podStartSLOduration=1.14707775 podStartE2EDuration="3.725666856s" totalImagesPullingTime="2.578589106s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 14:50:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:50:50.724734532 +0000 UTC m=+262.965012384" watchObservedRunningTime="2026-07-13 14:50:50.725666856 +0000 UTC m=+262.965944705" Jul 13 14:50:50.762120 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:50.762086 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e643ce43-0221-45f9-b421-c9320445f31b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-n9fx8\" (UID: \"e643ce43-0221-45f9-b421-c9320445f31b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n9fx8" Jul 13 14:50:50.762301 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:50.762143 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g8j5\" (UniqueName: \"kubernetes.io/projected/e643ce43-0221-45f9-b421-c9320445f31b-kube-api-access-5g8j5\") pod \"openshift-lws-operator-fd8ccff4c-n9fx8\" (UID: \"e643ce43-0221-45f9-b421-c9320445f31b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n9fx8" Jul 13 14:50:50.762462 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:50.762441 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e643ce43-0221-45f9-b421-c9320445f31b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-n9fx8\" (UID: \"e643ce43-0221-45f9-b421-c9320445f31b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n9fx8" Jul 13 14:50:50.770707 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:50.770678 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5g8j5\" (UniqueName: \"kubernetes.io/projected/e643ce43-0221-45f9-b421-c9320445f31b-kube-api-access-5g8j5\") pod \"openshift-lws-operator-fd8ccff4c-n9fx8\" (UID: \"e643ce43-0221-45f9-b421-c9320445f31b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n9fx8" Jul 13 14:50:50.905116 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:50.905022 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n9fx8" Jul 13 14:50:51.064092 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:51.064065 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n9fx8"] Jul 13 14:50:51.066402 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:50:51.066374 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode643ce43_0221_45f9_b421_c9320445f31b.slice/crio-b7a05cd6e923964da4b841cb4eeb8ffa5f732cbc80a802e674944f682ece84d0 WatchSource:0}: Error finding container b7a05cd6e923964da4b841cb4eeb8ffa5f732cbc80a802e674944f682ece84d0: Status 404 returned error can't find the container with id b7a05cd6e923964da4b841cb4eeb8ffa5f732cbc80a802e674944f682ece84d0 Jul 13 14:50:51.713514 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:51.713472 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n9fx8" event={"ID":"e643ce43-0221-45f9-b421-c9320445f31b","Type":"ContainerStarted","Data":"b7a05cd6e923964da4b841cb4eeb8ffa5f732cbc80a802e674944f682ece84d0"} Jul 13 14:50:53.723105 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:53.723068 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n9fx8" event={"ID":"e643ce43-0221-45f9-b421-c9320445f31b","Type":"ContainerStarted","Data":"4f938910042819bd48422512291e7baee04fd616c5b5e571fb0fc0ea6857a4cb"} Jul 13 14:50:53.741041 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:53.740986 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n9fx8" podStartSLOduration=2.108929615 podStartE2EDuration="3.740971583s" totalImagesPullingTime="1.632041968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 14:50:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:50:53.739629055 +0000 UTC m=+265.979906907" watchObservedRunningTime="2026-07-13 14:50:53.740971583 +0000 UTC m=+265.981249432" Jul 13 14:50:54.563033 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:54.562999 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:50:54.657861 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:54.657824 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:50:56.225783 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:56.225748 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:50:58.029182 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:58.029149 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmxpkj"] Jul 13 14:50:58.048622 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:58.048594 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmxpkj"] Jul 13 14:50:58.048776 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:58.048671 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmxpkj" Jul 13 14:50:58.051331 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:58.051310 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 13 14:50:58.051505 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:58.051361 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 13 14:50:58.052053 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:58.052039 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7zqjs\"" Jul 13 14:50:58.121529 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:58.121487 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j24fb\" (UniqueName: \"kubernetes.io/projected/c56256e5-1aab-45fd-aa83-5a8e5e1f0214-kube-api-access-j24fb\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmxpkj\" (UID: \"c56256e5-1aab-45fd-aa83-5a8e5e1f0214\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmxpkj" Jul 13 14:50:58.121735 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:58.121542 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c56256e5-1aab-45fd-aa83-5a8e5e1f0214-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmxpkj\" (UID: \"c56256e5-1aab-45fd-aa83-5a8e5e1f0214\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmxpkj" Jul 13 14:50:58.121735 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:58.121620 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c56256e5-1aab-45fd-aa83-5a8e5e1f0214-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmxpkj\" (UID: \"c56256e5-1aab-45fd-aa83-5a8e5e1f0214\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmxpkj" Jul 13 14:50:58.223031 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:58.222994 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j24fb\" (UniqueName: \"kubernetes.io/projected/c56256e5-1aab-45fd-aa83-5a8e5e1f0214-kube-api-access-j24fb\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmxpkj\" (UID: \"c56256e5-1aab-45fd-aa83-5a8e5e1f0214\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmxpkj" Jul 13 14:50:58.223241 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:58.223046 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c56256e5-1aab-45fd-aa83-5a8e5e1f0214-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmxpkj\" (UID: \"c56256e5-1aab-45fd-aa83-5a8e5e1f0214\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmxpkj" Jul 13 14:50:58.223241 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:58.223081 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c56256e5-1aab-45fd-aa83-5a8e5e1f0214-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmxpkj\" (UID: \"c56256e5-1aab-45fd-aa83-5a8e5e1f0214\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmxpkj" Jul 13 14:50:58.223513 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:58.223484 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c56256e5-1aab-45fd-aa83-5a8e5e1f0214-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmxpkj\" (UID: \"c56256e5-1aab-45fd-aa83-5a8e5e1f0214\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmxpkj" Jul 13 14:50:58.223610 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:58.223543 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c56256e5-1aab-45fd-aa83-5a8e5e1f0214-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmxpkj\" (UID: \"c56256e5-1aab-45fd-aa83-5a8e5e1f0214\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmxpkj" Jul 13 14:50:58.232485 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:58.232457 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j24fb\" (UniqueName: \"kubernetes.io/projected/c56256e5-1aab-45fd-aa83-5a8e5e1f0214-kube-api-access-j24fb\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmxpkj\" (UID: \"c56256e5-1aab-45fd-aa83-5a8e5e1f0214\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmxpkj" Jul 13 14:50:58.358435 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:58.358343 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmxpkj" Jul 13 14:50:58.510859 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:58.510834 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmxpkj"] Jul 13 14:50:58.513470 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:50:58.513439 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc56256e5_1aab_45fd_aa83_5a8e5e1f0214.slice/crio-84ca00de171533063b93a85987372fd44e8b24186b3adf0a1485efd3e0c45a96 WatchSource:0}: Error finding container 84ca00de171533063b93a85987372fd44e8b24186b3adf0a1485efd3e0c45a96: Status 404 returned error can't find the container with id 84ca00de171533063b93a85987372fd44e8b24186b3adf0a1485efd3e0c45a96 Jul 13 14:50:58.743869 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:58.743833 2568 generic.go:356] "Generic (PLEG): container finished" podID="c56256e5-1aab-45fd-aa83-5a8e5e1f0214" containerID="5d2b061218b501306ed5bbf9cf15d5849186b2880f6d0a8fb9c6df6d7056dcaa" exitCode=0 Jul 13 14:50:58.744037 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:58.743910 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmxpkj" event={"ID":"c56256e5-1aab-45fd-aa83-5a8e5e1f0214","Type":"ContainerDied","Data":"5d2b061218b501306ed5bbf9cf15d5849186b2880f6d0a8fb9c6df6d7056dcaa"} Jul 13 14:50:58.744037 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:50:58.743948 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmxpkj" event={"ID":"c56256e5-1aab-45fd-aa83-5a8e5e1f0214","Type":"ContainerStarted","Data":"84ca00de171533063b93a85987372fd44e8b24186b3adf0a1485efd3e0c45a96"} Jul 13 14:51:00.730495 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:00.730458 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:51:00.753953 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:00.753920 2568 generic.go:356] "Generic (PLEG): container finished" podID="c56256e5-1aab-45fd-aa83-5a8e5e1f0214" containerID="95e5358a0d056a7be41439a63db07b544ae7ca79189973a919fcba53c4219d5a" exitCode=0 Jul 13 14:51:00.754150 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:00.753982 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmxpkj" event={"ID":"c56256e5-1aab-45fd-aa83-5a8e5e1f0214","Type":"ContainerDied","Data":"95e5358a0d056a7be41439a63db07b544ae7ca79189973a919fcba53c4219d5a"} Jul 13 14:51:01.555895 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:01.555861 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:51:01.759881 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:01.759836 2568 generic.go:356] "Generic (PLEG): container finished" podID="c56256e5-1aab-45fd-aa83-5a8e5e1f0214" containerID="eadad61feca2aa0d1734844526e4fd46cd3a17b90c3896ba0ac920f74be59972" exitCode=0 Jul 13 14:51:01.759881 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:01.759887 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmxpkj" event={"ID":"c56256e5-1aab-45fd-aa83-5a8e5e1f0214","Type":"ContainerDied","Data":"eadad61feca2aa0d1734844526e4fd46cd3a17b90c3896ba0ac920f74be59972"} Jul 13 14:51:02.885667 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:02.885638 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmxpkj" Jul 13 14:51:03.064284 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:03.064245 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c56256e5-1aab-45fd-aa83-5a8e5e1f0214-bundle\") pod \"c56256e5-1aab-45fd-aa83-5a8e5e1f0214\" (UID: \"c56256e5-1aab-45fd-aa83-5a8e5e1f0214\") " Jul 13 14:51:03.064486 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:03.064318 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c56256e5-1aab-45fd-aa83-5a8e5e1f0214-util\") pod \"c56256e5-1aab-45fd-aa83-5a8e5e1f0214\" (UID: \"c56256e5-1aab-45fd-aa83-5a8e5e1f0214\") " Jul 13 14:51:03.064486 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:03.064345 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j24fb\" (UniqueName: \"kubernetes.io/projected/c56256e5-1aab-45fd-aa83-5a8e5e1f0214-kube-api-access-j24fb\") pod \"c56256e5-1aab-45fd-aa83-5a8e5e1f0214\" (UID: \"c56256e5-1aab-45fd-aa83-5a8e5e1f0214\") " Jul 13 14:51:03.065131 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:03.065095 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c56256e5-1aab-45fd-aa83-5a8e5e1f0214-bundle" (OuterVolumeSpecName: "bundle") pod "c56256e5-1aab-45fd-aa83-5a8e5e1f0214" (UID: "c56256e5-1aab-45fd-aa83-5a8e5e1f0214"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 14:51:03.066461 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:03.066437 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c56256e5-1aab-45fd-aa83-5a8e5e1f0214-kube-api-access-j24fb" (OuterVolumeSpecName: "kube-api-access-j24fb") pod "c56256e5-1aab-45fd-aa83-5a8e5e1f0214" (UID: "c56256e5-1aab-45fd-aa83-5a8e5e1f0214"). InnerVolumeSpecName "kube-api-access-j24fb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 14:51:03.072493 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:03.072467 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c56256e5-1aab-45fd-aa83-5a8e5e1f0214-util" (OuterVolumeSpecName: "util") pod "c56256e5-1aab-45fd-aa83-5a8e5e1f0214" (UID: "c56256e5-1aab-45fd-aa83-5a8e5e1f0214"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 14:51:03.165289 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:03.165254 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c56256e5-1aab-45fd-aa83-5a8e5e1f0214-bundle\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 14:51:03.165289 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:03.165281 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c56256e5-1aab-45fd-aa83-5a8e5e1f0214-util\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 14:51:03.165289 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:03.165292 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j24fb\" (UniqueName: \"kubernetes.io/projected/c56256e5-1aab-45fd-aa83-5a8e5e1f0214-kube-api-access-j24fb\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 14:51:03.770235 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:03.770210 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmxpkj" Jul 13 14:51:03.770400 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:03.770204 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmxpkj" event={"ID":"c56256e5-1aab-45fd-aa83-5a8e5e1f0214","Type":"ContainerDied","Data":"84ca00de171533063b93a85987372fd44e8b24186b3adf0a1485efd3e0c45a96"} Jul 13 14:51:03.770400 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:03.770302 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="84ca00de171533063b93a85987372fd44e8b24186b3adf0a1485efd3e0c45a96" Jul 13 14:51:08.365885 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:08.365842 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:51:08.455878 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:08.455834 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:51:08.967955 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:08.967919 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:51:09.288853 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.288823 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:51:09.390067 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.390035 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:51:09.427952 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.427922 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6496967d77-f5ptc"] Jul 13 14:51:09.428201 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.428190 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c56256e5-1aab-45fd-aa83-5a8e5e1f0214" containerName="pull" Jul 13 14:51:09.428245 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.428203 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c56256e5-1aab-45fd-aa83-5a8e5e1f0214" containerName="pull" Jul 13 14:51:09.428245 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.428214 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c56256e5-1aab-45fd-aa83-5a8e5e1f0214" containerName="extract" Jul 13 14:51:09.428245 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.428220 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c56256e5-1aab-45fd-aa83-5a8e5e1f0214" containerName="extract" Jul 13 14:51:09.428245 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.428233 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c56256e5-1aab-45fd-aa83-5a8e5e1f0214" containerName="util" Jul 13 14:51:09.428245 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.428239 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c56256e5-1aab-45fd-aa83-5a8e5e1f0214" containerName="util" Jul 13 14:51:09.428385 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.428288 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c56256e5-1aab-45fd-aa83-5a8e5e1f0214" containerName="extract" Jul 13 14:51:09.433957 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.433936 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6496967d77-f5ptc" Jul 13 14:51:09.437158 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.437133 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-knj5m\"" Jul 13 14:51:09.437303 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.437284 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 13 14:51:09.437546 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.437530 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 13 14:51:09.437659 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.437642 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 13 14:51:09.437984 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.437963 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 13 14:51:09.452096 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.452066 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6496967d77-f5ptc"] Jul 13 14:51:09.613150 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.613052 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:51:09.613380 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.613363 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbrrz\" (UniqueName: \"kubernetes.io/projected/94eff779-7543-4a52-af77-debeeb23f1f9-kube-api-access-kbrrz\") pod \"opendatahub-operator-controller-manager-6496967d77-f5ptc\" (UID: \"94eff779-7543-4a52-af77-debeeb23f1f9\") " pod="opendatahub/opendatahub-operator-controller-manager-6496967d77-f5ptc" Jul 13 14:51:09.613445 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.613392 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/94eff779-7543-4a52-af77-debeeb23f1f9-manifests\") pod \"opendatahub-operator-controller-manager-6496967d77-f5ptc\" (UID: \"94eff779-7543-4a52-af77-debeeb23f1f9\") " pod="opendatahub/opendatahub-operator-controller-manager-6496967d77-f5ptc" Jul 13 14:51:09.613483 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.613467 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94eff779-7543-4a52-af77-debeeb23f1f9-tmp\") pod \"opendatahub-operator-controller-manager-6496967d77-f5ptc\" (UID: \"94eff779-7543-4a52-af77-debeeb23f1f9\") " pod="opendatahub/opendatahub-operator-controller-manager-6496967d77-f5ptc" Jul 13 14:51:09.613523 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.613485 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/94eff779-7543-4a52-af77-debeeb23f1f9-webhook-cert\") pod \"opendatahub-operator-controller-manager-6496967d77-f5ptc\" (UID: \"94eff779-7543-4a52-af77-debeeb23f1f9\") " pod="opendatahub/opendatahub-operator-controller-manager-6496967d77-f5ptc" Jul 13 14:51:09.613558 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.613538 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/94eff779-7543-4a52-af77-debeeb23f1f9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6496967d77-f5ptc\" (UID: \"94eff779-7543-4a52-af77-debeeb23f1f9\") " pod="opendatahub/opendatahub-operator-controller-manager-6496967d77-f5ptc" Jul 13 14:51:09.713184 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.713155 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:51:09.714252 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.714232 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94eff779-7543-4a52-af77-debeeb23f1f9-tmp\") pod \"opendatahub-operator-controller-manager-6496967d77-f5ptc\" (UID: \"94eff779-7543-4a52-af77-debeeb23f1f9\") " pod="opendatahub/opendatahub-operator-controller-manager-6496967d77-f5ptc" Jul 13 14:51:09.714328 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.714261 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/94eff779-7543-4a52-af77-debeeb23f1f9-webhook-cert\") pod \"opendatahub-operator-controller-manager-6496967d77-f5ptc\" (UID: \"94eff779-7543-4a52-af77-debeeb23f1f9\") " pod="opendatahub/opendatahub-operator-controller-manager-6496967d77-f5ptc" Jul 13 14:51:09.714328 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.714300 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/94eff779-7543-4a52-af77-debeeb23f1f9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6496967d77-f5ptc\" (UID: \"94eff779-7543-4a52-af77-debeeb23f1f9\") " pod="opendatahub/opendatahub-operator-controller-manager-6496967d77-f5ptc" Jul 13 14:51:09.714406 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.714334 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbrrz\" (UniqueName: \"kubernetes.io/projected/94eff779-7543-4a52-af77-debeeb23f1f9-kube-api-access-kbrrz\") pod \"opendatahub-operator-controller-manager-6496967d77-f5ptc\" (UID: \"94eff779-7543-4a52-af77-debeeb23f1f9\") " pod="opendatahub/opendatahub-operator-controller-manager-6496967d77-f5ptc" Jul 13 14:51:09.714406 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.714366 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/94eff779-7543-4a52-af77-debeeb23f1f9-manifests\") pod \"opendatahub-operator-controller-manager-6496967d77-f5ptc\" (UID: \"94eff779-7543-4a52-af77-debeeb23f1f9\") " pod="opendatahub/opendatahub-operator-controller-manager-6496967d77-f5ptc" Jul 13 14:51:09.714773 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.714748 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/94eff779-7543-4a52-af77-debeeb23f1f9-manifests\") pod \"opendatahub-operator-controller-manager-6496967d77-f5ptc\" (UID: \"94eff779-7543-4a52-af77-debeeb23f1f9\") " pod="opendatahub/opendatahub-operator-controller-manager-6496967d77-f5ptc" Jul 13 14:51:09.716659 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.716630 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94eff779-7543-4a52-af77-debeeb23f1f9-tmp\") pod \"opendatahub-operator-controller-manager-6496967d77-f5ptc\" (UID: \"94eff779-7543-4a52-af77-debeeb23f1f9\") " pod="opendatahub/opendatahub-operator-controller-manager-6496967d77-f5ptc" Jul 13 14:51:09.716825 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.716807 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/94eff779-7543-4a52-af77-debeeb23f1f9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6496967d77-f5ptc\" (UID: \"94eff779-7543-4a52-af77-debeeb23f1f9\") " pod="opendatahub/opendatahub-operator-controller-manager-6496967d77-f5ptc" Jul 13 14:51:09.716902 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.716887 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/94eff779-7543-4a52-af77-debeeb23f1f9-webhook-cert\") pod \"opendatahub-operator-controller-manager-6496967d77-f5ptc\" (UID: \"94eff779-7543-4a52-af77-debeeb23f1f9\") " pod="opendatahub/opendatahub-operator-controller-manager-6496967d77-f5ptc" Jul 13 14:51:09.722645 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.722624 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbrrz\" (UniqueName: \"kubernetes.io/projected/94eff779-7543-4a52-af77-debeeb23f1f9-kube-api-access-kbrrz\") pod \"opendatahub-operator-controller-manager-6496967d77-f5ptc\" (UID: \"94eff779-7543-4a52-af77-debeeb23f1f9\") " pod="opendatahub/opendatahub-operator-controller-manager-6496967d77-f5ptc" Jul 13 14:51:09.744707 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.744668 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6496967d77-f5ptc" Jul 13 14:51:09.813762 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.813733 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:51:09.885818 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.885735 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6496967d77-f5ptc"] Jul 13 14:51:09.889322 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:51:09.889293 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94eff779_7543_4a52_af77_debeeb23f1f9.slice/crio-006230a34b9c9f5a752adad7503cdf4f0288370dc925f5f2e743d962dedb57cf WatchSource:0}: Error finding container 006230a34b9c9f5a752adad7503cdf4f0288370dc925f5f2e743d962dedb57cf: Status 404 returned error can't find the container with id 006230a34b9c9f5a752adad7503cdf4f0288370dc925f5f2e743d962dedb57cf Jul 13 14:51:09.891382 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.891357 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:51:09.969063 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:09.969025 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:51:10.070150 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:10.070097 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:51:10.182373 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:10.182278 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:51:10.803478 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:10.803440 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6496967d77-f5ptc" event={"ID":"94eff779-7543-4a52-af77-debeeb23f1f9","Type":"ContainerStarted","Data":"006230a34b9c9f5a752adad7503cdf4f0288370dc925f5f2e743d962dedb57cf"} Jul 13 14:51:13.818709 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:13.818671 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6496967d77-f5ptc" event={"ID":"94eff779-7543-4a52-af77-debeeb23f1f9","Type":"ContainerStarted","Data":"a97f9bebc16821a10a53df87d320b159b7698300f065f9e52a79b35703b2a009"} Jul 13 14:51:14.823660 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:14.823630 2568 generic.go:356] "Generic (PLEG): container finished" podID="94eff779-7543-4a52-af77-debeeb23f1f9" containerID="a97f9bebc16821a10a53df87d320b159b7698300f065f9e52a79b35703b2a009" exitCode=0 Jul 13 14:51:14.824027 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:14.823676 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6496967d77-f5ptc" event={"ID":"94eff779-7543-4a52-af77-debeeb23f1f9","Type":"ContainerDied","Data":"a97f9bebc16821a10a53df87d320b159b7698300f065f9e52a79b35703b2a009"} Jul 13 14:51:16.414857 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:16.414820 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:51:16.522588 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:16.522541 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:51:16.834949 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:16.834906 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6496967d77-f5ptc" event={"ID":"94eff779-7543-4a52-af77-debeeb23f1f9","Type":"ContainerStarted","Data":"6165bf030142f47811dfb7bb9eb07ba403706e99d95993ceb06d4b99c7ef8051"} Jul 13 14:51:16.835153 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:16.835045 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6496967d77-f5ptc" Jul 13 14:51:16.882693 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:16.882634 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6496967d77-f5ptc" podStartSLOduration=2.734995754 podStartE2EDuration="7.88261251s" totalImagesPullingTime="4.116298947s" totalInitContainerRuntime="1.031317809s" isStatefulPod=false podCreationTimestamp="2026-07-13 14:51:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:51:16.882021972 +0000 UTC m=+289.122299822" watchObservedRunningTime="2026-07-13 14:51:16.88261251 +0000 UTC m=+289.122890361" Jul 13 14:51:18.646961 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:18.646928 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:51:18.705338 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:18.705309 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:51:27.841718 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:27.841682 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6496967d77-f5ptc" Jul 13 14:51:28.134547 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:28.134453 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xgqvx_0bd25f5c-138c-4c60-9d20-525c8b12ebb1/ovn-acl-logging/0.log" Jul 13 14:51:28.137767 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:28.137736 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xgqvx_0bd25f5c-138c-4c60-9d20-525c8b12ebb1/ovn-acl-logging/0.log" Jul 13 14:51:28.141729 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:28.141711 2568 kubelet.go:1648] "Image garbage collection succeeded" Jul 13 14:51:30.446845 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:30.446811 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:51:30.930414 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:30.930373 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-84c9577b7-772ld"] Jul 13 14:51:30.933500 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:30.933482 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-84c9577b7-772ld" Jul 13 14:51:30.937514 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:30.937492 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 13 14:51:30.938014 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:30.937993 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 13 14:51:30.938103 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:30.938070 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 13 14:51:30.938103 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:30.938083 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 13 14:51:30.938223 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:30.938105 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-65clf\"" Jul 13 14:51:30.948096 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:30.948069 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-84c9577b7-772ld"] Jul 13 14:51:30.986015 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:30.985983 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4s7n2\" (UniqueName: \"kubernetes.io/projected/d6122581-51d2-48cc-b2f3-8ee6223e7bd7-kube-api-access-4s7n2\") pod \"kube-auth-proxy-84c9577b7-772ld\" (UID: \"d6122581-51d2-48cc-b2f3-8ee6223e7bd7\") " pod="openshift-ingress/kube-auth-proxy-84c9577b7-772ld" Jul 13 14:51:30.986197 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:30.986029 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6122581-51d2-48cc-b2f3-8ee6223e7bd7-tls-certs\") pod \"kube-auth-proxy-84c9577b7-772ld\" (UID: \"d6122581-51d2-48cc-b2f3-8ee6223e7bd7\") " pod="openshift-ingress/kube-auth-proxy-84c9577b7-772ld" Jul 13 14:51:30.986197 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:30.986046 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d6122581-51d2-48cc-b2f3-8ee6223e7bd7-tmp\") pod \"kube-auth-proxy-84c9577b7-772ld\" (UID: \"d6122581-51d2-48cc-b2f3-8ee6223e7bd7\") " pod="openshift-ingress/kube-auth-proxy-84c9577b7-772ld" Jul 13 14:51:31.086442 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:31.086406 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4s7n2\" (UniqueName: \"kubernetes.io/projected/d6122581-51d2-48cc-b2f3-8ee6223e7bd7-kube-api-access-4s7n2\") pod \"kube-auth-proxy-84c9577b7-772ld\" (UID: \"d6122581-51d2-48cc-b2f3-8ee6223e7bd7\") " pod="openshift-ingress/kube-auth-proxy-84c9577b7-772ld" Jul 13 14:51:31.086639 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:31.086456 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6122581-51d2-48cc-b2f3-8ee6223e7bd7-tls-certs\") pod \"kube-auth-proxy-84c9577b7-772ld\" (UID: \"d6122581-51d2-48cc-b2f3-8ee6223e7bd7\") " pod="openshift-ingress/kube-auth-proxy-84c9577b7-772ld" Jul 13 14:51:31.086639 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:31.086566 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d6122581-51d2-48cc-b2f3-8ee6223e7bd7-tmp\") pod \"kube-auth-proxy-84c9577b7-772ld\" (UID: \"d6122581-51d2-48cc-b2f3-8ee6223e7bd7\") " pod="openshift-ingress/kube-auth-proxy-84c9577b7-772ld" Jul 13 14:51:31.088617 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:31.088596 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d6122581-51d2-48cc-b2f3-8ee6223e7bd7-tmp\") pod \"kube-auth-proxy-84c9577b7-772ld\" (UID: \"d6122581-51d2-48cc-b2f3-8ee6223e7bd7\") " pod="openshift-ingress/kube-auth-proxy-84c9577b7-772ld" Jul 13 14:51:31.088934 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:31.088914 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6122581-51d2-48cc-b2f3-8ee6223e7bd7-tls-certs\") pod \"kube-auth-proxy-84c9577b7-772ld\" (UID: \"d6122581-51d2-48cc-b2f3-8ee6223e7bd7\") " pod="openshift-ingress/kube-auth-proxy-84c9577b7-772ld" Jul 13 14:51:31.107921 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:31.107892 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4s7n2\" (UniqueName: \"kubernetes.io/projected/d6122581-51d2-48cc-b2f3-8ee6223e7bd7-kube-api-access-4s7n2\") pod \"kube-auth-proxy-84c9577b7-772ld\" (UID: \"d6122581-51d2-48cc-b2f3-8ee6223e7bd7\") " pod="openshift-ingress/kube-auth-proxy-84c9577b7-772ld" Jul 13 14:51:31.242786 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:31.242694 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-84c9577b7-772ld" Jul 13 14:51:31.383626 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:31.383600 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-84c9577b7-772ld"] Jul 13 14:51:31.385650 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:51:31.385620 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6122581_51d2_48cc_b2f3_8ee6223e7bd7.slice/crio-dafe979df282600c6cd6677715519396b6982f37573f00c5d01e3abb489eb50b WatchSource:0}: Error finding container dafe979df282600c6cd6677715519396b6982f37573f00c5d01e3abb489eb50b: Status 404 returned error can't find the container with id dafe979df282600c6cd6677715519396b6982f37573f00c5d01e3abb489eb50b Jul 13 14:51:31.387523 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:31.387506 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 14:51:31.897635 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:31.897599 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-84c9577b7-772ld" event={"ID":"d6122581-51d2-48cc-b2f3-8ee6223e7bd7","Type":"ContainerStarted","Data":"dafe979df282600c6cd6677715519396b6982f37573f00c5d01e3abb489eb50b"} Jul 13 14:51:32.402994 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:32.402955 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4n846"] Jul 13 14:51:32.406115 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:32.406092 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-4n846" Jul 13 14:51:32.427742 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:32.427707 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4n846"] Jul 13 14:51:32.434143 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:32.434114 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-wjh4r\"" Jul 13 14:51:32.434297 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:32.434161 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 13 14:51:32.497756 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:32.497714 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b559bca0-d4cc-4a33-b63a-1c03a6c54a84-tls-certs\") pod \"model-serving-api-54d89bd5fc-4n846\" (UID: \"b559bca0-d4cc-4a33-b63a-1c03a6c54a84\") " pod="opendatahub/model-serving-api-54d89bd5fc-4n846" Jul 13 14:51:32.497943 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:32.497770 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69whf\" (UniqueName: \"kubernetes.io/projected/b559bca0-d4cc-4a33-b63a-1c03a6c54a84-kube-api-access-69whf\") pod \"model-serving-api-54d89bd5fc-4n846\" (UID: \"b559bca0-d4cc-4a33-b63a-1c03a6c54a84\") " pod="opendatahub/model-serving-api-54d89bd5fc-4n846" Jul 13 14:51:32.598667 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:32.598630 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b559bca0-d4cc-4a33-b63a-1c03a6c54a84-tls-certs\") pod \"model-serving-api-54d89bd5fc-4n846\" (UID: \"b559bca0-d4cc-4a33-b63a-1c03a6c54a84\") " pod="opendatahub/model-serving-api-54d89bd5fc-4n846" Jul 13 14:51:32.598837 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:32.598687 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69whf\" (UniqueName: \"kubernetes.io/projected/b559bca0-d4cc-4a33-b63a-1c03a6c54a84-kube-api-access-69whf\") pod \"model-serving-api-54d89bd5fc-4n846\" (UID: \"b559bca0-d4cc-4a33-b63a-1c03a6c54a84\") " pod="opendatahub/model-serving-api-54d89bd5fc-4n846" Jul 13 14:51:32.598837 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:51:32.598777 2568 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 13 14:51:32.598912 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:51:32.598843 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b559bca0-d4cc-4a33-b63a-1c03a6c54a84-tls-certs podName:b559bca0-d4cc-4a33-b63a-1c03a6c54a84 nodeName:}" failed. No retries permitted until 2026-07-13 14:51:33.098825715 +0000 UTC m=+305.339103541 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/b559bca0-d4cc-4a33-b63a-1c03a6c54a84-tls-certs") pod "model-serving-api-54d89bd5fc-4n846" (UID: "b559bca0-d4cc-4a33-b63a-1c03a6c54a84") : secret "model-serving-api-tls" not found Jul 13 14:51:32.608521 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:32.608500 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69whf\" (UniqueName: \"kubernetes.io/projected/b559bca0-d4cc-4a33-b63a-1c03a6c54a84-kube-api-access-69whf\") pod \"model-serving-api-54d89bd5fc-4n846\" (UID: \"b559bca0-d4cc-4a33-b63a-1c03a6c54a84\") " pod="opendatahub/model-serving-api-54d89bd5fc-4n846" Jul 13 14:51:33.103009 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:33.102960 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b559bca0-d4cc-4a33-b63a-1c03a6c54a84-tls-certs\") pod \"model-serving-api-54d89bd5fc-4n846\" (UID: \"b559bca0-d4cc-4a33-b63a-1c03a6c54a84\") " pod="opendatahub/model-serving-api-54d89bd5fc-4n846" Jul 13 14:51:33.105424 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:33.105402 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b559bca0-d4cc-4a33-b63a-1c03a6c54a84-tls-certs\") pod \"model-serving-api-54d89bd5fc-4n846\" (UID: \"b559bca0-d4cc-4a33-b63a-1c03a6c54a84\") " pod="opendatahub/model-serving-api-54d89bd5fc-4n846" Jul 13 14:51:33.318705 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:33.318668 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-4n846" Jul 13 14:51:33.469198 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:33.469098 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4n846"] Jul 13 14:51:33.471950 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:51:33.471920 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb559bca0_d4cc_4a33_b63a_1c03a6c54a84.slice/crio-a9c1249c993444359219d021cd84bdeee93a1e42b07b5a3fffdcceadc8bb4d04 WatchSource:0}: Error finding container a9c1249c993444359219d021cd84bdeee93a1e42b07b5a3fffdcceadc8bb4d04: Status 404 returned error can't find the container with id a9c1249c993444359219d021cd84bdeee93a1e42b07b5a3fffdcceadc8bb4d04 Jul 13 14:51:33.908262 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:33.908222 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-4n846" event={"ID":"b559bca0-d4cc-4a33-b63a-1c03a6c54a84","Type":"ContainerStarted","Data":"a9c1249c993444359219d021cd84bdeee93a1e42b07b5a3fffdcceadc8bb4d04"} Jul 13 14:51:34.857377 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:34.857338 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:51:34.914659 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:34.914622 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-84c9577b7-772ld" event={"ID":"d6122581-51d2-48cc-b2f3-8ee6223e7bd7","Type":"ContainerStarted","Data":"d48d99cf3fb9ad94b9f2b8a31ace73077684a0c6e17429a34e313c88852e508d"} Jul 13 14:51:34.933291 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:34.933240 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-84c9577b7-772ld" podStartSLOduration=2.022401251 podStartE2EDuration="4.933223601s" totalImagesPullingTime="2.91082235s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 14:51:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:51:34.93179652 +0000 UTC m=+307.172074393" watchObservedRunningTime="2026-07-13 14:51:34.933223601 +0000 UTC m=+307.173501450" Jul 13 14:51:36.925038 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:36.924985 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-4n846" event={"ID":"b559bca0-d4cc-4a33-b63a-1c03a6c54a84","Type":"ContainerStarted","Data":"63e381297cb844944684734f2652e6183456d443aaccba3c5a9ac147e0b7940f"} Jul 13 14:51:36.925472 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:36.925167 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-4n846" Jul 13 14:51:36.942956 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:36.942906 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-4n846" podStartSLOduration=2.129130306 podStartE2EDuration="4.942890981s" totalImagesPullingTime="2.813760675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 14:51:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:51:36.941902744 +0000 UTC m=+309.182180617" watchObservedRunningTime="2026-07-13 14:51:36.942890981 +0000 UTC m=+309.183168829" Jul 13 14:51:38.767121 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:38.767084 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:51:39.107932 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.107898 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t"] Jul 13 14:51:39.111454 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.111434 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" Jul 13 14:51:39.114139 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.114116 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 13 14:51:39.114323 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.114116 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-7hc98\"" Jul 13 14:51:39.125541 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.125513 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t"] Jul 13 14:51:39.152727 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.152689 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dfe27dc5-31d2-4bda-bddd-a1995f68e78b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zbt4t\" (UID: \"dfe27dc5-31d2-4bda-bddd-a1995f68e78b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" Jul 13 14:51:39.152889 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.152736 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dfe27dc5-31d2-4bda-bddd-a1995f68e78b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zbt4t\" (UID: \"dfe27dc5-31d2-4bda-bddd-a1995f68e78b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" Jul 13 14:51:39.152889 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.152759 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dfe27dc5-31d2-4bda-bddd-a1995f68e78b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zbt4t\" (UID: \"dfe27dc5-31d2-4bda-bddd-a1995f68e78b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" Jul 13 14:51:39.152889 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.152783 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dfe27dc5-31d2-4bda-bddd-a1995f68e78b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zbt4t\" (UID: \"dfe27dc5-31d2-4bda-bddd-a1995f68e78b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" Jul 13 14:51:39.152889 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.152809 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dfe27dc5-31d2-4bda-bddd-a1995f68e78b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zbt4t\" (UID: \"dfe27dc5-31d2-4bda-bddd-a1995f68e78b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" Jul 13 14:51:39.152889 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.152826 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfe27dc5-31d2-4bda-bddd-a1995f68e78b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zbt4t\" (UID: \"dfe27dc5-31d2-4bda-bddd-a1995f68e78b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" Jul 13 14:51:39.152889 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.152854 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh4zp\" (UniqueName: \"kubernetes.io/projected/dfe27dc5-31d2-4bda-bddd-a1995f68e78b-kube-api-access-xh4zp\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zbt4t\" (UID: \"dfe27dc5-31d2-4bda-bddd-a1995f68e78b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" Jul 13 14:51:39.153143 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.152907 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfe27dc5-31d2-4bda-bddd-a1995f68e78b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zbt4t\" (UID: \"dfe27dc5-31d2-4bda-bddd-a1995f68e78b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" Jul 13 14:51:39.153143 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.152963 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfe27dc5-31d2-4bda-bddd-a1995f68e78b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zbt4t\" (UID: \"dfe27dc5-31d2-4bda-bddd-a1995f68e78b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" Jul 13 14:51:39.254057 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.254021 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfe27dc5-31d2-4bda-bddd-a1995f68e78b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zbt4t\" (UID: \"dfe27dc5-31d2-4bda-bddd-a1995f68e78b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" Jul 13 14:51:39.254057 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.254061 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dfe27dc5-31d2-4bda-bddd-a1995f68e78b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zbt4t\" (UID: \"dfe27dc5-31d2-4bda-bddd-a1995f68e78b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" Jul 13 14:51:39.254289 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.254088 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dfe27dc5-31d2-4bda-bddd-a1995f68e78b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zbt4t\" (UID: \"dfe27dc5-31d2-4bda-bddd-a1995f68e78b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" Jul 13 14:51:39.254289 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.254114 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dfe27dc5-31d2-4bda-bddd-a1995f68e78b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zbt4t\" (UID: \"dfe27dc5-31d2-4bda-bddd-a1995f68e78b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" Jul 13 14:51:39.254289 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.254142 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dfe27dc5-31d2-4bda-bddd-a1995f68e78b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zbt4t\" (UID: \"dfe27dc5-31d2-4bda-bddd-a1995f68e78b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" Jul 13 14:51:39.254289 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.254178 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dfe27dc5-31d2-4bda-bddd-a1995f68e78b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zbt4t\" (UID: \"dfe27dc5-31d2-4bda-bddd-a1995f68e78b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" Jul 13 14:51:39.254289 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.254204 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfe27dc5-31d2-4bda-bddd-a1995f68e78b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zbt4t\" (UID: \"dfe27dc5-31d2-4bda-bddd-a1995f68e78b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" Jul 13 14:51:39.254289 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.254225 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh4zp\" (UniqueName: \"kubernetes.io/projected/dfe27dc5-31d2-4bda-bddd-a1995f68e78b-kube-api-access-xh4zp\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zbt4t\" (UID: \"dfe27dc5-31d2-4bda-bddd-a1995f68e78b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" Jul 13 14:51:39.254289 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.254265 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfe27dc5-31d2-4bda-bddd-a1995f68e78b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zbt4t\" (UID: \"dfe27dc5-31d2-4bda-bddd-a1995f68e78b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" Jul 13 14:51:39.254652 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.254499 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfe27dc5-31d2-4bda-bddd-a1995f68e78b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zbt4t\" (UID: \"dfe27dc5-31d2-4bda-bddd-a1995f68e78b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" Jul 13 14:51:39.254652 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.254546 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dfe27dc5-31d2-4bda-bddd-a1995f68e78b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zbt4t\" (UID: \"dfe27dc5-31d2-4bda-bddd-a1995f68e78b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" Jul 13 14:51:39.254652 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.254623 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfe27dc5-31d2-4bda-bddd-a1995f68e78b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zbt4t\" (UID: \"dfe27dc5-31d2-4bda-bddd-a1995f68e78b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" Jul 13 14:51:39.254803 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.254699 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfe27dc5-31d2-4bda-bddd-a1995f68e78b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zbt4t\" (UID: \"dfe27dc5-31d2-4bda-bddd-a1995f68e78b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" Jul 13 14:51:39.254951 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.254933 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dfe27dc5-31d2-4bda-bddd-a1995f68e78b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zbt4t\" (UID: \"dfe27dc5-31d2-4bda-bddd-a1995f68e78b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" Jul 13 14:51:39.256490 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.256470 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dfe27dc5-31d2-4bda-bddd-a1995f68e78b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zbt4t\" (UID: \"dfe27dc5-31d2-4bda-bddd-a1995f68e78b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" Jul 13 14:51:39.256738 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.256720 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dfe27dc5-31d2-4bda-bddd-a1995f68e78b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zbt4t\" (UID: \"dfe27dc5-31d2-4bda-bddd-a1995f68e78b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" Jul 13 14:51:39.264121 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.264090 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dfe27dc5-31d2-4bda-bddd-a1995f68e78b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zbt4t\" (UID: \"dfe27dc5-31d2-4bda-bddd-a1995f68e78b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" Jul 13 14:51:39.264347 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.264327 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh4zp\" (UniqueName: \"kubernetes.io/projected/dfe27dc5-31d2-4bda-bddd-a1995f68e78b-kube-api-access-xh4zp\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zbt4t\" (UID: \"dfe27dc5-31d2-4bda-bddd-a1995f68e78b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" Jul 13 14:51:39.421387 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.421294 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" Jul 13 14:51:39.429664 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.429637 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:51:39.548354 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.548323 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:51:39.564281 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.564256 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t"] Jul 13 14:51:39.566477 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:51:39.566453 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfe27dc5_31d2_4bda_bddd_a1995f68e78b.slice/crio-f9e67c054896217c00d35e02d4c9e115c10d9b9aee2c4e1942a3087eaa366ca5 WatchSource:0}: Error finding container f9e67c054896217c00d35e02d4c9e115c10d9b9aee2c4e1942a3087eaa366ca5: Status 404 returned error can't find the container with id f9e67c054896217c00d35e02d4c9e115c10d9b9aee2c4e1942a3087eaa366ca5 Jul 13 14:51:39.940150 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:39.940113 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" event={"ID":"dfe27dc5-31d2-4bda-bddd-a1995f68e78b","Type":"ContainerStarted","Data":"f9e67c054896217c00d35e02d4c9e115c10d9b9aee2c4e1942a3087eaa366ca5"} Jul 13 14:51:41.247460 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:41.247423 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-d4d649b46-9stcp"] Jul 13 14:51:41.253514 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:41.253487 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d4d649b46-9stcp" Jul 13 14:51:41.256201 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:41.256177 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 13 14:51:41.257389 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:41.257363 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-gz7nt\"" Jul 13 14:51:41.257514 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:41.257421 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 13 14:51:41.257514 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:41.257449 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 13 14:51:41.264968 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:41.264944 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d4d649b46-9stcp"] Jul 13 14:51:41.374453 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:41.374411 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2e347796-e195-4186-ac37-c22c554570c0-metrics-cert\") pod \"lws-controller-manager-d4d649b46-9stcp\" (UID: \"2e347796-e195-4186-ac37-c22c554570c0\") " pod="openshift-lws-operator/lws-controller-manager-d4d649b46-9stcp" Jul 13 14:51:41.374676 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:41.374540 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2e347796-e195-4186-ac37-c22c554570c0-manager-config\") pod \"lws-controller-manager-d4d649b46-9stcp\" (UID: \"2e347796-e195-4186-ac37-c22c554570c0\") " pod="openshift-lws-operator/lws-controller-manager-d4d649b46-9stcp" Jul 13 14:51:41.374676 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:41.374622 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2e347796-e195-4186-ac37-c22c554570c0-cert\") pod \"lws-controller-manager-d4d649b46-9stcp\" (UID: \"2e347796-e195-4186-ac37-c22c554570c0\") " pod="openshift-lws-operator/lws-controller-manager-d4d649b46-9stcp" Jul 13 14:51:41.374676 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:41.374652 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2d99\" (UniqueName: \"kubernetes.io/projected/2e347796-e195-4186-ac37-c22c554570c0-kube-api-access-c2d99\") pod \"lws-controller-manager-d4d649b46-9stcp\" (UID: \"2e347796-e195-4186-ac37-c22c554570c0\") " pod="openshift-lws-operator/lws-controller-manager-d4d649b46-9stcp" Jul 13 14:51:41.475651 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:41.475611 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2e347796-e195-4186-ac37-c22c554570c0-manager-config\") pod \"lws-controller-manager-d4d649b46-9stcp\" (UID: \"2e347796-e195-4186-ac37-c22c554570c0\") " pod="openshift-lws-operator/lws-controller-manager-d4d649b46-9stcp" Jul 13 14:51:41.475865 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:41.475686 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2e347796-e195-4186-ac37-c22c554570c0-cert\") pod \"lws-controller-manager-d4d649b46-9stcp\" (UID: \"2e347796-e195-4186-ac37-c22c554570c0\") " pod="openshift-lws-operator/lws-controller-manager-d4d649b46-9stcp" Jul 13 14:51:41.475865 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:41.475715 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2d99\" (UniqueName: \"kubernetes.io/projected/2e347796-e195-4186-ac37-c22c554570c0-kube-api-access-c2d99\") pod \"lws-controller-manager-d4d649b46-9stcp\" (UID: \"2e347796-e195-4186-ac37-c22c554570c0\") " pod="openshift-lws-operator/lws-controller-manager-d4d649b46-9stcp" Jul 13 14:51:41.475865 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:41.475757 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2e347796-e195-4186-ac37-c22c554570c0-metrics-cert\") pod \"lws-controller-manager-d4d649b46-9stcp\" (UID: \"2e347796-e195-4186-ac37-c22c554570c0\") " pod="openshift-lws-operator/lws-controller-manager-d4d649b46-9stcp" Jul 13 14:51:41.476476 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:41.476419 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2e347796-e195-4186-ac37-c22c554570c0-manager-config\") pod \"lws-controller-manager-d4d649b46-9stcp\" (UID: \"2e347796-e195-4186-ac37-c22c554570c0\") " pod="openshift-lws-operator/lws-controller-manager-d4d649b46-9stcp" Jul 13 14:51:41.478583 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:41.478552 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2e347796-e195-4186-ac37-c22c554570c0-cert\") pod \"lws-controller-manager-d4d649b46-9stcp\" (UID: \"2e347796-e195-4186-ac37-c22c554570c0\") " pod="openshift-lws-operator/lws-controller-manager-d4d649b46-9stcp" Jul 13 14:51:41.478713 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:41.478693 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2e347796-e195-4186-ac37-c22c554570c0-metrics-cert\") pod \"lws-controller-manager-d4d649b46-9stcp\" (UID: \"2e347796-e195-4186-ac37-c22c554570c0\") " pod="openshift-lws-operator/lws-controller-manager-d4d649b46-9stcp" Jul 13 14:51:41.487831 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:41.487809 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2d99\" (UniqueName: \"kubernetes.io/projected/2e347796-e195-4186-ac37-c22c554570c0-kube-api-access-c2d99\") pod \"lws-controller-manager-d4d649b46-9stcp\" (UID: \"2e347796-e195-4186-ac37-c22c554570c0\") " pod="openshift-lws-operator/lws-controller-manager-d4d649b46-9stcp" Jul 13 14:51:41.565545 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:41.565504 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d4d649b46-9stcp" Jul 13 14:51:42.216448 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:42.216406 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 14:51:42.216550 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:42.216497 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 14:51:42.216644 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:42.216546 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 14:51:42.298545 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:51:42.298517 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e347796_e195_4186_ac37_c22c554570c0.slice/crio-e46ca00c4058893e38df207669e484c77068fd74a24334e9470e88d1f8f3a2f6 WatchSource:0}: Error finding container e46ca00c4058893e38df207669e484c77068fd74a24334e9470e88d1f8f3a2f6: Status 404 returned error can't find the container with id e46ca00c4058893e38df207669e484c77068fd74a24334e9470e88d1f8f3a2f6 Jul 13 14:51:42.302045 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:42.302021 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d4d649b46-9stcp"] Jul 13 14:51:42.620810 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:42.620778 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:51:42.715186 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:42.715140 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:51:42.892643 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:42.892527 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:51:42.958741 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:42.958703 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d4d649b46-9stcp" event={"ID":"2e347796-e195-4186-ac37-c22c554570c0","Type":"ContainerStarted","Data":"e46ca00c4058893e38df207669e484c77068fd74a24334e9470e88d1f8f3a2f6"} Jul 13 14:51:42.960506 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:42.960478 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" event={"ID":"dfe27dc5-31d2-4bda-bddd-a1995f68e78b","Type":"ContainerStarted","Data":"00961038d1831741e7b484143bc9bc7dee7f0041f77201a2168f8cec9f91a7bd"} Jul 13 14:51:43.010658 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:43.010600 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" podStartSLOduration=1.362939897 podStartE2EDuration="4.010556715s" totalImagesPullingTime="2.647616818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 14:51:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:51:43.008931867 +0000 UTC m=+315.249209743" watchObservedRunningTime="2026-07-13 14:51:43.010556715 +0000 UTC m=+315.250834564" Jul 13 14:51:43.422268 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:43.422228 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" Jul 13 14:51:43.427682 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:43.427656 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" Jul 13 14:51:43.908086 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:43.908047 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:51:43.967241 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:43.967201 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d4d649b46-9stcp" event={"ID":"2e347796-e195-4186-ac37-c22c554570c0","Type":"ContainerStarted","Data":"b81a55d706b634762e8c0e37945933d5338dfffc280cc7736b4097d7dcf28332"} Jul 13 14:51:43.967438 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:43.967351 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-d4d649b46-9stcp" Jul 13 14:51:43.967438 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:43.967389 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" Jul 13 14:51:43.968238 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:43.968221 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zbt4t" Jul 13 14:51:44.003438 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:44.003405 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:51:44.032297 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:44.032230 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-d4d649b46-9stcp" podStartSLOduration=1.7846462029999999 podStartE2EDuration="3.032207596s" totalImagesPullingTime="1.247561393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 14:51:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:51:43.99542883 +0000 UTC m=+316.235706678" watchObservedRunningTime="2026-07-13 14:51:44.032207596 +0000 UTC m=+316.272485450" Jul 13 14:51:47.933903 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:47.933873 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-4n846" Jul 13 14:51:54.976346 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:51:54.976314 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-d4d649b46-9stcp" Jul 13 14:52:19.998730 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:19.998640 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:52:20.106229 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:20.106192 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:52:32.070285 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:32.070246 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:52:32.169437 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:32.169401 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:52:32.269983 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:32.269951 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:52:47.404680 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:47.404628 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk"] Jul 13 14:52:47.408066 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:47.408049 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk" Jul 13 14:52:47.410481 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:47.410454 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 13 14:52:47.410643 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:47.410492 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-v8c7p\"" Jul 13 14:52:47.411331 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:47.411315 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 13 14:52:47.416688 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:47.416659 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk"] Jul 13 14:52:47.494255 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:47.494225 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7360a6d7-1bce-4ccd-a3e3-111b9bcf051b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk\" (UID: \"7360a6d7-1bce-4ccd-a3e3-111b9bcf051b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk" Jul 13 14:52:47.494416 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:47.494270 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvm86\" (UniqueName: \"kubernetes.io/projected/7360a6d7-1bce-4ccd-a3e3-111b9bcf051b-kube-api-access-vvm86\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk\" (UID: \"7360a6d7-1bce-4ccd-a3e3-111b9bcf051b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk" Jul 13 14:52:47.494416 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:47.494333 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7360a6d7-1bce-4ccd-a3e3-111b9bcf051b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk\" (UID: \"7360a6d7-1bce-4ccd-a3e3-111b9bcf051b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk" Jul 13 14:52:47.595166 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:47.595122 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvm86\" (UniqueName: \"kubernetes.io/projected/7360a6d7-1bce-4ccd-a3e3-111b9bcf051b-kube-api-access-vvm86\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk\" (UID: \"7360a6d7-1bce-4ccd-a3e3-111b9bcf051b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk" Jul 13 14:52:47.595341 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:47.595176 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7360a6d7-1bce-4ccd-a3e3-111b9bcf051b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk\" (UID: \"7360a6d7-1bce-4ccd-a3e3-111b9bcf051b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk" Jul 13 14:52:47.595341 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:47.595236 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7360a6d7-1bce-4ccd-a3e3-111b9bcf051b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk\" (UID: \"7360a6d7-1bce-4ccd-a3e3-111b9bcf051b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk" Jul 13 14:52:47.595626 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:47.595563 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7360a6d7-1bce-4ccd-a3e3-111b9bcf051b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk\" (UID: \"7360a6d7-1bce-4ccd-a3e3-111b9bcf051b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk" Jul 13 14:52:47.595740 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:47.595663 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7360a6d7-1bce-4ccd-a3e3-111b9bcf051b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk\" (UID: \"7360a6d7-1bce-4ccd-a3e3-111b9bcf051b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk" Jul 13 14:52:47.604889 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:47.604865 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvm86\" (UniqueName: \"kubernetes.io/projected/7360a6d7-1bce-4ccd-a3e3-111b9bcf051b-kube-api-access-vvm86\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk\" (UID: \"7360a6d7-1bce-4ccd-a3e3-111b9bcf051b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk" Jul 13 14:52:47.717900 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:47.717817 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk" Jul 13 14:52:47.853483 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:47.853457 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk"] Jul 13 14:52:47.855924 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:52:47.855896 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7360a6d7_1bce_4ccd_a3e3_111b9bcf051b.slice/crio-0042c2fc90eb3c051ad10fe5643867d4d7d6eea8e3d287800895b87c97e0979b WatchSource:0}: Error finding container 0042c2fc90eb3c051ad10fe5643867d4d7d6eea8e3d287800895b87c97e0979b: Status 404 returned error can't find the container with id 0042c2fc90eb3c051ad10fe5643867d4d7d6eea8e3d287800895b87c97e0979b Jul 13 14:52:48.205615 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:48.205567 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw"] Jul 13 14:52:48.208971 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:48.208947 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw" Jul 13 14:52:48.218168 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:48.218146 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw"] Jul 13 14:52:48.257717 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:48.257687 2568 generic.go:356] "Generic (PLEG): container finished" podID="7360a6d7-1bce-4ccd-a3e3-111b9bcf051b" containerID="7bc2c8df6b1e4f3f86677e0fdc192431ffce22893fc89c68b243346b08c79a0f" exitCode=0 Jul 13 14:52:48.257901 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:48.257756 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk" event={"ID":"7360a6d7-1bce-4ccd-a3e3-111b9bcf051b","Type":"ContainerDied","Data":"7bc2c8df6b1e4f3f86677e0fdc192431ffce22893fc89c68b243346b08c79a0f"} Jul 13 14:52:48.257901 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:48.257801 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk" event={"ID":"7360a6d7-1bce-4ccd-a3e3-111b9bcf051b","Type":"ContainerStarted","Data":"0042c2fc90eb3c051ad10fe5643867d4d7d6eea8e3d287800895b87c97e0979b"} Jul 13 14:52:48.300151 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:48.300120 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7f81017-83f8-4937-b033-67823029c79b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw\" (UID: \"b7f81017-83f8-4937-b033-67823029c79b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw" Jul 13 14:52:48.300300 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:48.300155 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7f81017-83f8-4937-b033-67823029c79b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw\" (UID: \"b7f81017-83f8-4937-b033-67823029c79b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw" Jul 13 14:52:48.300300 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:48.300274 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9lr4\" (UniqueName: \"kubernetes.io/projected/b7f81017-83f8-4937-b033-67823029c79b-kube-api-access-d9lr4\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw\" (UID: \"b7f81017-83f8-4937-b033-67823029c79b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw" Jul 13 14:52:48.400936 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:48.400899 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9lr4\" (UniqueName: \"kubernetes.io/projected/b7f81017-83f8-4937-b033-67823029c79b-kube-api-access-d9lr4\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw\" (UID: \"b7f81017-83f8-4937-b033-67823029c79b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw" Jul 13 14:52:48.401150 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:48.400997 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7f81017-83f8-4937-b033-67823029c79b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw\" (UID: \"b7f81017-83f8-4937-b033-67823029c79b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw" Jul 13 14:52:48.401150 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:48.401021 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7f81017-83f8-4937-b033-67823029c79b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw\" (UID: \"b7f81017-83f8-4937-b033-67823029c79b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw" Jul 13 14:52:48.401432 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:48.401410 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7f81017-83f8-4937-b033-67823029c79b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw\" (UID: \"b7f81017-83f8-4937-b033-67823029c79b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw" Jul 13 14:52:48.401509 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:48.401440 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7f81017-83f8-4937-b033-67823029c79b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw\" (UID: \"b7f81017-83f8-4937-b033-67823029c79b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw" Jul 13 14:52:48.410063 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:48.410036 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9lr4\" (UniqueName: \"kubernetes.io/projected/b7f81017-83f8-4937-b033-67823029c79b-kube-api-access-d9lr4\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw\" (UID: \"b7f81017-83f8-4937-b033-67823029c79b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw" Jul 13 14:52:48.519430 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:48.519342 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw" Jul 13 14:52:48.604531 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:48.604498 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk"] Jul 13 14:52:48.611195 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:48.611173 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk" Jul 13 14:52:48.616445 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:48.616421 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk"] Jul 13 14:52:48.654391 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:48.654361 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw"] Jul 13 14:52:48.655673 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:52:48.655642 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7f81017_83f8_4937_b033_67823029c79b.slice/crio-ec996b3592441d3e6aa16da8b61ace1f1198f11531ca83075e27ca723723cd01 WatchSource:0}: Error finding container ec996b3592441d3e6aa16da8b61ace1f1198f11531ca83075e27ca723723cd01: Status 404 returned error can't find the container with id ec996b3592441d3e6aa16da8b61ace1f1198f11531ca83075e27ca723723cd01 Jul 13 14:52:48.702835 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:48.702800 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6pfp\" (UniqueName: \"kubernetes.io/projected/9a82918c-2764-47f5-aa3d-85096e2524b1-kube-api-access-w6pfp\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk\" (UID: \"9a82918c-2764-47f5-aa3d-85096e2524b1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk" Jul 13 14:52:48.703000 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:48.702894 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a82918c-2764-47f5-aa3d-85096e2524b1-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk\" (UID: \"9a82918c-2764-47f5-aa3d-85096e2524b1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk" Jul 13 14:52:48.703000 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:48.702946 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a82918c-2764-47f5-aa3d-85096e2524b1-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk\" (UID: \"9a82918c-2764-47f5-aa3d-85096e2524b1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk" Jul 13 14:52:48.706938 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:48.706914 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:52:48.803477 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:48.803443 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a82918c-2764-47f5-aa3d-85096e2524b1-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk\" (UID: \"9a82918c-2764-47f5-aa3d-85096e2524b1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk" Jul 13 14:52:48.803692 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:48.803495 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6pfp\" (UniqueName: \"kubernetes.io/projected/9a82918c-2764-47f5-aa3d-85096e2524b1-kube-api-access-w6pfp\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk\" (UID: \"9a82918c-2764-47f5-aa3d-85096e2524b1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk" Jul 13 14:52:48.803692 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:48.803559 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a82918c-2764-47f5-aa3d-85096e2524b1-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk\" (UID: \"9a82918c-2764-47f5-aa3d-85096e2524b1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk" Jul 13 14:52:48.803877 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:48.803858 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a82918c-2764-47f5-aa3d-85096e2524b1-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk\" (UID: \"9a82918c-2764-47f5-aa3d-85096e2524b1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk" Jul 13 14:52:48.803924 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:48.803882 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a82918c-2764-47f5-aa3d-85096e2524b1-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk\" (UID: \"9a82918c-2764-47f5-aa3d-85096e2524b1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk" Jul 13 14:52:48.812967 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:48.812937 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6pfp\" (UniqueName: \"kubernetes.io/projected/9a82918c-2764-47f5-aa3d-85096e2524b1-kube-api-access-w6pfp\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk\" (UID: \"9a82918c-2764-47f5-aa3d-85096e2524b1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk" Jul 13 14:52:48.931294 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:48.931260 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk" Jul 13 14:52:49.005753 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:49.005717 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc"] Jul 13 14:52:49.008806 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:49.008788 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc" Jul 13 14:52:49.018464 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:49.018432 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc"] Jul 13 14:52:49.105806 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:49.105765 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/04986b2b-a511-4c87-907e-45105dd3222a-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc\" (UID: \"04986b2b-a511-4c87-907e-45105dd3222a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc" Jul 13 14:52:49.149666 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:49.149632 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk"] Jul 13 14:52:49.152836 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:52:49.152801 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a82918c_2764_47f5_aa3d_85096e2524b1.slice/crio-19b566ee63320242b880301d3b9173a0d23ac7ace12a059ec5f8f4b7dc120d89 WatchSource:0}: Error finding container 19b566ee63320242b880301d3b9173a0d23ac7ace12a059ec5f8f4b7dc120d89: Status 404 returned error can't find the container with id 19b566ee63320242b880301d3b9173a0d23ac7ace12a059ec5f8f4b7dc120d89 Jul 13 14:52:49.207081 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:49.207051 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/04986b2b-a511-4c87-907e-45105dd3222a-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc\" (UID: \"04986b2b-a511-4c87-907e-45105dd3222a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc" Jul 13 14:52:49.207227 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:49.207109 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9dq4\" (UniqueName: \"kubernetes.io/projected/04986b2b-a511-4c87-907e-45105dd3222a-kube-api-access-q9dq4\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc\" (UID: \"04986b2b-a511-4c87-907e-45105dd3222a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc" Jul 13 14:52:49.207227 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:49.207153 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/04986b2b-a511-4c87-907e-45105dd3222a-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc\" (UID: \"04986b2b-a511-4c87-907e-45105dd3222a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc" Jul 13 14:52:49.207408 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:49.207389 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/04986b2b-a511-4c87-907e-45105dd3222a-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc\" (UID: \"04986b2b-a511-4c87-907e-45105dd3222a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc" Jul 13 14:52:49.263193 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:49.263154 2568 generic.go:356] "Generic (PLEG): container finished" podID="b7f81017-83f8-4937-b033-67823029c79b" containerID="1693f9e273e7371d39c17b7a51bb76916972d07dd831ddafa17e40a51a93ac6f" exitCode=0 Jul 13 14:52:49.263331 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:49.263218 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw" event={"ID":"b7f81017-83f8-4937-b033-67823029c79b","Type":"ContainerDied","Data":"1693f9e273e7371d39c17b7a51bb76916972d07dd831ddafa17e40a51a93ac6f"} Jul 13 14:52:49.263331 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:49.263265 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw" event={"ID":"b7f81017-83f8-4937-b033-67823029c79b","Type":"ContainerStarted","Data":"ec996b3592441d3e6aa16da8b61ace1f1198f11531ca83075e27ca723723cd01"} Jul 13 14:52:49.264994 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:49.264955 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk" event={"ID":"9a82918c-2764-47f5-aa3d-85096e2524b1","Type":"ContainerStarted","Data":"3c3ccc04fef033672d7125e9492a71df8ce79d730e88f5b78fb60a06e2a3f87f"} Jul 13 14:52:49.265228 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:49.265009 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk" event={"ID":"9a82918c-2764-47f5-aa3d-85096e2524b1","Type":"ContainerStarted","Data":"19b566ee63320242b880301d3b9173a0d23ac7ace12a059ec5f8f4b7dc120d89"} Jul 13 14:52:49.267002 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:49.266978 2568 generic.go:356] "Generic (PLEG): container finished" podID="7360a6d7-1bce-4ccd-a3e3-111b9bcf051b" containerID="791bbcd244def1dff114034f2dd4081764520a5a84b785168a100d8368470e51" exitCode=0 Jul 13 14:52:49.267101 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:49.267049 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk" event={"ID":"7360a6d7-1bce-4ccd-a3e3-111b9bcf051b","Type":"ContainerDied","Data":"791bbcd244def1dff114034f2dd4081764520a5a84b785168a100d8368470e51"} Jul 13 14:52:49.307736 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:49.307702 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/04986b2b-a511-4c87-907e-45105dd3222a-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc\" (UID: \"04986b2b-a511-4c87-907e-45105dd3222a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc" Jul 13 14:52:49.307868 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:49.307820 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9dq4\" (UniqueName: \"kubernetes.io/projected/04986b2b-a511-4c87-907e-45105dd3222a-kube-api-access-q9dq4\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc\" (UID: \"04986b2b-a511-4c87-907e-45105dd3222a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc" Jul 13 14:52:49.308079 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:49.308051 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/04986b2b-a511-4c87-907e-45105dd3222a-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc\" (UID: \"04986b2b-a511-4c87-907e-45105dd3222a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc" Jul 13 14:52:49.316853 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:49.316800 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9dq4\" (UniqueName: \"kubernetes.io/projected/04986b2b-a511-4c87-907e-45105dd3222a-kube-api-access-q9dq4\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc\" (UID: \"04986b2b-a511-4c87-907e-45105dd3222a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc" Jul 13 14:52:49.320103 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:49.320083 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc" Jul 13 14:52:49.461025 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:49.461000 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc"] Jul 13 14:52:49.462905 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:52:49.462880 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04986b2b_a511_4c87_907e_45105dd3222a.slice/crio-a21356a86f6965ddd5409ce3343e520c3111af17cc619a15a93b0b8e79aa26b9 WatchSource:0}: Error finding container a21356a86f6965ddd5409ce3343e520c3111af17cc619a15a93b0b8e79aa26b9: Status 404 returned error can't find the container with id a21356a86f6965ddd5409ce3343e520c3111af17cc619a15a93b0b8e79aa26b9 Jul 13 14:52:50.276988 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:50.276957 2568 generic.go:356] "Generic (PLEG): container finished" podID="9a82918c-2764-47f5-aa3d-85096e2524b1" containerID="3c3ccc04fef033672d7125e9492a71df8ce79d730e88f5b78fb60a06e2a3f87f" exitCode=0 Jul 13 14:52:50.277195 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:50.277041 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk" event={"ID":"9a82918c-2764-47f5-aa3d-85096e2524b1","Type":"ContainerDied","Data":"3c3ccc04fef033672d7125e9492a71df8ce79d730e88f5b78fb60a06e2a3f87f"} Jul 13 14:52:50.279064 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:50.279045 2568 generic.go:356] "Generic (PLEG): container finished" podID="7360a6d7-1bce-4ccd-a3e3-111b9bcf051b" containerID="a595ca30c1cfc9afa17067a427c01918cb42c1034f3c6d10c0c391463932de96" exitCode=0 Jul 13 14:52:50.279164 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:50.279120 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk" event={"ID":"7360a6d7-1bce-4ccd-a3e3-111b9bcf051b","Type":"ContainerDied","Data":"a595ca30c1cfc9afa17067a427c01918cb42c1034f3c6d10c0c391463932de96"} Jul 13 14:52:50.280448 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:50.280412 2568 generic.go:356] "Generic (PLEG): container finished" podID="04986b2b-a511-4c87-907e-45105dd3222a" containerID="381d048ad08d42cede796098f02d9173e79298d8de7f9d90401161184acc1c6c" exitCode=0 Jul 13 14:52:50.280537 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:50.280484 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc" event={"ID":"04986b2b-a511-4c87-907e-45105dd3222a","Type":"ContainerDied","Data":"381d048ad08d42cede796098f02d9173e79298d8de7f9d90401161184acc1c6c"} Jul 13 14:52:50.280537 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:50.280518 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc" event={"ID":"04986b2b-a511-4c87-907e-45105dd3222a","Type":"ContainerStarted","Data":"a21356a86f6965ddd5409ce3343e520c3111af17cc619a15a93b0b8e79aa26b9"} Jul 13 14:52:51.287871 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:51.287840 2568 generic.go:356] "Generic (PLEG): container finished" podID="b7f81017-83f8-4937-b033-67823029c79b" containerID="dc825b0e6dd960870d6809b5ccd96214f33237d18e2884f1a179272575dd8e5d" exitCode=0 Jul 13 14:52:51.288281 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:51.287912 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw" event={"ID":"b7f81017-83f8-4937-b033-67823029c79b","Type":"ContainerDied","Data":"dc825b0e6dd960870d6809b5ccd96214f33237d18e2884f1a179272575dd8e5d"} Jul 13 14:52:51.289925 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:51.289903 2568 generic.go:356] "Generic (PLEG): container finished" podID="04986b2b-a511-4c87-907e-45105dd3222a" containerID="a0e9f0e051d3c93c817102fa668c7d7872fcbbc46fab97fb43b1c5a5adf27a89" exitCode=0 Jul 13 14:52:51.290044 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:51.289984 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc" event={"ID":"04986b2b-a511-4c87-907e-45105dd3222a","Type":"ContainerDied","Data":"a0e9f0e051d3c93c817102fa668c7d7872fcbbc46fab97fb43b1c5a5adf27a89"} Jul 13 14:52:51.423978 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:51.423950 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk" Jul 13 14:52:51.521405 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:51.521375 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvm86\" (UniqueName: \"kubernetes.io/projected/7360a6d7-1bce-4ccd-a3e3-111b9bcf051b-kube-api-access-vvm86\") pod \"7360a6d7-1bce-4ccd-a3e3-111b9bcf051b\" (UID: \"7360a6d7-1bce-4ccd-a3e3-111b9bcf051b\") " Jul 13 14:52:51.521602 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:51.521422 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7360a6d7-1bce-4ccd-a3e3-111b9bcf051b-util\") pod \"7360a6d7-1bce-4ccd-a3e3-111b9bcf051b\" (UID: \"7360a6d7-1bce-4ccd-a3e3-111b9bcf051b\") " Jul 13 14:52:51.521602 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:51.521444 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7360a6d7-1bce-4ccd-a3e3-111b9bcf051b-bundle\") pod \"7360a6d7-1bce-4ccd-a3e3-111b9bcf051b\" (UID: \"7360a6d7-1bce-4ccd-a3e3-111b9bcf051b\") " Jul 13 14:52:51.523744 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:51.522412 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7360a6d7-1bce-4ccd-a3e3-111b9bcf051b-bundle" (OuterVolumeSpecName: "bundle") pod "7360a6d7-1bce-4ccd-a3e3-111b9bcf051b" (UID: "7360a6d7-1bce-4ccd-a3e3-111b9bcf051b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 14:52:51.529209 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:51.529178 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7360a6d7-1bce-4ccd-a3e3-111b9bcf051b-kube-api-access-vvm86" (OuterVolumeSpecName: "kube-api-access-vvm86") pod "7360a6d7-1bce-4ccd-a3e3-111b9bcf051b" (UID: "7360a6d7-1bce-4ccd-a3e3-111b9bcf051b"). InnerVolumeSpecName "kube-api-access-vvm86". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 14:52:51.529331 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:51.529244 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7360a6d7-1bce-4ccd-a3e3-111b9bcf051b-util" (OuterVolumeSpecName: "util") pod "7360a6d7-1bce-4ccd-a3e3-111b9bcf051b" (UID: "7360a6d7-1bce-4ccd-a3e3-111b9bcf051b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 14:52:51.622606 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:51.622494 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7360a6d7-1bce-4ccd-a3e3-111b9bcf051b-util\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 14:52:51.622606 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:51.622562 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7360a6d7-1bce-4ccd-a3e3-111b9bcf051b-bundle\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 14:52:51.622606 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:51.622601 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvm86\" (UniqueName: \"kubernetes.io/projected/7360a6d7-1bce-4ccd-a3e3-111b9bcf051b-kube-api-access-vvm86\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 14:52:52.296357 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:52.296318 2568 generic.go:356] "Generic (PLEG): container finished" podID="9a82918c-2764-47f5-aa3d-85096e2524b1" containerID="d1185b46fc9b9b7e942387d3122de935f1f9789d90c151a912141be6f14d9dd9" exitCode=0 Jul 13 14:52:52.296795 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:52.296397 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk" event={"ID":"9a82918c-2764-47f5-aa3d-85096e2524b1","Type":"ContainerDied","Data":"d1185b46fc9b9b7e942387d3122de935f1f9789d90c151a912141be6f14d9dd9"} Jul 13 14:52:52.298279 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:52.298256 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk" Jul 13 14:52:52.298375 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:52.298286 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk" event={"ID":"7360a6d7-1bce-4ccd-a3e3-111b9bcf051b","Type":"ContainerDied","Data":"0042c2fc90eb3c051ad10fe5643867d4d7d6eea8e3d287800895b87c97e0979b"} Jul 13 14:52:52.298375 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:52.298330 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0042c2fc90eb3c051ad10fe5643867d4d7d6eea8e3d287800895b87c97e0979b" Jul 13 14:52:52.300553 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:52.300533 2568 generic.go:356] "Generic (PLEG): container finished" podID="b7f81017-83f8-4937-b033-67823029c79b" containerID="d576d950c3ea2c7dcaad48bc43391ba3a3d53073749a54c72517e01321885ee9" exitCode=0 Jul 13 14:52:52.300727 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:52.300615 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw" event={"ID":"b7f81017-83f8-4937-b033-67823029c79b","Type":"ContainerDied","Data":"d576d950c3ea2c7dcaad48bc43391ba3a3d53073749a54c72517e01321885ee9"} Jul 13 14:52:52.302695 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:52.302674 2568 generic.go:356] "Generic (PLEG): container finished" podID="04986b2b-a511-4c87-907e-45105dd3222a" containerID="fb6929396a45cb7004e0f0f04bee52fbdf2059f0d20a39e1f2b3a0a173fe9784" exitCode=0 Jul 13 14:52:52.302773 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:52.302720 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc" event={"ID":"04986b2b-a511-4c87-907e-45105dd3222a","Type":"ContainerDied","Data":"fb6929396a45cb7004e0f0f04bee52fbdf2059f0d20a39e1f2b3a0a173fe9784"} Jul 13 14:52:53.309619 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:53.309561 2568 generic.go:356] "Generic (PLEG): container finished" podID="9a82918c-2764-47f5-aa3d-85096e2524b1" containerID="753a25ddb1cfc3219b040e830be3bda3b33e5520701c3ac399735241950e2678" exitCode=0 Jul 13 14:52:53.309619 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:53.309614 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk" event={"ID":"9a82918c-2764-47f5-aa3d-85096e2524b1","Type":"ContainerDied","Data":"753a25ddb1cfc3219b040e830be3bda3b33e5520701c3ac399735241950e2678"} Jul 13 14:52:53.435803 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:53.435777 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc" Jul 13 14:52:53.471098 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:53.471073 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw" Jul 13 14:52:53.537000 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:53.536962 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/04986b2b-a511-4c87-907e-45105dd3222a-bundle\") pod \"04986b2b-a511-4c87-907e-45105dd3222a\" (UID: \"04986b2b-a511-4c87-907e-45105dd3222a\") " Jul 13 14:52:53.537000 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:53.537004 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7f81017-83f8-4937-b033-67823029c79b-bundle\") pod \"b7f81017-83f8-4937-b033-67823029c79b\" (UID: \"b7f81017-83f8-4937-b033-67823029c79b\") " Jul 13 14:52:53.537212 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:53.537039 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9dq4\" (UniqueName: \"kubernetes.io/projected/04986b2b-a511-4c87-907e-45105dd3222a-kube-api-access-q9dq4\") pod \"04986b2b-a511-4c87-907e-45105dd3222a\" (UID: \"04986b2b-a511-4c87-907e-45105dd3222a\") " Jul 13 14:52:53.537212 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:53.537083 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7f81017-83f8-4937-b033-67823029c79b-util\") pod \"b7f81017-83f8-4937-b033-67823029c79b\" (UID: \"b7f81017-83f8-4937-b033-67823029c79b\") " Jul 13 14:52:53.537212 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:53.537105 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/04986b2b-a511-4c87-907e-45105dd3222a-util\") pod \"04986b2b-a511-4c87-907e-45105dd3222a\" (UID: \"04986b2b-a511-4c87-907e-45105dd3222a\") " Jul 13 14:52:53.537212 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:53.537150 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9lr4\" (UniqueName: \"kubernetes.io/projected/b7f81017-83f8-4937-b033-67823029c79b-kube-api-access-d9lr4\") pod \"b7f81017-83f8-4937-b033-67823029c79b\" (UID: \"b7f81017-83f8-4937-b033-67823029c79b\") " Jul 13 14:52:53.537722 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:53.537568 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7f81017-83f8-4937-b033-67823029c79b-bundle" (OuterVolumeSpecName: "bundle") pod "b7f81017-83f8-4937-b033-67823029c79b" (UID: "b7f81017-83f8-4937-b033-67823029c79b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 14:52:53.537722 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:53.537594 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04986b2b-a511-4c87-907e-45105dd3222a-bundle" (OuterVolumeSpecName: "bundle") pod "04986b2b-a511-4c87-907e-45105dd3222a" (UID: "04986b2b-a511-4c87-907e-45105dd3222a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 14:52:53.539258 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:53.539228 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7f81017-83f8-4937-b033-67823029c79b-kube-api-access-d9lr4" (OuterVolumeSpecName: "kube-api-access-d9lr4") pod "b7f81017-83f8-4937-b033-67823029c79b" (UID: "b7f81017-83f8-4937-b033-67823029c79b"). InnerVolumeSpecName "kube-api-access-d9lr4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 14:52:53.539350 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:53.539261 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04986b2b-a511-4c87-907e-45105dd3222a-kube-api-access-q9dq4" (OuterVolumeSpecName: "kube-api-access-q9dq4") pod "04986b2b-a511-4c87-907e-45105dd3222a" (UID: "04986b2b-a511-4c87-907e-45105dd3222a"). InnerVolumeSpecName "kube-api-access-q9dq4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 14:52:53.543081 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:53.543061 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04986b2b-a511-4c87-907e-45105dd3222a-util" (OuterVolumeSpecName: "util") pod "04986b2b-a511-4c87-907e-45105dd3222a" (UID: "04986b2b-a511-4c87-907e-45105dd3222a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 14:52:53.545416 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:53.545392 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7f81017-83f8-4937-b033-67823029c79b-util" (OuterVolumeSpecName: "util") pod "b7f81017-83f8-4937-b033-67823029c79b" (UID: "b7f81017-83f8-4937-b033-67823029c79b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 14:52:53.638663 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:53.638542 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9lr4\" (UniqueName: \"kubernetes.io/projected/b7f81017-83f8-4937-b033-67823029c79b-kube-api-access-d9lr4\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 14:52:53.638663 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:53.638602 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/04986b2b-a511-4c87-907e-45105dd3222a-bundle\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 14:52:53.638663 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:53.638616 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7f81017-83f8-4937-b033-67823029c79b-bundle\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 14:52:53.638663 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:53.638627 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9dq4\" (UniqueName: \"kubernetes.io/projected/04986b2b-a511-4c87-907e-45105dd3222a-kube-api-access-q9dq4\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 14:52:53.638663 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:53.638639 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7f81017-83f8-4937-b033-67823029c79b-util\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 14:52:53.638663 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:53.638650 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/04986b2b-a511-4c87-907e-45105dd3222a-util\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 14:52:54.317130 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:54.317086 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw" event={"ID":"b7f81017-83f8-4937-b033-67823029c79b","Type":"ContainerDied","Data":"ec996b3592441d3e6aa16da8b61ace1f1198f11531ca83075e27ca723723cd01"} Jul 13 14:52:54.317616 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:54.317147 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ec996b3592441d3e6aa16da8b61ace1f1198f11531ca83075e27ca723723cd01" Jul 13 14:52:54.317616 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:54.317098 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw" Jul 13 14:52:54.319113 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:54.319080 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc" event={"ID":"04986b2b-a511-4c87-907e-45105dd3222a","Type":"ContainerDied","Data":"a21356a86f6965ddd5409ce3343e520c3111af17cc619a15a93b0b8e79aa26b9"} Jul 13 14:52:54.319272 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:54.319140 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a21356a86f6965ddd5409ce3343e520c3111af17cc619a15a93b0b8e79aa26b9" Jul 13 14:52:54.319272 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:54.319106 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc" Jul 13 14:52:54.451035 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:54.451011 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk" Jul 13 14:52:54.544956 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:54.544868 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6pfp\" (UniqueName: \"kubernetes.io/projected/9a82918c-2764-47f5-aa3d-85096e2524b1-kube-api-access-w6pfp\") pod \"9a82918c-2764-47f5-aa3d-85096e2524b1\" (UID: \"9a82918c-2764-47f5-aa3d-85096e2524b1\") " Jul 13 14:52:54.544956 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:54.544900 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a82918c-2764-47f5-aa3d-85096e2524b1-bundle\") pod \"9a82918c-2764-47f5-aa3d-85096e2524b1\" (UID: \"9a82918c-2764-47f5-aa3d-85096e2524b1\") " Jul 13 14:52:54.544956 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:54.544936 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a82918c-2764-47f5-aa3d-85096e2524b1-util\") pod \"9a82918c-2764-47f5-aa3d-85096e2524b1\" (UID: \"9a82918c-2764-47f5-aa3d-85096e2524b1\") " Jul 13 14:52:54.545487 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:54.545459 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a82918c-2764-47f5-aa3d-85096e2524b1-bundle" (OuterVolumeSpecName: "bundle") pod "9a82918c-2764-47f5-aa3d-85096e2524b1" (UID: "9a82918c-2764-47f5-aa3d-85096e2524b1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 14:52:54.547292 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:54.547268 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a82918c-2764-47f5-aa3d-85096e2524b1-kube-api-access-w6pfp" (OuterVolumeSpecName: "kube-api-access-w6pfp") pod "9a82918c-2764-47f5-aa3d-85096e2524b1" (UID: "9a82918c-2764-47f5-aa3d-85096e2524b1"). InnerVolumeSpecName "kube-api-access-w6pfp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 14:52:54.593619 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:54.593549 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a82918c-2764-47f5-aa3d-85096e2524b1-util" (OuterVolumeSpecName: "util") pod "9a82918c-2764-47f5-aa3d-85096e2524b1" (UID: "9a82918c-2764-47f5-aa3d-85096e2524b1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 14:52:54.646028 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:54.645993 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a82918c-2764-47f5-aa3d-85096e2524b1-util\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 14:52:54.646028 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:54.646024 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6pfp\" (UniqueName: \"kubernetes.io/projected/9a82918c-2764-47f5-aa3d-85096e2524b1-kube-api-access-w6pfp\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 14:52:54.646028 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:54.646034 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a82918c-2764-47f5-aa3d-85096e2524b1-bundle\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 14:52:55.326010 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:55.325972 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk" event={"ID":"9a82918c-2764-47f5-aa3d-85096e2524b1","Type":"ContainerDied","Data":"19b566ee63320242b880301d3b9173a0d23ac7ace12a059ec5f8f4b7dc120d89"} Jul 13 14:52:55.326386 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:55.326031 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="19b566ee63320242b880301d3b9173a0d23ac7ace12a059ec5f8f4b7dc120d89" Jul 13 14:52:55.326386 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:52:55.325990 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk" Jul 13 14:53:01.018618 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:01.018561 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:02.602985 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:02.602907 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:03.908588 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:03.908528 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:04.387946 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:04.387915 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:05.200030 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:05.199984 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:08.797306 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:08.797275 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:09.021760 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:09.021725 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:09.123235 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:09.123143 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:11.083772 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:11.083694 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:11.587962 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:11.587911 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:12.096421 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:12.096392 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:13.931156 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:13.931121 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:14.029154 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:14.029119 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:14.173489 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:14.173463 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:17.046193 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:17.046154 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:17.136908 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:17.136848 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:22.465309 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:22.465274 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:22.558973 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:22.558925 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:22.906568 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:22.906531 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:23.002975 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:23.002922 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:24.617633 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:24.617605 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:24.700344 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:24.700286 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:24.800768 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:24.800723 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:26.040805 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:26.040773 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:26.142329 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:26.142297 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:26.246828 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:26.246793 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:26.343093 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:26.343004 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:28.739660 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:28.739596 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:28.840040 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:28.840007 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:31.288178 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:31.288120 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:35.399017 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:35.398981 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:35.501725 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:35.501690 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:40.792689 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:40.792654 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:40.860267 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:40.860209 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:41.389054 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:41.389020 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:41.446315 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:41.446285 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:44.800786 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:44.800753 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:44.899950 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:44.899915 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:47.570431 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:47.570395 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:53:47.671450 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:53:47.671417 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:54:16.675041 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:16.675007 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:54:16.822902 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:16.822869 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:54:29.398155 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:29.398110 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:54:29.493597 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:29.493552 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:54:29.596874 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:29.596844 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:54:30.225111 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.225076 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-rfqhx"] Jul 13 14:54:30.225374 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.225362 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7360a6d7-1bce-4ccd-a3e3-111b9bcf051b" containerName="extract" Jul 13 14:54:30.225417 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.225375 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7360a6d7-1bce-4ccd-a3e3-111b9bcf051b" containerName="extract" Jul 13 14:54:30.225417 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.225387 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7f81017-83f8-4937-b033-67823029c79b" containerName="extract" Jul 13 14:54:30.225417 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.225392 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7f81017-83f8-4937-b033-67823029c79b" containerName="extract" Jul 13 14:54:30.225417 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.225402 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04986b2b-a511-4c87-907e-45105dd3222a" containerName="extract" Jul 13 14:54:30.225417 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.225407 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="04986b2b-a511-4c87-907e-45105dd3222a" containerName="extract" Jul 13 14:54:30.225417 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.225412 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a82918c-2764-47f5-aa3d-85096e2524b1" containerName="pull" Jul 13 14:54:30.225417 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.225418 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a82918c-2764-47f5-aa3d-85096e2524b1" containerName="pull" Jul 13 14:54:30.225643 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.225427 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7f81017-83f8-4937-b033-67823029c79b" containerName="util" Jul 13 14:54:30.225643 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.225433 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7f81017-83f8-4937-b033-67823029c79b" containerName="util" Jul 13 14:54:30.225643 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.225439 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7360a6d7-1bce-4ccd-a3e3-111b9bcf051b" containerName="pull" Jul 13 14:54:30.225643 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.225444 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7360a6d7-1bce-4ccd-a3e3-111b9bcf051b" containerName="pull" Jul 13 14:54:30.225643 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.225450 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a82918c-2764-47f5-aa3d-85096e2524b1" containerName="extract" Jul 13 14:54:30.225643 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.225454 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a82918c-2764-47f5-aa3d-85096e2524b1" containerName="extract" Jul 13 14:54:30.225643 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.225463 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04986b2b-a511-4c87-907e-45105dd3222a" containerName="util" Jul 13 14:54:30.225643 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.225468 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="04986b2b-a511-4c87-907e-45105dd3222a" containerName="util" Jul 13 14:54:30.225643 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.225474 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04986b2b-a511-4c87-907e-45105dd3222a" containerName="pull" Jul 13 14:54:30.225643 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.225479 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="04986b2b-a511-4c87-907e-45105dd3222a" containerName="pull" Jul 13 14:54:30.225643 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.225488 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7360a6d7-1bce-4ccd-a3e3-111b9bcf051b" containerName="util" Jul 13 14:54:30.225643 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.225492 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7360a6d7-1bce-4ccd-a3e3-111b9bcf051b" containerName="util" Jul 13 14:54:30.225643 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.225499 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a82918c-2764-47f5-aa3d-85096e2524b1" containerName="util" Jul 13 14:54:30.225643 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.225504 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a82918c-2764-47f5-aa3d-85096e2524b1" containerName="util" Jul 13 14:54:30.225643 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.225510 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7f81017-83f8-4937-b033-67823029c79b" containerName="pull" Jul 13 14:54:30.225643 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.225515 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7f81017-83f8-4937-b033-67823029c79b" containerName="pull" Jul 13 14:54:30.225643 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.225561 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7360a6d7-1bce-4ccd-a3e3-111b9bcf051b" containerName="extract" Jul 13 14:54:30.225643 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.225568 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7f81017-83f8-4937-b033-67823029c79b" containerName="extract" Jul 13 14:54:30.225643 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.225587 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a82918c-2764-47f5-aa3d-85096e2524b1" containerName="extract" Jul 13 14:54:30.225643 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.225594 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="04986b2b-a511-4c87-907e-45105dd3222a" containerName="extract" Jul 13 14:54:30.228163 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.228147 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rfqhx" Jul 13 14:54:30.230388 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.230366 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 13 14:54:30.231369 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.231350 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-j2n7z\"" Jul 13 14:54:30.231474 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.231381 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 13 14:54:30.236493 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.236466 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-rfqhx"] Jul 13 14:54:30.256086 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.256061 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:54:30.316083 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.316053 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:54:30.352282 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.352250 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tj4b\" (UniqueName: \"kubernetes.io/projected/9b6b6e9f-37f0-43fa-b1c7-ee27a17ca529-kube-api-access-8tj4b\") pod \"authorino-58cd955675-rfqhx\" (UID: \"9b6b6e9f-37f0-43fa-b1c7-ee27a17ca529\") " pod="kuadrant-system/authorino-58cd955675-rfqhx" Jul 13 14:54:30.426751 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.426716 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-mzt7d"] Jul 13 14:54:30.429753 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.429733 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mzt7d" Jul 13 14:54:30.437398 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.437376 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mzt7d"] Jul 13 14:54:30.453095 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.453064 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tj4b\" (UniqueName: \"kubernetes.io/projected/9b6b6e9f-37f0-43fa-b1c7-ee27a17ca529-kube-api-access-8tj4b\") pod \"authorino-58cd955675-rfqhx\" (UID: \"9b6b6e9f-37f0-43fa-b1c7-ee27a17ca529\") " pod="kuadrant-system/authorino-58cd955675-rfqhx" Jul 13 14:54:30.461721 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.461700 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tj4b\" (UniqueName: \"kubernetes.io/projected/9b6b6e9f-37f0-43fa-b1c7-ee27a17ca529-kube-api-access-8tj4b\") pod \"authorino-58cd955675-rfqhx\" (UID: \"9b6b6e9f-37f0-43fa-b1c7-ee27a17ca529\") " pod="kuadrant-system/authorino-58cd955675-rfqhx" Jul 13 14:54:30.537531 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.537501 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rfqhx" Jul 13 14:54:30.553733 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.553705 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnwhp\" (UniqueName: \"kubernetes.io/projected/f5bdff5f-1914-477c-bc91-2e990829ef59-kube-api-access-xnwhp\") pod \"authorino-75d556bd77-mzt7d\" (UID: \"f5bdff5f-1914-477c-bc91-2e990829ef59\") " pod="kuadrant-system/authorino-75d556bd77-mzt7d" Jul 13 14:54:30.655157 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.655103 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnwhp\" (UniqueName: \"kubernetes.io/projected/f5bdff5f-1914-477c-bc91-2e990829ef59-kube-api-access-xnwhp\") pod \"authorino-75d556bd77-mzt7d\" (UID: \"f5bdff5f-1914-477c-bc91-2e990829ef59\") " pod="kuadrant-system/authorino-75d556bd77-mzt7d" Jul 13 14:54:30.664011 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.663987 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnwhp\" (UniqueName: \"kubernetes.io/projected/f5bdff5f-1914-477c-bc91-2e990829ef59-kube-api-access-xnwhp\") pod \"authorino-75d556bd77-mzt7d\" (UID: \"f5bdff5f-1914-477c-bc91-2e990829ef59\") " pod="kuadrant-system/authorino-75d556bd77-mzt7d" Jul 13 14:54:30.667183 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.667157 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-rfqhx"] Jul 13 14:54:30.738871 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.738835 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mzt7d" Jul 13 14:54:30.788847 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.788763 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rfqhx" event={"ID":"9b6b6e9f-37f0-43fa-b1c7-ee27a17ca529","Type":"ContainerStarted","Data":"c0b8a8a29b529f7b9c4adbffd34c1494e232fb93d1ffad55ae08a9ec163975ab"} Jul 13 14:54:30.870042 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:30.870014 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mzt7d"] Jul 13 14:54:30.870865 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:54:30.870841 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf5bdff5f_1914_477c_bc91_2e990829ef59.slice/crio-9cb1cbc41114cb0be36ac001d9ad7201c7d9876f101c9b1d0da8df03ea69bf9c WatchSource:0}: Error finding container 9cb1cbc41114cb0be36ac001d9ad7201c7d9876f101c9b1d0da8df03ea69bf9c: Status 404 returned error can't find the container with id 9cb1cbc41114cb0be36ac001d9ad7201c7d9876f101c9b1d0da8df03ea69bf9c Jul 13 14:54:31.801489 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:31.801452 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mzt7d" event={"ID":"f5bdff5f-1914-477c-bc91-2e990829ef59","Type":"ContainerStarted","Data":"9cb1cbc41114cb0be36ac001d9ad7201c7d9876f101c9b1d0da8df03ea69bf9c"} Jul 13 14:54:34.819383 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:34.819345 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rfqhx" event={"ID":"9b6b6e9f-37f0-43fa-b1c7-ee27a17ca529","Type":"ContainerStarted","Data":"6eb3bff07c7e6fe81b4bb103ff64da52e4696852dab6eef79c5b6d0260b1abaa"} Jul 13 14:54:34.821027 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:34.820997 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mzt7d" event={"ID":"f5bdff5f-1914-477c-bc91-2e990829ef59","Type":"ContainerStarted","Data":"b222b0cf7441ee28b3457a91d6f5d31a069349a16c31024e937e126fde5980ca"} Jul 13 14:54:34.834825 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:34.834762 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-rfqhx" podStartSLOduration=1.12409066 podStartE2EDuration="4.834743664s" totalImagesPullingTime="3.710653004s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 14:54:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:54:34.834411503 +0000 UTC m=+487.074689356" watchObservedRunningTime="2026-07-13 14:54:34.834743664 +0000 UTC m=+487.075021515" Jul 13 14:54:34.849975 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:34.849918 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-mzt7d" podStartSLOduration=1.349630672 podStartE2EDuration="4.849900758s" totalImagesPullingTime="3.500270086s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 14:54:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:54:34.84816498 +0000 UTC m=+487.088442829" watchObservedRunningTime="2026-07-13 14:54:34.849900758 +0000 UTC m=+487.090178607" Jul 13 14:54:34.877639 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:34.877603 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-rfqhx"] Jul 13 14:54:36.830983 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:36.830943 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-rfqhx" podUID="9b6b6e9f-37f0-43fa-b1c7-ee27a17ca529" containerName="authorino" containerID="cri-o://6eb3bff07c7e6fe81b4bb103ff64da52e4696852dab6eef79c5b6d0260b1abaa" gracePeriod=30 Jul 13 14:54:37.079913 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:37.079889 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rfqhx" Jul 13 14:54:37.212340 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:37.212250 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8tj4b\" (UniqueName: \"kubernetes.io/projected/9b6b6e9f-37f0-43fa-b1c7-ee27a17ca529-kube-api-access-8tj4b\") pod \"9b6b6e9f-37f0-43fa-b1c7-ee27a17ca529\" (UID: \"9b6b6e9f-37f0-43fa-b1c7-ee27a17ca529\") " Jul 13 14:54:37.214293 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:37.214269 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b6b6e9f-37f0-43fa-b1c7-ee27a17ca529-kube-api-access-8tj4b" (OuterVolumeSpecName: "kube-api-access-8tj4b") pod "9b6b6e9f-37f0-43fa-b1c7-ee27a17ca529" (UID: "9b6b6e9f-37f0-43fa-b1c7-ee27a17ca529"). InnerVolumeSpecName "kube-api-access-8tj4b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 14:54:37.313072 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:37.313039 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8tj4b\" (UniqueName: \"kubernetes.io/projected/9b6b6e9f-37f0-43fa-b1c7-ee27a17ca529-kube-api-access-8tj4b\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 14:54:37.837593 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:37.837533 2568 generic.go:356] "Generic (PLEG): container finished" podID="9b6b6e9f-37f0-43fa-b1c7-ee27a17ca529" containerID="6eb3bff07c7e6fe81b4bb103ff64da52e4696852dab6eef79c5b6d0260b1abaa" exitCode=0 Jul 13 14:54:37.837593 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:37.837594 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rfqhx" event={"ID":"9b6b6e9f-37f0-43fa-b1c7-ee27a17ca529","Type":"ContainerDied","Data":"6eb3bff07c7e6fe81b4bb103ff64da52e4696852dab6eef79c5b6d0260b1abaa"} Jul 13 14:54:37.838092 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:37.837628 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rfqhx" event={"ID":"9b6b6e9f-37f0-43fa-b1c7-ee27a17ca529","Type":"ContainerDied","Data":"c0b8a8a29b529f7b9c4adbffd34c1494e232fb93d1ffad55ae08a9ec163975ab"} Jul 13 14:54:37.838092 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:37.837645 2568 scope.go:117] "RemoveContainer" containerID="6eb3bff07c7e6fe81b4bb103ff64da52e4696852dab6eef79c5b6d0260b1abaa" Jul 13 14:54:37.838092 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:37.837596 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rfqhx" Jul 13 14:54:37.846566 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:37.846545 2568 scope.go:117] "RemoveContainer" containerID="6eb3bff07c7e6fe81b4bb103ff64da52e4696852dab6eef79c5b6d0260b1abaa" Jul 13 14:54:37.846922 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:54:37.846904 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6eb3bff07c7e6fe81b4bb103ff64da52e4696852dab6eef79c5b6d0260b1abaa\": container with ID starting with 6eb3bff07c7e6fe81b4bb103ff64da52e4696852dab6eef79c5b6d0260b1abaa not found: ID does not exist" containerID="6eb3bff07c7e6fe81b4bb103ff64da52e4696852dab6eef79c5b6d0260b1abaa" Jul 13 14:54:37.846970 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:37.846931 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6eb3bff07c7e6fe81b4bb103ff64da52e4696852dab6eef79c5b6d0260b1abaa"} err="failed to get container status \"6eb3bff07c7e6fe81b4bb103ff64da52e4696852dab6eef79c5b6d0260b1abaa\": rpc error: code = NotFound desc = could not find container \"6eb3bff07c7e6fe81b4bb103ff64da52e4696852dab6eef79c5b6d0260b1abaa\": container with ID starting with 6eb3bff07c7e6fe81b4bb103ff64da52e4696852dab6eef79c5b6d0260b1abaa not found: ID does not exist" Jul 13 14:54:37.863479 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:37.863450 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-rfqhx"] Jul 13 14:54:37.866478 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:37.866454 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-rfqhx"] Jul 13 14:54:38.272261 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:38.272181 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b6b6e9f-37f0-43fa-b1c7-ee27a17ca529" path="/var/lib/kubelet/pods/9b6b6e9f-37f0-43fa-b1c7-ee27a17ca529/volumes" Jul 13 14:54:49.116044 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:54:49.116011 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:55:00.988465 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:55:00.988419 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:55:01.324782 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:55:01.324750 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:55:06.434288 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:55:06.434254 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mzt7d"] Jul 13 14:55:06.434694 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:55:06.434434 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-mzt7d" podUID="f5bdff5f-1914-477c-bc91-2e990829ef59" containerName="authorino" containerID="cri-o://b222b0cf7441ee28b3457a91d6f5d31a069349a16c31024e937e126fde5980ca" gracePeriod=30 Jul 13 14:55:06.692121 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:55:06.692058 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mzt7d" Jul 13 14:55:06.769527 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:55:06.769490 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xnwhp\" (UniqueName: \"kubernetes.io/projected/f5bdff5f-1914-477c-bc91-2e990829ef59-kube-api-access-xnwhp\") pod \"f5bdff5f-1914-477c-bc91-2e990829ef59\" (UID: \"f5bdff5f-1914-477c-bc91-2e990829ef59\") " Jul 13 14:55:06.771564 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:55:06.771542 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5bdff5f-1914-477c-bc91-2e990829ef59-kube-api-access-xnwhp" (OuterVolumeSpecName: "kube-api-access-xnwhp") pod "f5bdff5f-1914-477c-bc91-2e990829ef59" (UID: "f5bdff5f-1914-477c-bc91-2e990829ef59"). InnerVolumeSpecName "kube-api-access-xnwhp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 14:55:06.870505 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:55:06.870465 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnwhp\" (UniqueName: \"kubernetes.io/projected/f5bdff5f-1914-477c-bc91-2e990829ef59-kube-api-access-xnwhp\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 14:55:06.976936 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:55:06.976841 2568 generic.go:356] "Generic (PLEG): container finished" podID="f5bdff5f-1914-477c-bc91-2e990829ef59" containerID="b222b0cf7441ee28b3457a91d6f5d31a069349a16c31024e937e126fde5980ca" exitCode=0 Jul 13 14:55:06.976936 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:55:06.976883 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mzt7d" Jul 13 14:55:06.977114 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:55:06.976932 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mzt7d" event={"ID":"f5bdff5f-1914-477c-bc91-2e990829ef59","Type":"ContainerDied","Data":"b222b0cf7441ee28b3457a91d6f5d31a069349a16c31024e937e126fde5980ca"} Jul 13 14:55:06.977114 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:55:06.976971 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mzt7d" event={"ID":"f5bdff5f-1914-477c-bc91-2e990829ef59","Type":"ContainerDied","Data":"9cb1cbc41114cb0be36ac001d9ad7201c7d9876f101c9b1d0da8df03ea69bf9c"} Jul 13 14:55:06.977114 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:55:06.976992 2568 scope.go:117] "RemoveContainer" containerID="b222b0cf7441ee28b3457a91d6f5d31a069349a16c31024e937e126fde5980ca" Jul 13 14:55:06.986098 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:55:06.986079 2568 scope.go:117] "RemoveContainer" containerID="b222b0cf7441ee28b3457a91d6f5d31a069349a16c31024e937e126fde5980ca" Jul 13 14:55:06.986369 ip-10-0-136-96 kubenswrapper[2568]: E0713 14:55:06.986351 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b222b0cf7441ee28b3457a91d6f5d31a069349a16c31024e937e126fde5980ca\": container with ID starting with b222b0cf7441ee28b3457a91d6f5d31a069349a16c31024e937e126fde5980ca not found: ID does not exist" containerID="b222b0cf7441ee28b3457a91d6f5d31a069349a16c31024e937e126fde5980ca" Jul 13 14:55:06.986443 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:55:06.986377 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b222b0cf7441ee28b3457a91d6f5d31a069349a16c31024e937e126fde5980ca"} err="failed to get container status \"b222b0cf7441ee28b3457a91d6f5d31a069349a16c31024e937e126fde5980ca\": rpc error: code = NotFound desc = could not find container \"b222b0cf7441ee28b3457a91d6f5d31a069349a16c31024e937e126fde5980ca\": container with ID starting with b222b0cf7441ee28b3457a91d6f5d31a069349a16c31024e937e126fde5980ca not found: ID does not exist" Jul 13 14:55:06.999094 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:55:06.999066 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mzt7d"] Jul 13 14:55:07.001272 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:55:07.001250 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mzt7d"] Jul 13 14:55:07.596055 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:55:07.596025 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:55:08.277720 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:55:08.277683 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5bdff5f-1914-477c-bc91-2e990829ef59" path="/var/lib/kubelet/pods/f5bdff5f-1914-477c-bc91-2e990829ef59/volumes" Jul 13 14:55:10.982076 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:55:10.982045 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:55:11.077302 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:55:11.077268 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:55:11.166115 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:55:11.166084 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:55:13.952640 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:55:13.952606 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:55:14.068091 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:55:14.068061 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:55:46.105333 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:55:46.105292 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:55:46.209648 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:55:46.209610 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:55:46.333782 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:55:46.333750 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:55:47.001336 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:55:47.001299 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:55:47.086607 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:55:47.086554 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:55:56.924440 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:55:56.924408 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:55:57.186792 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:55:57.186707 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:56:04.196482 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:04.196442 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv"] Jul 13 14:56:04.196972 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:04.196917 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b6b6e9f-37f0-43fa-b1c7-ee27a17ca529" containerName="authorino" Jul 13 14:56:04.196972 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:04.196935 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b6b6e9f-37f0-43fa-b1c7-ee27a17ca529" containerName="authorino" Jul 13 14:56:04.196972 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:04.196956 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5bdff5f-1914-477c-bc91-2e990829ef59" containerName="authorino" Jul 13 14:56:04.196972 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:04.196965 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5bdff5f-1914-477c-bc91-2e990829ef59" containerName="authorino" Jul 13 14:56:04.197172 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:04.197057 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b6b6e9f-37f0-43fa-b1c7-ee27a17ca529" containerName="authorino" Jul 13 14:56:04.197172 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:04.197070 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5bdff5f-1914-477c-bc91-2e990829ef59" containerName="authorino" Jul 13 14:56:04.199938 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:04.199918 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv" Jul 13 14:56:04.202480 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:04.202456 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 13 14:56:04.203455 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:04.203428 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 13 14:56:04.203563 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:04.203454 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 13 14:56:04.203657 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:04.203611 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-8467h\"" Jul 13 14:56:04.210896 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:04.210874 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv"] Jul 13 14:56:04.227011 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:04.226975 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:56:04.246264 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:04.246225 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bflf7\" (UniqueName: \"kubernetes.io/projected/a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab-kube-api-access-bflf7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv\" (UID: \"a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv" Jul 13 14:56:04.246437 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:04.246277 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv\" (UID: \"a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv" Jul 13 14:56:04.246437 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:04.246301 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv\" (UID: \"a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv" Jul 13 14:56:04.246437 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:04.246379 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv\" (UID: \"a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv" Jul 13 14:56:04.246437 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:04.246420 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv\" (UID: \"a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv" Jul 13 14:56:04.246654 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:04.246479 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv\" (UID: \"a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv" Jul 13 14:56:04.347628 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:04.347563 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv\" (UID: \"a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv" Jul 13 14:56:04.347809 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:04.347663 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bflf7\" (UniqueName: \"kubernetes.io/projected/a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab-kube-api-access-bflf7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv\" (UID: \"a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv" Jul 13 14:56:04.347809 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:04.347698 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv\" (UID: \"a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv" Jul 13 14:56:04.347809 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:04.347723 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv\" (UID: \"a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv" Jul 13 14:56:04.348010 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:04.347871 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv\" (UID: \"a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv" Jul 13 14:56:04.348010 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:04.347930 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv\" (UID: \"a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv" Jul 13 14:56:04.348151 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:04.348138 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv\" (UID: \"a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv" Jul 13 14:56:04.348210 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:04.348188 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv\" (UID: \"a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv" Jul 13 14:56:04.348318 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:04.348299 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv\" (UID: \"a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv" Jul 13 14:56:04.349928 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:04.349909 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv\" (UID: \"a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv" Jul 13 14:56:04.350214 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:04.350196 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv\" (UID: \"a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv" Jul 13 14:56:04.356801 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:04.356775 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bflf7\" (UniqueName: \"kubernetes.io/projected/a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab-kube-api-access-bflf7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv\" (UID: \"a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv" Jul 13 14:56:04.510499 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:04.510408 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv" Jul 13 14:56:04.641132 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:04.641108 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv"] Jul 13 14:56:04.642703 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:56:04.642678 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda08294c3_7fc6_4c60_8bd7_1e2d297ca3ab.slice/crio-43435dffde9a599ab9e879d4ed091ba3a076d0d41365e6d809a2feb6ca481887 WatchSource:0}: Error finding container 43435dffde9a599ab9e879d4ed091ba3a076d0d41365e6d809a2feb6ca481887: Status 404 returned error can't find the container with id 43435dffde9a599ab9e879d4ed091ba3a076d0d41365e6d809a2feb6ca481887 Jul 13 14:56:05.270796 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:05.270757 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv" event={"ID":"a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab","Type":"ContainerStarted","Data":"43435dffde9a599ab9e879d4ed091ba3a076d0d41365e6d809a2feb6ca481887"} Jul 13 14:56:06.276276 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:06.276226 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv" event={"ID":"a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab","Type":"ContainerStarted","Data":"28546bc862cbaeb669709515ed7d39130aefa844ff960c16a6c5493ca9773087"} Jul 13 14:56:06.294890 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:06.294839 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv" podStartSLOduration=1.189160122 podStartE2EDuration="2.294821606s" totalImagesPullingTime="1.105661484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 14:56:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:56:06.293382875 +0000 UTC m=+578.533660724" watchObservedRunningTime="2026-07-13 14:56:06.294821606 +0000 UTC m=+578.535099454" Jul 13 14:56:09.815525 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:09.815490 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:56:11.492774 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:11.492736 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:56:11.594521 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:11.594484 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:56:14.510813 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:14.510776 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv" Jul 13 14:56:14.511195 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:14.510856 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv" Jul 13 14:56:14.523451 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:14.523425 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv" Jul 13 14:56:15.329432 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:15.329406 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv" Jul 13 14:56:15.551005 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:15.550973 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:56:15.618899 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:15.618823 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:56:16.381123 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:16.381094 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:56:17.977642 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:17.977610 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:56:24.393119 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:24.393084 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25"] Jul 13 14:56:24.396158 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:24.396140 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25" Jul 13 14:56:24.398451 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:24.398422 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 13 14:56:24.405500 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:24.405475 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25"] Jul 13 14:56:24.500239 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:24.500206 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:56:24.526622 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:24.526566 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dac9630f-90ed-4882-8465-b3954148c835-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25\" (UID: \"dac9630f-90ed-4882-8465-b3954148c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25" Jul 13 14:56:24.526761 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:24.526629 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dac9630f-90ed-4882-8465-b3954148c835-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25\" (UID: \"dac9630f-90ed-4882-8465-b3954148c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25" Jul 13 14:56:24.526761 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:24.526666 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dac9630f-90ed-4882-8465-b3954148c835-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25\" (UID: \"dac9630f-90ed-4882-8465-b3954148c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25" Jul 13 14:56:24.526761 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:24.526730 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dac9630f-90ed-4882-8465-b3954148c835-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25\" (UID: \"dac9630f-90ed-4882-8465-b3954148c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25" Jul 13 14:56:24.526761 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:24.526756 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rthmn\" (UniqueName: \"kubernetes.io/projected/dac9630f-90ed-4882-8465-b3954148c835-kube-api-access-rthmn\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25\" (UID: \"dac9630f-90ed-4882-8465-b3954148c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25" Jul 13 14:56:24.526893 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:24.526802 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dac9630f-90ed-4882-8465-b3954148c835-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25\" (UID: \"dac9630f-90ed-4882-8465-b3954148c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25" Jul 13 14:56:24.627879 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:24.627840 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dac9630f-90ed-4882-8465-b3954148c835-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25\" (UID: \"dac9630f-90ed-4882-8465-b3954148c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25" Jul 13 14:56:24.627879 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:24.627883 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rthmn\" (UniqueName: \"kubernetes.io/projected/dac9630f-90ed-4882-8465-b3954148c835-kube-api-access-rthmn\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25\" (UID: \"dac9630f-90ed-4882-8465-b3954148c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25" Jul 13 14:56:24.628102 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:24.627936 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dac9630f-90ed-4882-8465-b3954148c835-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25\" (UID: \"dac9630f-90ed-4882-8465-b3954148c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25" Jul 13 14:56:24.628102 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:24.627957 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dac9630f-90ed-4882-8465-b3954148c835-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25\" (UID: \"dac9630f-90ed-4882-8465-b3954148c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25" Jul 13 14:56:24.628102 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:24.627972 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dac9630f-90ed-4882-8465-b3954148c835-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25\" (UID: \"dac9630f-90ed-4882-8465-b3954148c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25" Jul 13 14:56:24.628102 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:24.628007 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dac9630f-90ed-4882-8465-b3954148c835-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25\" (UID: \"dac9630f-90ed-4882-8465-b3954148c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25" Jul 13 14:56:24.628330 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:24.628303 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dac9630f-90ed-4882-8465-b3954148c835-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25\" (UID: \"dac9630f-90ed-4882-8465-b3954148c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25" Jul 13 14:56:24.628389 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:24.628328 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dac9630f-90ed-4882-8465-b3954148c835-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25\" (UID: \"dac9630f-90ed-4882-8465-b3954148c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25" Jul 13 14:56:24.628434 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:24.628414 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dac9630f-90ed-4882-8465-b3954148c835-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25\" (UID: \"dac9630f-90ed-4882-8465-b3954148c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25" Jul 13 14:56:24.630305 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:24.630281 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dac9630f-90ed-4882-8465-b3954148c835-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25\" (UID: \"dac9630f-90ed-4882-8465-b3954148c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25" Jul 13 14:56:24.630439 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:24.630420 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dac9630f-90ed-4882-8465-b3954148c835-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25\" (UID: \"dac9630f-90ed-4882-8465-b3954148c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25" Jul 13 14:56:24.637427 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:24.637402 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rthmn\" (UniqueName: \"kubernetes.io/projected/dac9630f-90ed-4882-8465-b3954148c835-kube-api-access-rthmn\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25\" (UID: \"dac9630f-90ed-4882-8465-b3954148c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25" Jul 13 14:56:24.707243 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:24.707143 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25" Jul 13 14:56:24.839025 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:24.838995 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25"] Jul 13 14:56:24.840373 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:56:24.840347 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddac9630f_90ed_4882_8465_b3954148c835.slice/crio-91958c9143227a6444710125463feedc30beb6f810bc2a1ab48be1096eab9f1e WatchSource:0}: Error finding container 91958c9143227a6444710125463feedc30beb6f810bc2a1ab48be1096eab9f1e: Status 404 returned error can't find the container with id 91958c9143227a6444710125463feedc30beb6f810bc2a1ab48be1096eab9f1e Jul 13 14:56:25.365786 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:25.365749 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25" event={"ID":"dac9630f-90ed-4882-8465-b3954148c835","Type":"ContainerStarted","Data":"b253ac457df0661c6b90a529be9a046c4aae79018eca94b3b45f28292aac1348"} Jul 13 14:56:25.365997 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:25.365813 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25" event={"ID":"dac9630f-90ed-4882-8465-b3954148c835","Type":"ContainerStarted","Data":"91958c9143227a6444710125463feedc30beb6f810bc2a1ab48be1096eab9f1e"} Jul 13 14:56:25.384080 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:25.384026 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25" podStartSLOduration=1.210880834 podStartE2EDuration="1.38400537s" totalImagesPullingTime="173.124536ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 14:56:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:56:25.38306137 +0000 UTC m=+597.623339240" watchObservedRunningTime="2026-07-13 14:56:25.38400537 +0000 UTC m=+597.624283265" Jul 13 14:56:28.171338 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:28.171309 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xgqvx_0bd25f5c-138c-4c60-9d20-525c8b12ebb1/ovn-acl-logging/0.log" Jul 13 14:56:28.171784 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:28.171358 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xgqvx_0bd25f5c-138c-4c60-9d20-525c8b12ebb1/ovn-acl-logging/0.log" Jul 13 14:56:32.633720 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:32.633683 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:56:34.708092 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:34.708048 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25" Jul 13 14:56:34.708485 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:34.708124 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25" Jul 13 14:56:34.720755 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:34.720721 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25" Jul 13 14:56:35.426993 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:35.426955 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25" Jul 13 14:56:52.600906 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:52.600831 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n"] Jul 13 14:56:52.603977 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:52.603959 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n" Jul 13 14:56:52.606275 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:52.606257 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 13 14:56:52.613406 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:52.613384 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n"] Jul 13 14:56:52.629525 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:52.629496 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:56:52.680623 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:52.680561 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d0ab6f4-cec5-48c7-a369-deb499405d34-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n\" (UID: \"2d0ab6f4-cec5-48c7-a369-deb499405d34\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n" Jul 13 14:56:52.680781 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:52.680641 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfbjn\" (UniqueName: \"kubernetes.io/projected/2d0ab6f4-cec5-48c7-a369-deb499405d34-kube-api-access-dfbjn\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n\" (UID: \"2d0ab6f4-cec5-48c7-a369-deb499405d34\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n" Jul 13 14:56:52.680781 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:52.680673 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d0ab6f4-cec5-48c7-a369-deb499405d34-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n\" (UID: \"2d0ab6f4-cec5-48c7-a369-deb499405d34\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n" Jul 13 14:56:52.680781 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:52.680756 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d0ab6f4-cec5-48c7-a369-deb499405d34-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n\" (UID: \"2d0ab6f4-cec5-48c7-a369-deb499405d34\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n" Jul 13 14:56:52.680885 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:52.680818 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d0ab6f4-cec5-48c7-a369-deb499405d34-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n\" (UID: \"2d0ab6f4-cec5-48c7-a369-deb499405d34\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n" Jul 13 14:56:52.680885 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:52.680838 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d0ab6f4-cec5-48c7-a369-deb499405d34-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n\" (UID: \"2d0ab6f4-cec5-48c7-a369-deb499405d34\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n" Jul 13 14:56:52.781562 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:52.781526 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d0ab6f4-cec5-48c7-a369-deb499405d34-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n\" (UID: \"2d0ab6f4-cec5-48c7-a369-deb499405d34\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n" Jul 13 14:56:52.781748 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:52.781603 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d0ab6f4-cec5-48c7-a369-deb499405d34-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n\" (UID: \"2d0ab6f4-cec5-48c7-a369-deb499405d34\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n" Jul 13 14:56:52.781748 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:52.781632 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d0ab6f4-cec5-48c7-a369-deb499405d34-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n\" (UID: \"2d0ab6f4-cec5-48c7-a369-deb499405d34\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n" Jul 13 14:56:52.781748 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:52.781706 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d0ab6f4-cec5-48c7-a369-deb499405d34-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n\" (UID: \"2d0ab6f4-cec5-48c7-a369-deb499405d34\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n" Jul 13 14:56:52.781748 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:52.781732 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfbjn\" (UniqueName: \"kubernetes.io/projected/2d0ab6f4-cec5-48c7-a369-deb499405d34-kube-api-access-dfbjn\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n\" (UID: \"2d0ab6f4-cec5-48c7-a369-deb499405d34\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n" Jul 13 14:56:52.781945 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:52.781758 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d0ab6f4-cec5-48c7-a369-deb499405d34-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n\" (UID: \"2d0ab6f4-cec5-48c7-a369-deb499405d34\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n" Jul 13 14:56:52.782092 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:52.782067 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d0ab6f4-cec5-48c7-a369-deb499405d34-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n\" (UID: \"2d0ab6f4-cec5-48c7-a369-deb499405d34\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n" Jul 13 14:56:52.782174 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:52.782095 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d0ab6f4-cec5-48c7-a369-deb499405d34-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n\" (UID: \"2d0ab6f4-cec5-48c7-a369-deb499405d34\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n" Jul 13 14:56:52.782174 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:52.782136 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d0ab6f4-cec5-48c7-a369-deb499405d34-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n\" (UID: \"2d0ab6f4-cec5-48c7-a369-deb499405d34\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n" Jul 13 14:56:52.783872 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:52.783853 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d0ab6f4-cec5-48c7-a369-deb499405d34-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n\" (UID: \"2d0ab6f4-cec5-48c7-a369-deb499405d34\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n" Jul 13 14:56:52.784140 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:52.784121 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d0ab6f4-cec5-48c7-a369-deb499405d34-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n\" (UID: \"2d0ab6f4-cec5-48c7-a369-deb499405d34\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n" Jul 13 14:56:52.791230 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:52.791209 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfbjn\" (UniqueName: \"kubernetes.io/projected/2d0ab6f4-cec5-48c7-a369-deb499405d34-kube-api-access-dfbjn\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n\" (UID: \"2d0ab6f4-cec5-48c7-a369-deb499405d34\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n" Jul 13 14:56:52.914119 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:52.914014 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n" Jul 13 14:56:53.045394 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:53.045367 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n"] Jul 13 14:56:53.046794 ip-10-0-136-96 kubenswrapper[2568]: W0713 14:56:53.046757 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2d0ab6f4_cec5_48c7_a369_deb499405d34.slice/crio-56d2cadec632530e246ae85a1c75667acaa7a3d25412cf11fbf6d816acd624b4 WatchSource:0}: Error finding container 56d2cadec632530e246ae85a1c75667acaa7a3d25412cf11fbf6d816acd624b4: Status 404 returned error can't find the container with id 56d2cadec632530e246ae85a1c75667acaa7a3d25412cf11fbf6d816acd624b4 Jul 13 14:56:53.048087 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:53.048068 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 14:56:53.503162 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:53.503122 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n" event={"ID":"2d0ab6f4-cec5-48c7-a369-deb499405d34","Type":"ContainerStarted","Data":"43689e6144586a9f757296a9343b75e873234166c41e3cfcd8af3552f3f4e15d"} Jul 13 14:56:53.503162 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:53.503168 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n" event={"ID":"2d0ab6f4-cec5-48c7-a369-deb499405d34","Type":"ContainerStarted","Data":"56d2cadec632530e246ae85a1c75667acaa7a3d25412cf11fbf6d816acd624b4"} Jul 13 14:56:53.525755 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:56:53.522434 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n" podStartSLOduration=1.311263543 podStartE2EDuration="1.522414059s" totalImagesPullingTime="211.150516ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 14:56:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 14:56:53.519493714 +0000 UTC m=+625.759771563" watchObservedRunningTime="2026-07-13 14:56:53.522414059 +0000 UTC m=+625.762691915" Jul 13 14:57:02.915202 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:57:02.915140 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n" Jul 13 14:57:02.915202 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:57:02.915212 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n" Jul 13 14:57:02.928162 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:57:02.928135 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n" Jul 13 14:57:03.564196 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:57:03.564167 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n" Jul 13 14:57:38.648398 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:57:38.648364 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:57:48.760349 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:57:48.760310 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:57:48.853799 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:57:48.853768 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:57:49.972743 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:57:49.972708 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:57:50.074374 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:57:50.074335 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:58:13.628698 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:58:13.628658 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:59:01.858283 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:59:01.858251 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:59:03.466113 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:59:03.466074 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:59:47.278201 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:59:47.278167 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:59:47.378847 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:59:47.378815 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:59:47.499730 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:59:47.499685 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:59:53.621115 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:59:53.621035 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:59:53.680116 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:59:53.680081 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:59:53.761831 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:59:53.761796 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:59:53.864961 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:59:53.864924 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:59:53.971817 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:59:53.971719 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 14:59:59.973078 ip-10-0-136-96 kubenswrapper[2568]: I0713 14:59:59.973042 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:00:00.032797 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:00:00.032755 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:01:23.854419 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:01:23.854331 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:01:28.205087 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:01:28.205058 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xgqvx_0bd25f5c-138c-4c60-9d20-525c8b12ebb1/ovn-acl-logging/0.log" Jul 13 15:01:28.206995 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:01:28.206969 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xgqvx_0bd25f5c-138c-4c60-9d20-525c8b12ebb1/ovn-acl-logging/0.log" Jul 13 15:02:04.975310 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:02:04.975262 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:04:46.936819 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:04:46.936742 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:04:46.995323 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:04:46.995286 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:04:58.232163 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:04:58.232126 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:06:28.238774 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:06:28.238746 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xgqvx_0bd25f5c-138c-4c60-9d20-525c8b12ebb1/ovn-acl-logging/0.log" Jul 13 15:06:28.242387 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:06:28.242362 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xgqvx_0bd25f5c-138c-4c60-9d20-525c8b12ebb1/ovn-acl-logging/0.log" Jul 13 15:08:50.368951 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:08:50.368872 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:10:53.694520 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:10:53.694418 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:11:05.293497 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:11:05.293461 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:11:05.356527 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:11:05.356486 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:11:28.270289 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:11:28.270256 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xgqvx_0bd25f5c-138c-4c60-9d20-525c8b12ebb1/ovn-acl-logging/0.log" Jul 13 15:11:28.275309 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:11:28.275288 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xgqvx_0bd25f5c-138c-4c60-9d20-525c8b12ebb1/ovn-acl-logging/0.log" Jul 13 15:11:32.119292 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:11:32.119252 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:11:39.563717 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:11:39.563679 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:11:44.749711 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:11:44.749670 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:11:47.989888 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:11:47.989847 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:11:48.089886 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:11:48.089850 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:11:48.510989 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:11:48.510940 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:11:54.993773 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:11:54.993684 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:11:55.149445 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:11:55.149401 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:11:55.286344 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:11:55.286311 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:11:55.444319 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:11:55.444284 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:11:56.464905 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:11:56.464862 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:11:56.580113 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:11:56.580082 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:11:56.681655 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:11:56.681625 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:11:56.809908 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:11:56.809854 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:11:57.232844 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:11:57.232747 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:11:57.336900 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:11:57.336867 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:11:59.125760 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:11:59.125731 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:01.749418 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:01.749380 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:01.913133 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:01.913092 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:02.023342 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:02.023316 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:02.303057 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:02.302942 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:03.516648 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:03.516564 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:08.815212 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:08.815181 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:08.944966 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:08.944928 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:09.132639 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:09.132546 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:09.427042 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:09.426958 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:10.292397 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:10.292357 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:10.406543 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:10.406506 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:10.583543 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:10.583451 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:10.846625 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:10.846521 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:11.316349 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:11.316316 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:11.413914 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:11.413875 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:12.389752 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:12.389719 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:15.440100 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:15.440063 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:15.904844 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:15.904808 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:21.676092 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:21.676056 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:21.788996 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:21.788952 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:22.995940 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:22.995905 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:27.911680 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:27.911643 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:28.015920 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:28.015873 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:28.092220 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:28.092176 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:29.221277 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:29.221232 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:29.321047 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:29.321012 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:29.455364 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:29.455329 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:29.965153 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:29.965119 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:30.065411 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:30.065364 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:30.165798 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:30.165761 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:31.483246 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:31.483212 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:33.762593 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:33.762538 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:34.010542 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:34.010502 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:34.155304 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:34.154887 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:34.240844 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:34.240805 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:36.058811 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:36.058769 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:40.523503 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:40.523467 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:40.590939 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:40.590904 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:41.073115 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:41.073081 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:41.167020 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:41.166987 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:41.268866 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:41.268836 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:42.864738 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:42.864704 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:48.923163 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:48.923126 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:49.026773 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:49.026738 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:49.124718 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:49.124676 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:49.225109 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:49.225011 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:49.701129 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:49.701097 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:12:49.848285 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:12:49.848252 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:13:16.210988 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:13:16.210950 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:13:22.300826 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:13:22.300785 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:13:22.414148 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:13:22.414115 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:13:22.500369 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:13:22.500229 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:13:38.487399 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:13:38.487357 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:13:44.765429 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:13:44.765388 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:13:44.832617 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:13:44.832563 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:13:44.924972 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:13:44.924930 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:13:45.024927 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:13:45.024890 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:13:55.680777 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:13:55.680738 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:13:58.024196 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:13:58.024159 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:03.102597 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:03.102530 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:03.202475 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:03.202433 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:04.065283 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:04.065243 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:04.174421 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:04.174388 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:04.274145 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:04.274112 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:06.127900 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:06.127854 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:12.166803 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:12.166761 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:12.267235 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:12.267196 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:12.921351 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:12.921317 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:18.952022 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:18.951986 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:19.012023 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:19.011985 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:19.335437 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:19.335397 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:19.713044 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:19.712957 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:19.882517 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:19.882482 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:22.754754 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:22.754666 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:22.849497 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:22.849454 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:22.978513 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:22.978474 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:23.612015 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:23.611976 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:23.712330 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:23.712286 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:23.813734 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:23.813697 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:24.845513 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:24.845479 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:24.945911 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:24.945874 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:25.669030 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:25.668995 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:25.767954 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:25.767900 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:27.316003 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:27.315737 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:28.058715 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:28.058675 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:28.150888 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:28.150850 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:28.542071 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:28.542037 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:35.327097 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:35.327052 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:35.429887 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:35.429848 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:35.816594 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:35.816534 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:36.179244 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:36.179139 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:36.279765 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:36.279731 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:42.907504 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:42.907457 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:43.007676 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:43.007640 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:43.196912 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:43.196830 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:43.437653 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:43.437613 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:43.575970 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:43.575939 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:43.672717 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:43.672683 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:43.809416 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:43.809381 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:49.856312 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:49.856275 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:49.915616 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:49.915499 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:49.998904 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:49.998869 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:50.153692 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:50.153599 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:50.253125 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:50.253090 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:56.574497 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:56.574450 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:14:57.102168 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:14:57.102127 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:15:03.758844 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:03.758808 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:15:04.213804 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:04.213761 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:15:04.586373 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:04.586334 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:15:08.125755 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:08.125716 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:15:08.298505 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:08.298472 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:15:08.456766 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:08.456674 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:15:09.107887 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:09.107847 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:15:09.235733 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:09.235698 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:15:12.573468 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:12.573429 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:15:31.698444 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:31.698409 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh"] Jul 13 15:15:31.701818 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:31.701794 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh" Jul 13 15:15:31.705306 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:31.705276 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-59ce9939\"/\"default-dockercfg-bwppx\"" Jul 13 15:15:31.705425 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:31.705338 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-59ce9939\"/\"e2e-lifecycle-model-59ce9939-kserve-self-signed-certs\"" Jul 13 15:15:31.705425 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:31.705390 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-59ce9939\"/\"kube-root-ca.crt\"" Jul 13 15:15:31.705425 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:31.705418 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-59ce9939\"/\"openshift-service-ca.crt\"" Jul 13 15:15:31.713686 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:31.713664 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh"] Jul 13 15:15:31.733836 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:31.733796 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:15:31.881290 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:31.881258 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/168c452d-d8fb-41dd-a952-bc7c0096d2e5-dshm\") pod \"e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh\" (UID: \"168c452d-d8fb-41dd-a952-bc7c0096d2e5\") " pod="ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh" Jul 13 15:15:31.881451 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:31.881297 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/168c452d-d8fb-41dd-a952-bc7c0096d2e5-tmp-dir\") pod \"e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh\" (UID: \"168c452d-d8fb-41dd-a952-bc7c0096d2e5\") " pod="ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh" Jul 13 15:15:31.881451 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:31.881335 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtv4c\" (UniqueName: \"kubernetes.io/projected/168c452d-d8fb-41dd-a952-bc7c0096d2e5-kube-api-access-gtv4c\") pod \"e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh\" (UID: \"168c452d-d8fb-41dd-a952-bc7c0096d2e5\") " pod="ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh" Jul 13 15:15:31.881451 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:31.881369 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/168c452d-d8fb-41dd-a952-bc7c0096d2e5-model-cache\") pod \"e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh\" (UID: \"168c452d-d8fb-41dd-a952-bc7c0096d2e5\") " pod="ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh" Jul 13 15:15:31.881451 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:31.881405 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/168c452d-d8fb-41dd-a952-bc7c0096d2e5-home\") pod \"e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh\" (UID: \"168c452d-d8fb-41dd-a952-bc7c0096d2e5\") " pod="ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh" Jul 13 15:15:31.881629 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:31.881456 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/168c452d-d8fb-41dd-a952-bc7c0096d2e5-tls-certs\") pod \"e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh\" (UID: \"168c452d-d8fb-41dd-a952-bc7c0096d2e5\") " pod="ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh" Jul 13 15:15:31.982605 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:31.982476 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtv4c\" (UniqueName: \"kubernetes.io/projected/168c452d-d8fb-41dd-a952-bc7c0096d2e5-kube-api-access-gtv4c\") pod \"e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh\" (UID: \"168c452d-d8fb-41dd-a952-bc7c0096d2e5\") " pod="ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh" Jul 13 15:15:31.982605 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:31.982526 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/168c452d-d8fb-41dd-a952-bc7c0096d2e5-model-cache\") pod \"e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh\" (UID: \"168c452d-d8fb-41dd-a952-bc7c0096d2e5\") " pod="ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh" Jul 13 15:15:31.982605 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:31.982562 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/168c452d-d8fb-41dd-a952-bc7c0096d2e5-home\") pod \"e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh\" (UID: \"168c452d-d8fb-41dd-a952-bc7c0096d2e5\") " pod="ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh" Jul 13 15:15:31.982877 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:31.982640 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/168c452d-d8fb-41dd-a952-bc7c0096d2e5-tls-certs\") pod \"e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh\" (UID: \"168c452d-d8fb-41dd-a952-bc7c0096d2e5\") " pod="ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh" Jul 13 15:15:31.982877 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:31.982663 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/168c452d-d8fb-41dd-a952-bc7c0096d2e5-dshm\") pod \"e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh\" (UID: \"168c452d-d8fb-41dd-a952-bc7c0096d2e5\") " pod="ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh" Jul 13 15:15:31.982877 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:31.982685 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/168c452d-d8fb-41dd-a952-bc7c0096d2e5-tmp-dir\") pod \"e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh\" (UID: \"168c452d-d8fb-41dd-a952-bc7c0096d2e5\") " pod="ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh" Jul 13 15:15:31.983041 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:31.983017 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/168c452d-d8fb-41dd-a952-bc7c0096d2e5-model-cache\") pod \"e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh\" (UID: \"168c452d-d8fb-41dd-a952-bc7c0096d2e5\") " pod="ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh" Jul 13 15:15:31.983109 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:31.983077 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/168c452d-d8fb-41dd-a952-bc7c0096d2e5-tmp-dir\") pod \"e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh\" (UID: \"168c452d-d8fb-41dd-a952-bc7c0096d2e5\") " pod="ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh" Jul 13 15:15:31.983163 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:31.983113 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/168c452d-d8fb-41dd-a952-bc7c0096d2e5-home\") pod \"e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh\" (UID: \"168c452d-d8fb-41dd-a952-bc7c0096d2e5\") " pod="ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh" Jul 13 15:15:31.984884 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:31.984868 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/168c452d-d8fb-41dd-a952-bc7c0096d2e5-dshm\") pod \"e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh\" (UID: \"168c452d-d8fb-41dd-a952-bc7c0096d2e5\") " pod="ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh" Jul 13 15:15:31.985220 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:31.985204 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/168c452d-d8fb-41dd-a952-bc7c0096d2e5-tls-certs\") pod \"e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh\" (UID: \"168c452d-d8fb-41dd-a952-bc7c0096d2e5\") " pod="ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh" Jul 13 15:15:31.992479 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:31.992454 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtv4c\" (UniqueName: \"kubernetes.io/projected/168c452d-d8fb-41dd-a952-bc7c0096d2e5-kube-api-access-gtv4c\") pod \"e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh\" (UID: \"168c452d-d8fb-41dd-a952-bc7c0096d2e5\") " pod="ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh" Jul 13 15:15:32.011472 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:32.011442 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh" Jul 13 15:15:32.146390 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:32.146368 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh"] Jul 13 15:15:32.148614 ip-10-0-136-96 kubenswrapper[2568]: W0713 15:15:32.148581 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod168c452d_d8fb_41dd_a952_bc7c0096d2e5.slice/crio-3f2938208cc828bcf49043328ffaf1647996b725b5886fe776c602a81cbfdc70 WatchSource:0}: Error finding container 3f2938208cc828bcf49043328ffaf1647996b725b5886fe776c602a81cbfdc70: Status 404 returned error can't find the container with id 3f2938208cc828bcf49043328ffaf1647996b725b5886fe776c602a81cbfdc70 Jul 13 15:15:33.034214 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:33.034177 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh" event={"ID":"168c452d-d8fb-41dd-a952-bc7c0096d2e5","Type":"ContainerStarted","Data":"5ed52d71ce3384204ba9c76174a4976a98a9a0443429ddd6de85f236e1b1612c"} Jul 13 15:15:33.034626 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:33.034224 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh" event={"ID":"168c452d-d8fb-41dd-a952-bc7c0096d2e5","Type":"ContainerStarted","Data":"3f2938208cc828bcf49043328ffaf1647996b725b5886fe776c602a81cbfdc70"} Jul 13 15:15:33.054852 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:33.054804 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh" podStartSLOduration=2.054787076 podStartE2EDuration="2.054787076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 15:15:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 15:15:33.053017732 +0000 UTC m=+1745.293295581" watchObservedRunningTime="2026-07-13 15:15:33.054787076 +0000 UTC m=+1745.295064925" Jul 13 15:15:42.012253 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:42.012215 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh" Jul 13 15:15:42.012706 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:42.012291 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh" Jul 13 15:15:42.025100 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:42.025076 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh" Jul 13 15:15:42.091805 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:42.091770 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh" Jul 13 15:15:50.403482 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:50.403434 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:15:50.813310 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:50.813273 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:15:56.525181 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:15:56.525136 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:16:01.506812 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:01.506772 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh"] Jul 13 15:16:01.631164 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:01.631129 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:16:16.526695 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:16.526637 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh" podUID="168c452d-d8fb-41dd-a952-bc7c0096d2e5" containerName="main" containerID="cri-o://5ed52d71ce3384204ba9c76174a4976a98a9a0443429ddd6de85f236e1b1612c" gracePeriod=45 Jul 13 15:16:16.775214 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:16.775186 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh" Jul 13 15:16:16.879949 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:16.879864 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/168c452d-d8fb-41dd-a952-bc7c0096d2e5-model-cache\") pod \"168c452d-d8fb-41dd-a952-bc7c0096d2e5\" (UID: \"168c452d-d8fb-41dd-a952-bc7c0096d2e5\") " Jul 13 15:16:16.879949 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:16.879905 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtv4c\" (UniqueName: \"kubernetes.io/projected/168c452d-d8fb-41dd-a952-bc7c0096d2e5-kube-api-access-gtv4c\") pod \"168c452d-d8fb-41dd-a952-bc7c0096d2e5\" (UID: \"168c452d-d8fb-41dd-a952-bc7c0096d2e5\") " Jul 13 15:16:16.879949 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:16.879937 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/168c452d-d8fb-41dd-a952-bc7c0096d2e5-home\") pod \"168c452d-d8fb-41dd-a952-bc7c0096d2e5\" (UID: \"168c452d-d8fb-41dd-a952-bc7c0096d2e5\") " Jul 13 15:16:16.880193 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:16.879974 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/168c452d-d8fb-41dd-a952-bc7c0096d2e5-dshm\") pod \"168c452d-d8fb-41dd-a952-bc7c0096d2e5\" (UID: \"168c452d-d8fb-41dd-a952-bc7c0096d2e5\") " Jul 13 15:16:16.880193 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:16.880033 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/168c452d-d8fb-41dd-a952-bc7c0096d2e5-tls-certs\") pod \"168c452d-d8fb-41dd-a952-bc7c0096d2e5\" (UID: \"168c452d-d8fb-41dd-a952-bc7c0096d2e5\") " Jul 13 15:16:16.880193 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:16.880072 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/168c452d-d8fb-41dd-a952-bc7c0096d2e5-tmp-dir\") pod \"168c452d-d8fb-41dd-a952-bc7c0096d2e5\" (UID: \"168c452d-d8fb-41dd-a952-bc7c0096d2e5\") " Jul 13 15:16:16.880193 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:16.880178 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/168c452d-d8fb-41dd-a952-bc7c0096d2e5-model-cache" (OuterVolumeSpecName: "model-cache") pod "168c452d-d8fb-41dd-a952-bc7c0096d2e5" (UID: "168c452d-d8fb-41dd-a952-bc7c0096d2e5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:16:16.880401 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:16.880232 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/168c452d-d8fb-41dd-a952-bc7c0096d2e5-home" (OuterVolumeSpecName: "home") pod "168c452d-d8fb-41dd-a952-bc7c0096d2e5" (UID: "168c452d-d8fb-41dd-a952-bc7c0096d2e5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:16:16.880460 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:16.880405 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/168c452d-d8fb-41dd-a952-bc7c0096d2e5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "168c452d-d8fb-41dd-a952-bc7c0096d2e5" (UID: "168c452d-d8fb-41dd-a952-bc7c0096d2e5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:16:16.880460 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:16.880442 2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/168c452d-d8fb-41dd-a952-bc7c0096d2e5-model-cache\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:16:16.880535 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:16.880461 2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/168c452d-d8fb-41dd-a952-bc7c0096d2e5-home\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:16:16.882195 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:16.882167 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/168c452d-d8fb-41dd-a952-bc7c0096d2e5-kube-api-access-gtv4c" (OuterVolumeSpecName: "kube-api-access-gtv4c") pod "168c452d-d8fb-41dd-a952-bc7c0096d2e5" (UID: "168c452d-d8fb-41dd-a952-bc7c0096d2e5"). InnerVolumeSpecName "kube-api-access-gtv4c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 15:16:16.882195 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:16.882176 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/168c452d-d8fb-41dd-a952-bc7c0096d2e5-dshm" (OuterVolumeSpecName: "dshm") pod "168c452d-d8fb-41dd-a952-bc7c0096d2e5" (UID: "168c452d-d8fb-41dd-a952-bc7c0096d2e5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:16:16.882355 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:16.882167 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/168c452d-d8fb-41dd-a952-bc7c0096d2e5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "168c452d-d8fb-41dd-a952-bc7c0096d2e5" (UID: "168c452d-d8fb-41dd-a952-bc7c0096d2e5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 15:16:16.981724 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:16.981671 2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/168c452d-d8fb-41dd-a952-bc7c0096d2e5-tls-certs\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:16:16.981724 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:16.981719 2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/168c452d-d8fb-41dd-a952-bc7c0096d2e5-tmp-dir\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:16:16.981724 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:16.981729 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtv4c\" (UniqueName: \"kubernetes.io/projected/168c452d-d8fb-41dd-a952-bc7c0096d2e5-kube-api-access-gtv4c\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:16:16.981724 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:16.981738 2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/168c452d-d8fb-41dd-a952-bc7c0096d2e5-dshm\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:16:17.268047 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:17.267965 2568 generic.go:356] "Generic (PLEG): container finished" podID="168c452d-d8fb-41dd-a952-bc7c0096d2e5" containerID="5ed52d71ce3384204ba9c76174a4976a98a9a0443429ddd6de85f236e1b1612c" exitCode=0 Jul 13 15:16:17.268047 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:17.268003 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh" Jul 13 15:16:17.268047 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:17.268039 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh" event={"ID":"168c452d-d8fb-41dd-a952-bc7c0096d2e5","Type":"ContainerDied","Data":"5ed52d71ce3384204ba9c76174a4976a98a9a0443429ddd6de85f236e1b1612c"} Jul 13 15:16:17.268276 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:17.268071 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh" event={"ID":"168c452d-d8fb-41dd-a952-bc7c0096d2e5","Type":"ContainerDied","Data":"3f2938208cc828bcf49043328ffaf1647996b725b5886fe776c602a81cbfdc70"} Jul 13 15:16:17.268276 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:17.268088 2568 scope.go:117] "RemoveContainer" containerID="5ed52d71ce3384204ba9c76174a4976a98a9a0443429ddd6de85f236e1b1612c" Jul 13 15:16:17.277443 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:17.277424 2568 scope.go:117] "RemoveContainer" containerID="5ed52d71ce3384204ba9c76174a4976a98a9a0443429ddd6de85f236e1b1612c" Jul 13 15:16:17.277730 ip-10-0-136-96 kubenswrapper[2568]: E0713 15:16:17.277710 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ed52d71ce3384204ba9c76174a4976a98a9a0443429ddd6de85f236e1b1612c\": container with ID starting with 5ed52d71ce3384204ba9c76174a4976a98a9a0443429ddd6de85f236e1b1612c not found: ID does not exist" containerID="5ed52d71ce3384204ba9c76174a4976a98a9a0443429ddd6de85f236e1b1612c" Jul 13 15:16:17.277779 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:17.277738 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ed52d71ce3384204ba9c76174a4976a98a9a0443429ddd6de85f236e1b1612c"} err="failed to get container status \"5ed52d71ce3384204ba9c76174a4976a98a9a0443429ddd6de85f236e1b1612c\": rpc error: code = NotFound desc = could not find container \"5ed52d71ce3384204ba9c76174a4976a98a9a0443429ddd6de85f236e1b1612c\": container with ID starting with 5ed52d71ce3384204ba9c76174a4976a98a9a0443429ddd6de85f236e1b1612c not found: ID does not exist" Jul 13 15:16:17.292056 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:17.292029 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh"] Jul 13 15:16:17.296082 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:17.296061 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-59ce9939/e2e-lifecycle-model-59ce9939-kserve-55595fdf6c-4gkkh"] Jul 13 15:16:18.272183 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:18.272146 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="168c452d-d8fb-41dd-a952-bc7c0096d2e5" path="/var/lib/kubelet/pods/168c452d-d8fb-41dd-a952-bc7c0096d2e5/volumes" Jul 13 15:16:28.307387 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:28.307360 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xgqvx_0bd25f5c-138c-4c60-9d20-525c8b12ebb1/ovn-acl-logging/0.log" Jul 13 15:16:28.314934 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:28.314913 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xgqvx_0bd25f5c-138c-4c60-9d20-525c8b12ebb1/ovn-acl-logging/0.log" Jul 13 15:16:38.415531 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:38.415500 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:16:39.110432 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:39.110398 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:16:45.007995 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:45.007954 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:16:51.151696 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:51.151657 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:16:51.277485 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:51.277456 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:16:51.493523 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:51.493431 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:16:51.701342 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:51.701307 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:16:52.912520 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:52.912476 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:16:53.039647 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:53.039605 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:16:53.868803 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:53.868763 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:16:53.963118 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:53.963085 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:16:54.076334 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:54.076300 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:16:55.801038 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:16:55.800998 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:17:02.481280 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:02.481240 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:17:02.698492 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:02.698456 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:17:03.191417 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:03.191383 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:17:10.314940 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:10.314901 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:17:10.544297 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:10.544264 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:17:10.776763 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:10.776728 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:17:12.875155 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:12.875122 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:17:12.982859 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:12.982812 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:17:21.185899 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:21.185858 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:17:21.612795 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:21.612738 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:17:42.588627 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:42.588552 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:17:49.234607 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:49.234523 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:17:49.410691 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:49.410660 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:17:49.649586 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:49.649535 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:17:49.961526 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:49.961393 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:17:50.216184 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:50.216086 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:17:52.805782 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:52.805742 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:17:52.937777 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:52.937746 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:17:53.081614 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:53.081502 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:17:53.507104 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:53.507012 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:17:53.608947 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:53.608909 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:17:57.210452 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:57.210413 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:17:57.592097 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:57.592060 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:17:57.906027 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:57.905910 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx"] Jul 13 15:17:57.906263 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:57.906249 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="168c452d-d8fb-41dd-a952-bc7c0096d2e5" containerName="main" Jul 13 15:17:57.906309 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:57.906265 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="168c452d-d8fb-41dd-a952-bc7c0096d2e5" containerName="main" Jul 13 15:17:57.906344 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:57.906320 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="168c452d-d8fb-41dd-a952-bc7c0096d2e5" containerName="main" Jul 13 15:17:57.909297 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:57.909275 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:17:57.911984 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:57.911963 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-3dcb39-openshift-default-dockercfg-vggg6\"" Jul 13 15:17:57.946412 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:57.946375 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx"] Jul 13 15:17:57.965289 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:57.965250 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:17:58.045976 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.045932 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/894b568d-1704-4593-8af8-c1507e735dee-credential-socket\") pod \"e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:17:58.046169 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.045988 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhqr9\" (UniqueName: \"kubernetes.io/projected/894b568d-1704-4593-8af8-c1507e735dee-kube-api-access-zhqr9\") pod \"e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:17:58.046169 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.046109 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/894b568d-1704-4593-8af8-c1507e735dee-istiod-ca-cert\") pod \"e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:17:58.046169 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.046155 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/894b568d-1704-4593-8af8-c1507e735dee-istio-token\") pod \"e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:17:58.046303 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.046186 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/894b568d-1704-4593-8af8-c1507e735dee-istio-envoy\") pod \"e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:17:58.046303 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.046204 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/894b568d-1704-4593-8af8-c1507e735dee-istio-podinfo\") pod \"e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:17:58.046303 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.046227 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/894b568d-1704-4593-8af8-c1507e735dee-workload-socket\") pod \"e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:17:58.046303 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.046245 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/894b568d-1704-4593-8af8-c1507e735dee-istio-data\") pod \"e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:17:58.046303 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.046279 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/894b568d-1704-4593-8af8-c1507e735dee-workload-certs\") pod \"e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:17:58.147468 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.147431 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/894b568d-1704-4593-8af8-c1507e735dee-istio-token\") pod \"e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:17:58.147468 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.147471 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/894b568d-1704-4593-8af8-c1507e735dee-istio-envoy\") pod \"e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:17:58.147779 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.147487 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/894b568d-1704-4593-8af8-c1507e735dee-istio-podinfo\") pod \"e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:17:58.147779 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.147510 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/894b568d-1704-4593-8af8-c1507e735dee-workload-socket\") pod \"e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:17:58.147779 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.147527 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/894b568d-1704-4593-8af8-c1507e735dee-istio-data\") pod \"e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:17:58.147779 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.147547 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/894b568d-1704-4593-8af8-c1507e735dee-workload-certs\") pod \"e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:17:58.147779 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.147606 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/894b568d-1704-4593-8af8-c1507e735dee-credential-socket\") pod \"e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:17:58.147779 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.147631 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhqr9\" (UniqueName: \"kubernetes.io/projected/894b568d-1704-4593-8af8-c1507e735dee-kube-api-access-zhqr9\") pod \"e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:17:58.147779 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.147708 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/894b568d-1704-4593-8af8-c1507e735dee-istiod-ca-cert\") pod \"e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:17:58.148131 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.148090 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/894b568d-1704-4593-8af8-c1507e735dee-workload-socket\") pod \"e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:17:58.148131 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.148108 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/894b568d-1704-4593-8af8-c1507e735dee-istio-data\") pod \"e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:17:58.148246 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.148167 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/894b568d-1704-4593-8af8-c1507e735dee-workload-certs\") pod \"e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:17:58.148246 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.148187 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/894b568d-1704-4593-8af8-c1507e735dee-credential-socket\") pod \"e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:17:58.148594 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.148548 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/894b568d-1704-4593-8af8-c1507e735dee-istiod-ca-cert\") pod \"e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:17:58.149854 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.149834 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/894b568d-1704-4593-8af8-c1507e735dee-istio-envoy\") pod \"e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:17:58.150132 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.150114 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/894b568d-1704-4593-8af8-c1507e735dee-istio-podinfo\") pod \"e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:17:58.158767 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.158705 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/894b568d-1704-4593-8af8-c1507e735dee-istio-token\") pod \"e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:17:58.158942 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.158922 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhqr9\" (UniqueName: \"kubernetes.io/projected/894b568d-1704-4593-8af8-c1507e735dee-kube-api-access-zhqr9\") pod \"e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:17:58.219511 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.219476 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:17:58.360509 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.360476 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx"] Jul 13 15:17:58.362917 ip-10-0-136-96 kubenswrapper[2568]: W0713 15:17:58.362888 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod894b568d_1704_4593_8af8_c1507e735dee.slice/crio-f0f9cc0d2f0d9952c2f8b805f68e653dc56d9d42b75be45feca78fce83ca1d8a WatchSource:0}: Error finding container f0f9cc0d2f0d9952c2f8b805f68e653dc56d9d42b75be45feca78fce83ca1d8a: Status 404 returned error can't find the container with id f0f9cc0d2f0d9952c2f8b805f68e653dc56d9d42b75be45feca78fce83ca1d8a Jul 13 15:17:58.364989 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.364960 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 15:17:58.365049 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.365026 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 15:17:58.365087 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.365052 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 15:17:58.467790 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.467722 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:17:58.772428 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.772337 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" event={"ID":"894b568d-1704-4593-8af8-c1507e735dee","Type":"ContainerStarted","Data":"fa916cd524c7ce19e31f49a867d689592938c5eecd0a3444cba79741dafcc85b"} Jul 13 15:17:58.772428 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.772418 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" event={"ID":"894b568d-1704-4593-8af8-c1507e735dee","Type":"ContainerStarted","Data":"f0f9cc0d2f0d9952c2f8b805f68e653dc56d9d42b75be45feca78fce83ca1d8a"} Jul 13 15:17:58.803249 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:58.803200 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" podStartSLOduration=1.803184334 podStartE2EDuration="1.803184334s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 15:17:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 15:17:58.800656147 +0000 UTC m=+1891.040934018" watchObservedRunningTime="2026-07-13 15:17:58.803184334 +0000 UTC m=+1891.043462181" Jul 13 15:17:59.219749 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:59.219714 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:17:59.224154 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:59.224123 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:17:59.777274 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:59.777244 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:17:59.778201 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:17:59.778177 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:18:00.818006 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:00.817940 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:18:00.921762 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:00.921720 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:18:01.026396 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:01.026360 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:18:01.294621 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:01.294563 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:18:01.519815 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:01.519781 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:18:01.622415 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:01.622302 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:18:05.382739 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:05.382701 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:18:14.529045 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:14.529004 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v"] Jul 13 15:18:14.532242 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:14.532219 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v" Jul 13 15:18:14.537084 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:14.537058 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-d5c92a\"/\"kube-root-ca.crt\"" Jul 13 15:18:14.537209 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:14.537070 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-d5c92a\"/\"openshift-service-ca.crt\"" Jul 13 15:18:14.538406 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:14.538384 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-d5c92a\"/\"test-model-d5c92a-kserve-self-signed-certs\"" Jul 13 15:18:14.539334 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:14.539177 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-d5c92a\"/\"default-dockercfg-w7kfq\"" Jul 13 15:18:14.550695 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:14.550661 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v"] Jul 13 15:18:14.626625 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:14.626560 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:18:14.703431 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:14.703396 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b2f35a1-06aa-4494-bbfe-850970365063-tmp-dir\") pod \"test-model-d5c92a-kserve-794485d8f-bh45v\" (UID: \"7b2f35a1-06aa-4494-bbfe-850970365063\") " pod="ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v" Jul 13 15:18:14.703431 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:14.703440 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b2f35a1-06aa-4494-bbfe-850970365063-dshm\") pod \"test-model-d5c92a-kserve-794485d8f-bh45v\" (UID: \"7b2f35a1-06aa-4494-bbfe-850970365063\") " pod="ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v" Jul 13 15:18:14.703698 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:14.703606 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b2f35a1-06aa-4494-bbfe-850970365063-tls-certs\") pod \"test-model-d5c92a-kserve-794485d8f-bh45v\" (UID: \"7b2f35a1-06aa-4494-bbfe-850970365063\") " pod="ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v" Jul 13 15:18:14.703698 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:14.703667 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b2f35a1-06aa-4494-bbfe-850970365063-model-cache\") pod \"test-model-d5c92a-kserve-794485d8f-bh45v\" (UID: \"7b2f35a1-06aa-4494-bbfe-850970365063\") " pod="ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v" Jul 13 15:18:14.703819 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:14.703716 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b2f35a1-06aa-4494-bbfe-850970365063-home\") pod \"test-model-d5c92a-kserve-794485d8f-bh45v\" (UID: \"7b2f35a1-06aa-4494-bbfe-850970365063\") " pod="ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v" Jul 13 15:18:14.703819 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:14.703740 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gl4gl\" (UniqueName: \"kubernetes.io/projected/7b2f35a1-06aa-4494-bbfe-850970365063-kube-api-access-gl4gl\") pod \"test-model-d5c92a-kserve-794485d8f-bh45v\" (UID: \"7b2f35a1-06aa-4494-bbfe-850970365063\") " pod="ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v" Jul 13 15:18:14.805077 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:14.804977 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b2f35a1-06aa-4494-bbfe-850970365063-tls-certs\") pod \"test-model-d5c92a-kserve-794485d8f-bh45v\" (UID: \"7b2f35a1-06aa-4494-bbfe-850970365063\") " pod="ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v" Jul 13 15:18:14.805077 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:14.805047 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b2f35a1-06aa-4494-bbfe-850970365063-model-cache\") pod \"test-model-d5c92a-kserve-794485d8f-bh45v\" (UID: \"7b2f35a1-06aa-4494-bbfe-850970365063\") " pod="ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v" Jul 13 15:18:14.805304 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:14.805104 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b2f35a1-06aa-4494-bbfe-850970365063-home\") pod \"test-model-d5c92a-kserve-794485d8f-bh45v\" (UID: \"7b2f35a1-06aa-4494-bbfe-850970365063\") " pod="ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v" Jul 13 15:18:14.805304 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:14.805130 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gl4gl\" (UniqueName: \"kubernetes.io/projected/7b2f35a1-06aa-4494-bbfe-850970365063-kube-api-access-gl4gl\") pod \"test-model-d5c92a-kserve-794485d8f-bh45v\" (UID: \"7b2f35a1-06aa-4494-bbfe-850970365063\") " pod="ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v" Jul 13 15:18:14.805304 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:14.805185 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b2f35a1-06aa-4494-bbfe-850970365063-tmp-dir\") pod \"test-model-d5c92a-kserve-794485d8f-bh45v\" (UID: \"7b2f35a1-06aa-4494-bbfe-850970365063\") " pod="ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v" Jul 13 15:18:14.805443 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:14.805341 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b2f35a1-06aa-4494-bbfe-850970365063-dshm\") pod \"test-model-d5c92a-kserve-794485d8f-bh45v\" (UID: \"7b2f35a1-06aa-4494-bbfe-850970365063\") " pod="ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v" Jul 13 15:18:14.805511 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:14.805490 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b2f35a1-06aa-4494-bbfe-850970365063-model-cache\") pod \"test-model-d5c92a-kserve-794485d8f-bh45v\" (UID: \"7b2f35a1-06aa-4494-bbfe-850970365063\") " pod="ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v" Jul 13 15:18:14.805593 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:14.805516 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b2f35a1-06aa-4494-bbfe-850970365063-home\") pod \"test-model-d5c92a-kserve-794485d8f-bh45v\" (UID: \"7b2f35a1-06aa-4494-bbfe-850970365063\") " pod="ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v" Jul 13 15:18:14.805638 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:14.805563 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b2f35a1-06aa-4494-bbfe-850970365063-tmp-dir\") pod \"test-model-d5c92a-kserve-794485d8f-bh45v\" (UID: \"7b2f35a1-06aa-4494-bbfe-850970365063\") " pod="ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v" Jul 13 15:18:14.807519 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:14.807462 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b2f35a1-06aa-4494-bbfe-850970365063-dshm\") pod \"test-model-d5c92a-kserve-794485d8f-bh45v\" (UID: \"7b2f35a1-06aa-4494-bbfe-850970365063\") " pod="ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v" Jul 13 15:18:14.807715 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:14.807700 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b2f35a1-06aa-4494-bbfe-850970365063-tls-certs\") pod \"test-model-d5c92a-kserve-794485d8f-bh45v\" (UID: \"7b2f35a1-06aa-4494-bbfe-850970365063\") " pod="ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v" Jul 13 15:18:14.815385 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:14.815349 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gl4gl\" (UniqueName: \"kubernetes.io/projected/7b2f35a1-06aa-4494-bbfe-850970365063-kube-api-access-gl4gl\") pod \"test-model-d5c92a-kserve-794485d8f-bh45v\" (UID: \"7b2f35a1-06aa-4494-bbfe-850970365063\") " pod="ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v" Jul 13 15:18:14.841885 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:14.841848 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v" Jul 13 15:18:14.986715 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:14.986690 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v"] Jul 13 15:18:14.989241 ip-10-0-136-96 kubenswrapper[2568]: W0713 15:18:14.989213 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7b2f35a1_06aa_4494_bbfe_850970365063.slice/crio-daa6108a5744455e924fe20ef4ca4ba04a8cee466c4c67d8b445f69c36dd24e8 WatchSource:0}: Error finding container daa6108a5744455e924fe20ef4ca4ba04a8cee466c4c67d8b445f69c36dd24e8: Status 404 returned error can't find the container with id daa6108a5744455e924fe20ef4ca4ba04a8cee466c4c67d8b445f69c36dd24e8 Jul 13 15:18:15.819735 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:15.819697 2568 patch_prober.go:28] interesting pod/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 13 15:18:15.821067 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:15.819758 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" podUID="894b568d-1704-4593-8af8-c1507e735dee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 13 15:18:15.867731 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:15.867692 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v" event={"ID":"7b2f35a1-06aa-4494-bbfe-850970365063","Type":"ContainerStarted","Data":"de6c37a6870cafeefb2f18bba1a004143db31686b98cc35993ff60c5b288f582"} Jul 13 15:18:15.867731 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:15.867733 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v" event={"ID":"7b2f35a1-06aa-4494-bbfe-850970365063","Type":"ContainerStarted","Data":"daa6108a5744455e924fe20ef4ca4ba04a8cee466c4c67d8b445f69c36dd24e8"} Jul 13 15:18:15.887961 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:15.887904 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v" podStartSLOduration=1.887888504 podStartE2EDuration="1.887888504s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 15:18:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 15:18:15.886347178 +0000 UTC m=+1908.126625064" watchObservedRunningTime="2026-07-13 15:18:15.887888504 +0000 UTC m=+1908.128166369" Jul 13 15:18:16.051050 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:16.050958 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:18:16.150381 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:16.150286 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:18:16.874268 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:16.874232 2568 generic.go:356] "Generic (PLEG): container finished" podID="894b568d-1704-4593-8af8-c1507e735dee" containerID="fa916cd524c7ce19e31f49a867d689592938c5eecd0a3444cba79741dafcc85b" exitCode=0 Jul 13 15:18:16.874741 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:16.874305 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" event={"ID":"894b568d-1704-4593-8af8-c1507e735dee","Type":"ContainerDied","Data":"fa916cd524c7ce19e31f49a867d689592938c5eecd0a3444cba79741dafcc85b"} Jul 13 15:18:16.874858 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:16.874842 2568 scope.go:117] "RemoveContainer" containerID="fa916cd524c7ce19e31f49a867d689592938c5eecd0a3444cba79741dafcc85b" Jul 13 15:18:16.875806 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:16.875778 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 15:18:16.875899 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:16.875851 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 15:18:16.875899 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:16.875887 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 15:18:17.881537 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:17.881503 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" event={"ID":"894b568d-1704-4593-8af8-c1507e735dee","Type":"ContainerStarted","Data":"f6e7165336e0421f95c3038bce51fd5a81cf911df04fe9ebd36e91fc45134e13"} Jul 13 15:18:18.220332 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:18.220225 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:18:18.220500 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:18.220441 2568 patch_prober.go:28] interesting pod/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 13 15:18:18.220547 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:18.220510 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" podUID="894b568d-1704-4593-8af8-c1507e735dee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 13 15:18:19.220042 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:19.220000 2568 patch_prober.go:28] interesting pod/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 13 15:18:19.220445 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:19.220062 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" podUID="894b568d-1704-4593-8af8-c1507e735dee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 13 15:18:20.220070 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:20.220032 2568 patch_prober.go:28] interesting pod/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 13 15:18:20.220478 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:20.220088 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" podUID="894b568d-1704-4593-8af8-c1507e735dee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 13 15:18:21.220427 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:21.220387 2568 patch_prober.go:28] interesting pod/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 13 15:18:21.221123 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:21.221086 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" podUID="894b568d-1704-4593-8af8-c1507e735dee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 13 15:18:22.219979 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:22.219940 2568 patch_prober.go:28] interesting pod/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 13 15:18:22.220292 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:22.220257 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" podUID="894b568d-1704-4593-8af8-c1507e735dee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 13 15:18:22.382862 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:22.382818 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:18:22.484747 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:22.484650 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:18:23.220798 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:23.220752 2568 patch_prober.go:28] interesting pod/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 13 15:18:23.221030 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:23.220839 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" podUID="894b568d-1704-4593-8af8-c1507e735dee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 13 15:18:24.220279 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:24.220235 2568 patch_prober.go:28] interesting pod/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 13 15:18:24.221029 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:24.220996 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" podUID="894b568d-1704-4593-8af8-c1507e735dee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 13 15:18:24.843006 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:24.842956 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v" Jul 13 15:18:24.843193 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:24.843023 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v" Jul 13 15:18:24.855771 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:24.855737 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v" Jul 13 15:18:24.945281 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:24.945242 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v" Jul 13 15:18:25.220647 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:25.220521 2568 patch_prober.go:28] interesting pod/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 13 15:18:25.221279 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:25.221241 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" podUID="894b568d-1704-4593-8af8-c1507e735dee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 13 15:18:26.224514 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:26.224485 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:18:26.224922 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:26.224865 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:18:26.225661 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:26.225644 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:18:28.800228 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:28.800195 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k"] Jul 13 15:18:28.802555 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:28.802538 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k" Jul 13 15:18:28.806085 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:28.806063 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-3dcb39\"/\"default-dockercfg-ktczg\"" Jul 13 15:18:28.807104 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:28.807083 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-3dcb39\"/\"test-model-3dcb39-kserve-self-signed-certs\"" Jul 13 15:18:28.807230 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:28.807092 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-3dcb39\"/\"openshift-service-ca.crt\"" Jul 13 15:18:28.807230 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:28.807106 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-3dcb39\"/\"kube-root-ca.crt\"" Jul 13 15:18:28.816262 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:28.816235 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k"] Jul 13 15:18:28.834458 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:28.834410 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c05396ca-6ed2-467e-a3fd-87a295191d92-model-cache\") pod \"test-model-3dcb39-kserve-65894b6bcc-jcs6k\" (UID: \"c05396ca-6ed2-467e-a3fd-87a295191d92\") " pod="ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k" Jul 13 15:18:28.834654 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:28.834489 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c05396ca-6ed2-467e-a3fd-87a295191d92-home\") pod \"test-model-3dcb39-kserve-65894b6bcc-jcs6k\" (UID: \"c05396ca-6ed2-467e-a3fd-87a295191d92\") " pod="ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k" Jul 13 15:18:28.834654 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:28.834517 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c05396ca-6ed2-467e-a3fd-87a295191d92-tls-certs\") pod \"test-model-3dcb39-kserve-65894b6bcc-jcs6k\" (UID: \"c05396ca-6ed2-467e-a3fd-87a295191d92\") " pod="ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k" Jul 13 15:18:28.834654 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:28.834537 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c05396ca-6ed2-467e-a3fd-87a295191d92-tmp-dir\") pod \"test-model-3dcb39-kserve-65894b6bcc-jcs6k\" (UID: \"c05396ca-6ed2-467e-a3fd-87a295191d92\") " pod="ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k" Jul 13 15:18:28.834654 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:28.834626 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c05396ca-6ed2-467e-a3fd-87a295191d92-dshm\") pod \"test-model-3dcb39-kserve-65894b6bcc-jcs6k\" (UID: \"c05396ca-6ed2-467e-a3fd-87a295191d92\") " pod="ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k" Jul 13 15:18:28.834881 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:28.834678 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flmqx\" (UniqueName: \"kubernetes.io/projected/c05396ca-6ed2-467e-a3fd-87a295191d92-kube-api-access-flmqx\") pod \"test-model-3dcb39-kserve-65894b6bcc-jcs6k\" (UID: \"c05396ca-6ed2-467e-a3fd-87a295191d92\") " pod="ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k" Jul 13 15:18:28.835392 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:28.835357 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:18:28.935333 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:28.935298 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c05396ca-6ed2-467e-a3fd-87a295191d92-home\") pod \"test-model-3dcb39-kserve-65894b6bcc-jcs6k\" (UID: \"c05396ca-6ed2-467e-a3fd-87a295191d92\") " pod="ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k" Jul 13 15:18:28.935333 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:28.935337 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c05396ca-6ed2-467e-a3fd-87a295191d92-tls-certs\") pod \"test-model-3dcb39-kserve-65894b6bcc-jcs6k\" (UID: \"c05396ca-6ed2-467e-a3fd-87a295191d92\") " pod="ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k" Jul 13 15:18:28.935613 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:28.935359 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c05396ca-6ed2-467e-a3fd-87a295191d92-tmp-dir\") pod \"test-model-3dcb39-kserve-65894b6bcc-jcs6k\" (UID: \"c05396ca-6ed2-467e-a3fd-87a295191d92\") " pod="ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k" Jul 13 15:18:28.935613 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:28.935409 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c05396ca-6ed2-467e-a3fd-87a295191d92-dshm\") pod \"test-model-3dcb39-kserve-65894b6bcc-jcs6k\" (UID: \"c05396ca-6ed2-467e-a3fd-87a295191d92\") " pod="ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k" Jul 13 15:18:28.935613 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:28.935456 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flmqx\" (UniqueName: \"kubernetes.io/projected/c05396ca-6ed2-467e-a3fd-87a295191d92-kube-api-access-flmqx\") pod \"test-model-3dcb39-kserve-65894b6bcc-jcs6k\" (UID: \"c05396ca-6ed2-467e-a3fd-87a295191d92\") " pod="ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k" Jul 13 15:18:28.935613 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:28.935496 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c05396ca-6ed2-467e-a3fd-87a295191d92-model-cache\") pod \"test-model-3dcb39-kserve-65894b6bcc-jcs6k\" (UID: \"c05396ca-6ed2-467e-a3fd-87a295191d92\") " pod="ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k" Jul 13 15:18:28.935826 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:28.935763 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c05396ca-6ed2-467e-a3fd-87a295191d92-home\") pod \"test-model-3dcb39-kserve-65894b6bcc-jcs6k\" (UID: \"c05396ca-6ed2-467e-a3fd-87a295191d92\") " pod="ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k" Jul 13 15:18:28.935826 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:28.935781 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c05396ca-6ed2-467e-a3fd-87a295191d92-tmp-dir\") pod \"test-model-3dcb39-kserve-65894b6bcc-jcs6k\" (UID: \"c05396ca-6ed2-467e-a3fd-87a295191d92\") " pod="ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k" Jul 13 15:18:28.935920 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:28.935821 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c05396ca-6ed2-467e-a3fd-87a295191d92-model-cache\") pod \"test-model-3dcb39-kserve-65894b6bcc-jcs6k\" (UID: \"c05396ca-6ed2-467e-a3fd-87a295191d92\") " pod="ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k" Jul 13 15:18:28.937741 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:28.937709 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c05396ca-6ed2-467e-a3fd-87a295191d92-dshm\") pod \"test-model-3dcb39-kserve-65894b6bcc-jcs6k\" (UID: \"c05396ca-6ed2-467e-a3fd-87a295191d92\") " pod="ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k" Jul 13 15:18:28.938099 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:28.938080 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c05396ca-6ed2-467e-a3fd-87a295191d92-tls-certs\") pod \"test-model-3dcb39-kserve-65894b6bcc-jcs6k\" (UID: \"c05396ca-6ed2-467e-a3fd-87a295191d92\") " pod="ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k" Jul 13 15:18:28.945158 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:28.945134 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flmqx\" (UniqueName: \"kubernetes.io/projected/c05396ca-6ed2-467e-a3fd-87a295191d92-kube-api-access-flmqx\") pod \"test-model-3dcb39-kserve-65894b6bcc-jcs6k\" (UID: \"c05396ca-6ed2-467e-a3fd-87a295191d92\") " pod="ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k" Jul 13 15:18:29.111544 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:29.111450 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k" Jul 13 15:18:29.256195 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:29.256168 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k"] Jul 13 15:18:29.257012 ip-10-0-136-96 kubenswrapper[2568]: W0713 15:18:29.256980 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc05396ca_6ed2_467e_a3fd_87a295191d92.slice/crio-1f54927cd5b15f4da1f359accddd1664500f78b6385b2ba96d973aa2b407cb48 WatchSource:0}: Error finding container 1f54927cd5b15f4da1f359accddd1664500f78b6385b2ba96d973aa2b407cb48: Status 404 returned error can't find the container with id 1f54927cd5b15f4da1f359accddd1664500f78b6385b2ba96d973aa2b407cb48 Jul 13 15:18:29.949792 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:29.949713 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k" event={"ID":"c05396ca-6ed2-467e-a3fd-87a295191d92","Type":"ContainerStarted","Data":"b1179260cf16029d409c865e01ee3d663e6d5fbd0dfd5a3545042825f97d004e"} Jul 13 15:18:29.949792 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:29.949754 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k" event={"ID":"c05396ca-6ed2-467e-a3fd-87a295191d92","Type":"ContainerStarted","Data":"1f54927cd5b15f4da1f359accddd1664500f78b6385b2ba96d973aa2b407cb48"} Jul 13 15:18:29.970892 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:29.970845 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k" podStartSLOduration=1.9708284470000001 podStartE2EDuration="1.970828447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 15:18:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 15:18:29.969049383 +0000 UTC m=+1922.209327231" watchObservedRunningTime="2026-07-13 15:18:29.970828447 +0000 UTC m=+1922.211106295" Jul 13 15:18:39.112162 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:39.112126 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k" Jul 13 15:18:39.112697 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:39.112197 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k" Jul 13 15:18:39.124562 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:39.124540 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k" Jul 13 15:18:40.012958 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:40.012929 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k" Jul 13 15:18:48.830035 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:48.829999 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v"] Jul 13 15:18:48.853246 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:48.853210 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:18:49.837737 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:49.837704 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k"] Jul 13 15:18:49.857892 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:49.857856 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:18:50.036881 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:50.036852 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:18:50.446129 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:50.446093 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:18:50.558680 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:50.558630 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:18:50.661517 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:50.661482 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:18:50.893835 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:50.893803 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:18:55.525334 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:18:55.525297 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:19:03.850048 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:03.850003 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v" podUID="7b2f35a1-06aa-4494-bbfe-850970365063" containerName="main" containerID="cri-o://de6c37a6870cafeefb2f18bba1a004143db31686b98cc35993ff60c5b288f582" gracePeriod=45 Jul 13 15:19:04.089915 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:04.089891 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v" Jul 13 15:19:04.133508 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:04.133415 2568 generic.go:356] "Generic (PLEG): container finished" podID="7b2f35a1-06aa-4494-bbfe-850970365063" containerID="de6c37a6870cafeefb2f18bba1a004143db31686b98cc35993ff60c5b288f582" exitCode=0 Jul 13 15:19:04.133508 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:04.133463 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v" event={"ID":"7b2f35a1-06aa-4494-bbfe-850970365063","Type":"ContainerDied","Data":"de6c37a6870cafeefb2f18bba1a004143db31686b98cc35993ff60c5b288f582"} Jul 13 15:19:04.133508 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:04.133466 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v" Jul 13 15:19:04.133508 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:04.133491 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v" event={"ID":"7b2f35a1-06aa-4494-bbfe-850970365063","Type":"ContainerDied","Data":"daa6108a5744455e924fe20ef4ca4ba04a8cee466c4c67d8b445f69c36dd24e8"} Jul 13 15:19:04.133508 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:04.133508 2568 scope.go:117] "RemoveContainer" containerID="de6c37a6870cafeefb2f18bba1a004143db31686b98cc35993ff60c5b288f582" Jul 13 15:19:04.142355 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:04.142338 2568 scope.go:117] "RemoveContainer" containerID="de6c37a6870cafeefb2f18bba1a004143db31686b98cc35993ff60c5b288f582" Jul 13 15:19:04.142646 ip-10-0-136-96 kubenswrapper[2568]: E0713 15:19:04.142623 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de6c37a6870cafeefb2f18bba1a004143db31686b98cc35993ff60c5b288f582\": container with ID starting with de6c37a6870cafeefb2f18bba1a004143db31686b98cc35993ff60c5b288f582 not found: ID does not exist" containerID="de6c37a6870cafeefb2f18bba1a004143db31686b98cc35993ff60c5b288f582" Jul 13 15:19:04.142714 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:04.142659 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de6c37a6870cafeefb2f18bba1a004143db31686b98cc35993ff60c5b288f582"} err="failed to get container status \"de6c37a6870cafeefb2f18bba1a004143db31686b98cc35993ff60c5b288f582\": rpc error: code = NotFound desc = could not find container \"de6c37a6870cafeefb2f18bba1a004143db31686b98cc35993ff60c5b288f582\": container with ID starting with de6c37a6870cafeefb2f18bba1a004143db31686b98cc35993ff60c5b288f582 not found: ID does not exist" Jul 13 15:19:04.260516 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:04.260476 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b2f35a1-06aa-4494-bbfe-850970365063-tmp-dir\") pod \"7b2f35a1-06aa-4494-bbfe-850970365063\" (UID: \"7b2f35a1-06aa-4494-bbfe-850970365063\") " Jul 13 15:19:04.260516 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:04.260517 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b2f35a1-06aa-4494-bbfe-850970365063-home\") pod \"7b2f35a1-06aa-4494-bbfe-850970365063\" (UID: \"7b2f35a1-06aa-4494-bbfe-850970365063\") " Jul 13 15:19:04.260758 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:04.260551 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gl4gl\" (UniqueName: \"kubernetes.io/projected/7b2f35a1-06aa-4494-bbfe-850970365063-kube-api-access-gl4gl\") pod \"7b2f35a1-06aa-4494-bbfe-850970365063\" (UID: \"7b2f35a1-06aa-4494-bbfe-850970365063\") " Jul 13 15:19:04.260758 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:04.260626 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b2f35a1-06aa-4494-bbfe-850970365063-tls-certs\") pod \"7b2f35a1-06aa-4494-bbfe-850970365063\" (UID: \"7b2f35a1-06aa-4494-bbfe-850970365063\") " Jul 13 15:19:04.260758 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:04.260655 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b2f35a1-06aa-4494-bbfe-850970365063-dshm\") pod \"7b2f35a1-06aa-4494-bbfe-850970365063\" (UID: \"7b2f35a1-06aa-4494-bbfe-850970365063\") " Jul 13 15:19:04.260758 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:04.260704 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b2f35a1-06aa-4494-bbfe-850970365063-model-cache\") pod \"7b2f35a1-06aa-4494-bbfe-850970365063\" (UID: \"7b2f35a1-06aa-4494-bbfe-850970365063\") " Jul 13 15:19:04.260955 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:04.260831 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b2f35a1-06aa-4494-bbfe-850970365063-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7b2f35a1-06aa-4494-bbfe-850970365063" (UID: "7b2f35a1-06aa-4494-bbfe-850970365063"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:19:04.260955 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:04.260851 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b2f35a1-06aa-4494-bbfe-850970365063-home" (OuterVolumeSpecName: "home") pod "7b2f35a1-06aa-4494-bbfe-850970365063" (UID: "7b2f35a1-06aa-4494-bbfe-850970365063"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:19:04.261070 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:04.261030 2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b2f35a1-06aa-4494-bbfe-850970365063-tmp-dir\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:19:04.261070 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:04.261050 2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b2f35a1-06aa-4494-bbfe-850970365063-home\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:19:04.261166 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:04.261091 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b2f35a1-06aa-4494-bbfe-850970365063-model-cache" (OuterVolumeSpecName: "model-cache") pod "7b2f35a1-06aa-4494-bbfe-850970365063" (UID: "7b2f35a1-06aa-4494-bbfe-850970365063"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:19:04.262966 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:04.262933 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b2f35a1-06aa-4494-bbfe-850970365063-kube-api-access-gl4gl" (OuterVolumeSpecName: "kube-api-access-gl4gl") pod "7b2f35a1-06aa-4494-bbfe-850970365063" (UID: "7b2f35a1-06aa-4494-bbfe-850970365063"). InnerVolumeSpecName "kube-api-access-gl4gl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 15:19:04.263078 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:04.262960 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b2f35a1-06aa-4494-bbfe-850970365063-dshm" (OuterVolumeSpecName: "dshm") pod "7b2f35a1-06aa-4494-bbfe-850970365063" (UID: "7b2f35a1-06aa-4494-bbfe-850970365063"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:19:04.263078 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:04.263013 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b2f35a1-06aa-4494-bbfe-850970365063-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7b2f35a1-06aa-4494-bbfe-850970365063" (UID: "7b2f35a1-06aa-4494-bbfe-850970365063"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 15:19:04.362293 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:04.362257 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gl4gl\" (UniqueName: \"kubernetes.io/projected/7b2f35a1-06aa-4494-bbfe-850970365063-kube-api-access-gl4gl\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:19:04.362293 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:04.362287 2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b2f35a1-06aa-4494-bbfe-850970365063-tls-certs\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:19:04.362293 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:04.362297 2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b2f35a1-06aa-4494-bbfe-850970365063-dshm\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:19:04.362524 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:04.362306 2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b2f35a1-06aa-4494-bbfe-850970365063-model-cache\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:19:04.451739 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:04.451656 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v"] Jul 13 15:19:04.460942 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:04.460910 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-d5c92a/test-model-d5c92a-kserve-794485d8f-bh45v"] Jul 13 15:19:04.858715 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:04.858657 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k" podUID="c05396ca-6ed2-467e-a3fd-87a295191d92" containerName="main" containerID="cri-o://b1179260cf16029d409c865e01ee3d663e6d5fbd0dfd5a3545042825f97d004e" gracePeriod=45 Jul 13 15:19:05.111501 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:05.111440 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k" Jul 13 15:19:05.140529 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:05.140500 2568 generic.go:356] "Generic (PLEG): container finished" podID="c05396ca-6ed2-467e-a3fd-87a295191d92" containerID="b1179260cf16029d409c865e01ee3d663e6d5fbd0dfd5a3545042825f97d004e" exitCode=0 Jul 13 15:19:05.140700 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:05.140547 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k" Jul 13 15:19:05.140700 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:05.140592 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k" event={"ID":"c05396ca-6ed2-467e-a3fd-87a295191d92","Type":"ContainerDied","Data":"b1179260cf16029d409c865e01ee3d663e6d5fbd0dfd5a3545042825f97d004e"} Jul 13 15:19:05.140700 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:05.140633 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k" event={"ID":"c05396ca-6ed2-467e-a3fd-87a295191d92","Type":"ContainerDied","Data":"1f54927cd5b15f4da1f359accddd1664500f78b6385b2ba96d973aa2b407cb48"} Jul 13 15:19:05.140700 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:05.140657 2568 scope.go:117] "RemoveContainer" containerID="b1179260cf16029d409c865e01ee3d663e6d5fbd0dfd5a3545042825f97d004e" Jul 13 15:19:05.150107 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:05.150077 2568 scope.go:117] "RemoveContainer" containerID="b1179260cf16029d409c865e01ee3d663e6d5fbd0dfd5a3545042825f97d004e" Jul 13 15:19:05.150378 ip-10-0-136-96 kubenswrapper[2568]: E0713 15:19:05.150359 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1179260cf16029d409c865e01ee3d663e6d5fbd0dfd5a3545042825f97d004e\": container with ID starting with b1179260cf16029d409c865e01ee3d663e6d5fbd0dfd5a3545042825f97d004e not found: ID does not exist" containerID="b1179260cf16029d409c865e01ee3d663e6d5fbd0dfd5a3545042825f97d004e" Jul 13 15:19:05.150443 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:05.150386 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1179260cf16029d409c865e01ee3d663e6d5fbd0dfd5a3545042825f97d004e"} err="failed to get container status \"b1179260cf16029d409c865e01ee3d663e6d5fbd0dfd5a3545042825f97d004e\": rpc error: code = NotFound desc = could not find container \"b1179260cf16029d409c865e01ee3d663e6d5fbd0dfd5a3545042825f97d004e\": container with ID starting with b1179260cf16029d409c865e01ee3d663e6d5fbd0dfd5a3545042825f97d004e not found: ID does not exist" Jul 13 15:19:05.168691 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:05.168660 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c05396ca-6ed2-467e-a3fd-87a295191d92-dshm\") pod \"c05396ca-6ed2-467e-a3fd-87a295191d92\" (UID: \"c05396ca-6ed2-467e-a3fd-87a295191d92\") " Jul 13 15:19:05.168786 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:05.168717 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c05396ca-6ed2-467e-a3fd-87a295191d92-model-cache\") pod \"c05396ca-6ed2-467e-a3fd-87a295191d92\" (UID: \"c05396ca-6ed2-467e-a3fd-87a295191d92\") " Jul 13 15:19:05.168786 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:05.168744 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-flmqx\" (UniqueName: \"kubernetes.io/projected/c05396ca-6ed2-467e-a3fd-87a295191d92-kube-api-access-flmqx\") pod \"c05396ca-6ed2-467e-a3fd-87a295191d92\" (UID: \"c05396ca-6ed2-467e-a3fd-87a295191d92\") " Jul 13 15:19:05.168786 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:05.168763 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c05396ca-6ed2-467e-a3fd-87a295191d92-tls-certs\") pod \"c05396ca-6ed2-467e-a3fd-87a295191d92\" (UID: \"c05396ca-6ed2-467e-a3fd-87a295191d92\") " Jul 13 15:19:05.168947 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:05.168799 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c05396ca-6ed2-467e-a3fd-87a295191d92-home\") pod \"c05396ca-6ed2-467e-a3fd-87a295191d92\" (UID: \"c05396ca-6ed2-467e-a3fd-87a295191d92\") " Jul 13 15:19:05.168947 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:05.168873 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c05396ca-6ed2-467e-a3fd-87a295191d92-tmp-dir\") pod \"c05396ca-6ed2-467e-a3fd-87a295191d92\" (UID: \"c05396ca-6ed2-467e-a3fd-87a295191d92\") " Jul 13 15:19:05.169049 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:05.169016 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c05396ca-6ed2-467e-a3fd-87a295191d92-model-cache" (OuterVolumeSpecName: "model-cache") pod "c05396ca-6ed2-467e-a3fd-87a295191d92" (UID: "c05396ca-6ed2-467e-a3fd-87a295191d92"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:19:05.169207 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:05.169182 2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c05396ca-6ed2-467e-a3fd-87a295191d92-model-cache\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:19:05.169207 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:05.169186 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c05396ca-6ed2-467e-a3fd-87a295191d92-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c05396ca-6ed2-467e-a3fd-87a295191d92" (UID: "c05396ca-6ed2-467e-a3fd-87a295191d92"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:19:05.169345 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:05.169216 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c05396ca-6ed2-467e-a3fd-87a295191d92-home" (OuterVolumeSpecName: "home") pod "c05396ca-6ed2-467e-a3fd-87a295191d92" (UID: "c05396ca-6ed2-467e-a3fd-87a295191d92"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:19:05.170853 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:05.170831 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c05396ca-6ed2-467e-a3fd-87a295191d92-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c05396ca-6ed2-467e-a3fd-87a295191d92" (UID: "c05396ca-6ed2-467e-a3fd-87a295191d92"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 15:19:05.170922 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:05.170882 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c05396ca-6ed2-467e-a3fd-87a295191d92-dshm" (OuterVolumeSpecName: "dshm") pod "c05396ca-6ed2-467e-a3fd-87a295191d92" (UID: "c05396ca-6ed2-467e-a3fd-87a295191d92"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:19:05.170973 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:05.170948 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c05396ca-6ed2-467e-a3fd-87a295191d92-kube-api-access-flmqx" (OuterVolumeSpecName: "kube-api-access-flmqx") pod "c05396ca-6ed2-467e-a3fd-87a295191d92" (UID: "c05396ca-6ed2-467e-a3fd-87a295191d92"). InnerVolumeSpecName "kube-api-access-flmqx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 15:19:05.270343 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:05.270307 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-flmqx\" (UniqueName: \"kubernetes.io/projected/c05396ca-6ed2-467e-a3fd-87a295191d92-kube-api-access-flmqx\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:19:05.270343 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:05.270344 2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c05396ca-6ed2-467e-a3fd-87a295191d92-tls-certs\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:19:05.270343 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:05.270358 2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c05396ca-6ed2-467e-a3fd-87a295191d92-home\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:19:05.270625 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:05.270370 2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c05396ca-6ed2-467e-a3fd-87a295191d92-tmp-dir\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:19:05.270625 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:05.270382 2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c05396ca-6ed2-467e-a3fd-87a295191d92-dshm\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:19:05.463467 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:05.463429 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k"] Jul 13 15:19:05.470753 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:05.470723 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-3dcb39/test-model-3dcb39-kserve-65894b6bcc-jcs6k"] Jul 13 15:19:06.272515 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:06.272478 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b2f35a1-06aa-4494-bbfe-850970365063" path="/var/lib/kubelet/pods/7b2f35a1-06aa-4494-bbfe-850970365063/volumes" Jul 13 15:19:06.272947 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:06.272932 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c05396ca-6ed2-467e-a3fd-87a295191d92" path="/var/lib/kubelet/pods/c05396ca-6ed2-467e-a3fd-87a295191d92/volumes" Jul 13 15:19:25.665557 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:25.665453 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:19:26.837018 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:26.836977 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:19:32.717379 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:32.717341 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:19:33.016697 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:33.016558 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:19:33.248842 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:33.248797 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:19:33.517994 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:33.517960 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:19:33.932954 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:33.932907 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:19:34.032182 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:34.032121 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:19:34.129651 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:34.129617 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:19:34.268755 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:34.268659 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:19:40.322564 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:40.322530 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:19:40.671684 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:40.671567 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:19:40.799530 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:40.799495 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx"] Jul 13 15:19:40.799720 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:40.799697 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" podUID="894b568d-1704-4593-8af8-c1507e735dee" containerName="istio-proxy" containerID="cri-o://f6e7165336e0421f95c3038bce51fd5a81cf911df04fe9ebd36e91fc45134e13" gracePeriod=30 Jul 13 15:19:40.833128 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:40.833087 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:19:41.010542 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:41.010451 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:19:41.227156 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:41.227122 2568 patch_prober.go:28] interesting pod/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 13 15:19:41.227343 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:41.227184 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" podUID="894b568d-1704-4593-8af8-c1507e735dee" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 15:19:41.393357 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:41.393314 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:19:41.744054 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:41.743971 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:19:41.959090 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:41.959029 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:19:45.208429 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:45.208375 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:19:45.304207 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:45.304173 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:19:45.403349 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:45.403310 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:19:45.551862 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:45.551831 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:19:45.848495 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:45.848461 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:19:45.947709 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:45.947672 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:19:46.050213 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.050184 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:19:46.143300 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.143201 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/894b568d-1704-4593-8af8-c1507e735dee-istio-data\") pod \"894b568d-1704-4593-8af8-c1507e735dee\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " Jul 13 15:19:46.143300 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.143270 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/894b568d-1704-4593-8af8-c1507e735dee-workload-socket\") pod \"894b568d-1704-4593-8af8-c1507e735dee\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " Jul 13 15:19:46.143539 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.143313 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/894b568d-1704-4593-8af8-c1507e735dee-credential-socket\") pod \"894b568d-1704-4593-8af8-c1507e735dee\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " Jul 13 15:19:46.143539 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.143347 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/894b568d-1704-4593-8af8-c1507e735dee-istio-token\") pod \"894b568d-1704-4593-8af8-c1507e735dee\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " Jul 13 15:19:46.143539 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.143382 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/894b568d-1704-4593-8af8-c1507e735dee-istio-podinfo\") pod \"894b568d-1704-4593-8af8-c1507e735dee\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " Jul 13 15:19:46.143539 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.143421 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zhqr9\" (UniqueName: \"kubernetes.io/projected/894b568d-1704-4593-8af8-c1507e735dee-kube-api-access-zhqr9\") pod \"894b568d-1704-4593-8af8-c1507e735dee\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " Jul 13 15:19:46.143539 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.143451 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/894b568d-1704-4593-8af8-c1507e735dee-workload-certs\") pod \"894b568d-1704-4593-8af8-c1507e735dee\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " Jul 13 15:19:46.143539 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.143500 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/894b568d-1704-4593-8af8-c1507e735dee-istio-envoy\") pod \"894b568d-1704-4593-8af8-c1507e735dee\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " Jul 13 15:19:46.143539 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.143531 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/894b568d-1704-4593-8af8-c1507e735dee-istiod-ca-cert\") pod \"894b568d-1704-4593-8af8-c1507e735dee\" (UID: \"894b568d-1704-4593-8af8-c1507e735dee\") " Jul 13 15:19:46.143904 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.143524 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/894b568d-1704-4593-8af8-c1507e735dee-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "894b568d-1704-4593-8af8-c1507e735dee" (UID: "894b568d-1704-4593-8af8-c1507e735dee"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:19:46.143904 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.143666 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/894b568d-1704-4593-8af8-c1507e735dee-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "894b568d-1704-4593-8af8-c1507e735dee" (UID: "894b568d-1704-4593-8af8-c1507e735dee"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:19:46.143904 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.143794 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/894b568d-1704-4593-8af8-c1507e735dee-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "894b568d-1704-4593-8af8-c1507e735dee" (UID: "894b568d-1704-4593-8af8-c1507e735dee"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:19:46.144069 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.143939 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/894b568d-1704-4593-8af8-c1507e735dee-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "894b568d-1704-4593-8af8-c1507e735dee" (UID: "894b568d-1704-4593-8af8-c1507e735dee"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 15:19:46.144217 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.144164 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/894b568d-1704-4593-8af8-c1507e735dee-istio-data" (OuterVolumeSpecName: "istio-data") pod "894b568d-1704-4593-8af8-c1507e735dee" (UID: "894b568d-1704-4593-8af8-c1507e735dee"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:19:46.145900 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.145876 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/894b568d-1704-4593-8af8-c1507e735dee-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "894b568d-1704-4593-8af8-c1507e735dee" (UID: "894b568d-1704-4593-8af8-c1507e735dee"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:19:46.146031 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.146010 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/894b568d-1704-4593-8af8-c1507e735dee-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "894b568d-1704-4593-8af8-c1507e735dee" (UID: "894b568d-1704-4593-8af8-c1507e735dee"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 15:19:46.146096 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.146027 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/894b568d-1704-4593-8af8-c1507e735dee-istio-token" (OuterVolumeSpecName: "istio-token") pod "894b568d-1704-4593-8af8-c1507e735dee" (UID: "894b568d-1704-4593-8af8-c1507e735dee"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 15:19:46.146096 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.146052 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/894b568d-1704-4593-8af8-c1507e735dee-kube-api-access-zhqr9" (OuterVolumeSpecName: "kube-api-access-zhqr9") pod "894b568d-1704-4593-8af8-c1507e735dee" (UID: "894b568d-1704-4593-8af8-c1507e735dee"). InnerVolumeSpecName "kube-api-access-zhqr9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 15:19:46.244410 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.244371 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/894b568d-1704-4593-8af8-c1507e735dee-istiod-ca-cert\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:19:46.244410 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.244403 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/894b568d-1704-4593-8af8-c1507e735dee-istio-data\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:19:46.244410 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.244413 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/894b568d-1704-4593-8af8-c1507e735dee-workload-socket\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:19:46.244896 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.244422 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/894b568d-1704-4593-8af8-c1507e735dee-credential-socket\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:19:46.244896 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.244431 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/894b568d-1704-4593-8af8-c1507e735dee-istio-token\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:19:46.244896 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.244440 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/894b568d-1704-4593-8af8-c1507e735dee-istio-podinfo\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:19:46.244896 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.244447 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhqr9\" (UniqueName: \"kubernetes.io/projected/894b568d-1704-4593-8af8-c1507e735dee-kube-api-access-zhqr9\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:19:46.244896 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.244458 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/894b568d-1704-4593-8af8-c1507e735dee-workload-certs\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:19:46.244896 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.244466 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/894b568d-1704-4593-8af8-c1507e735dee-istio-envoy\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:19:46.352898 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.352868 2568 generic.go:356] "Generic (PLEG): container finished" podID="894b568d-1704-4593-8af8-c1507e735dee" containerID="f6e7165336e0421f95c3038bce51fd5a81cf911df04fe9ebd36e91fc45134e13" exitCode=0 Jul 13 15:19:46.353070 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.352911 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" event={"ID":"894b568d-1704-4593-8af8-c1507e735dee","Type":"ContainerDied","Data":"f6e7165336e0421f95c3038bce51fd5a81cf911df04fe9ebd36e91fc45134e13"} Jul 13 15:19:46.353070 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.352916 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" Jul 13 15:19:46.353070 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.352956 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx" event={"ID":"894b568d-1704-4593-8af8-c1507e735dee","Type":"ContainerDied","Data":"f0f9cc0d2f0d9952c2f8b805f68e653dc56d9d42b75be45feca78fce83ca1d8a"} Jul 13 15:19:46.353070 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.352999 2568 scope.go:117] "RemoveContainer" containerID="f6e7165336e0421f95c3038bce51fd5a81cf911df04fe9ebd36e91fc45134e13" Jul 13 15:19:46.361700 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.361681 2568 scope.go:117] "RemoveContainer" containerID="fa916cd524c7ce19e31f49a867d689592938c5eecd0a3444cba79741dafcc85b" Jul 13 15:19:46.369949 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.369873 2568 scope.go:117] "RemoveContainer" containerID="f6e7165336e0421f95c3038bce51fd5a81cf911df04fe9ebd36e91fc45134e13" Jul 13 15:19:46.370257 ip-10-0-136-96 kubenswrapper[2568]: E0713 15:19:46.370234 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6e7165336e0421f95c3038bce51fd5a81cf911df04fe9ebd36e91fc45134e13\": container with ID starting with f6e7165336e0421f95c3038bce51fd5a81cf911df04fe9ebd36e91fc45134e13 not found: ID does not exist" containerID="f6e7165336e0421f95c3038bce51fd5a81cf911df04fe9ebd36e91fc45134e13" Jul 13 15:19:46.370356 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.370265 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6e7165336e0421f95c3038bce51fd5a81cf911df04fe9ebd36e91fc45134e13"} err="failed to get container status \"f6e7165336e0421f95c3038bce51fd5a81cf911df04fe9ebd36e91fc45134e13\": rpc error: code = NotFound desc = could not find container \"f6e7165336e0421f95c3038bce51fd5a81cf911df04fe9ebd36e91fc45134e13\": container with ID starting with f6e7165336e0421f95c3038bce51fd5a81cf911df04fe9ebd36e91fc45134e13 not found: ID does not exist" Jul 13 15:19:46.370356 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.370282 2568 scope.go:117] "RemoveContainer" containerID="fa916cd524c7ce19e31f49a867d689592938c5eecd0a3444cba79741dafcc85b" Jul 13 15:19:46.370557 ip-10-0-136-96 kubenswrapper[2568]: E0713 15:19:46.370534 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa916cd524c7ce19e31f49a867d689592938c5eecd0a3444cba79741dafcc85b\": container with ID starting with fa916cd524c7ce19e31f49a867d689592938c5eecd0a3444cba79741dafcc85b not found: ID does not exist" containerID="fa916cd524c7ce19e31f49a867d689592938c5eecd0a3444cba79741dafcc85b" Jul 13 15:19:46.370700 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.370567 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa916cd524c7ce19e31f49a867d689592938c5eecd0a3444cba79741dafcc85b"} err="failed to get container status \"fa916cd524c7ce19e31f49a867d689592938c5eecd0a3444cba79741dafcc85b\": rpc error: code = NotFound desc = could not find container \"fa916cd524c7ce19e31f49a867d689592938c5eecd0a3444cba79741dafcc85b\": container with ID starting with fa916cd524c7ce19e31f49a867d689592938c5eecd0a3444cba79741dafcc85b not found: ID does not exist" Jul 13 15:19:46.371491 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.371473 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx"] Jul 13 15:19:46.380409 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:46.378842 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-3dcb39-openshift-default-7ff479b85f-zmrqx"] Jul 13 15:19:48.272566 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:48.272532 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="894b568d-1704-4593-8af8-c1507e735dee" path="/var/lib/kubelet/pods/894b568d-1704-4593-8af8-c1507e735dee/volumes" Jul 13 15:19:49.825461 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:49.825420 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:19:50.323526 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:50.323489 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:19:50.470338 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:50.470301 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:19:50.665951 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:50.665853 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:19:51.050865 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:51.050833 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:19:53.282706 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:53.282673 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:19:53.414485 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:53.414445 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:19:53.512590 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:53.512550 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:19:53.736862 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:53.736769 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:19:54.046842 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:54.046807 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:19:54.147537 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:54.147501 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:19:57.798367 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:19:57.798327 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:06.408403 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:06.408344 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:06.817276 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:06.817242 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:06.918986 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:06.918946 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:07.020064 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:07.020029 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:07.163967 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:07.163874 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:13.624670 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:13.624629 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:13.724695 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:13.724662 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:14.034315 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:14.034276 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:14.266957 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:14.266921 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:14.482316 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:14.482223 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:14.850663 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:14.850628 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:14.951671 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:14.951637 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:15.051314 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:15.051272 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:15.121816 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:15.121729 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:21.162222 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:21.162183 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:21.264178 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:21.264140 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:21.442124 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:21.442041 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:21.686827 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:21.686784 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:22.063173 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:22.063136 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:22.440874 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:22.440771 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:22.538648 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:22.538607 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:25.412330 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:25.412266 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:25.511809 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:25.511772 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:25.675928 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:25.675834 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:26.220513 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:26.220473 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:26.324719 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:26.324674 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:29.943742 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:29.943652 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:30.294700 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:30.294661 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:30.640290 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:30.640179 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:30.755444 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:30.755414 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:33.614837 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:33.614793 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:33.721310 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:33.721276 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:33.819989 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:33.819949 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:33.920093 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:33.920002 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:34.377055 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:34.377021 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:34.477802 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:34.477767 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:38.086289 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:38.086251 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:48.233652 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.233607 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8"] Jul 13 15:20:48.234133 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.233913 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b2f35a1-06aa-4494-bbfe-850970365063" containerName="main" Jul 13 15:20:48.234133 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.233923 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b2f35a1-06aa-4494-bbfe-850970365063" containerName="main" Jul 13 15:20:48.234133 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.233940 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="894b568d-1704-4593-8af8-c1507e735dee" containerName="istio-proxy" Jul 13 15:20:48.234133 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.233946 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="894b568d-1704-4593-8af8-c1507e735dee" containerName="istio-proxy" Jul 13 15:20:48.234133 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.233955 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="894b568d-1704-4593-8af8-c1507e735dee" containerName="istio-proxy" Jul 13 15:20:48.234133 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.233961 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="894b568d-1704-4593-8af8-c1507e735dee" containerName="istio-proxy" Jul 13 15:20:48.234133 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.233969 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c05396ca-6ed2-467e-a3fd-87a295191d92" containerName="main" Jul 13 15:20:48.234133 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.233974 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c05396ca-6ed2-467e-a3fd-87a295191d92" containerName="main" Jul 13 15:20:48.234133 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.234023 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="894b568d-1704-4593-8af8-c1507e735dee" containerName="istio-proxy" Jul 13 15:20:48.234133 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.234030 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="894b568d-1704-4593-8af8-c1507e735dee" containerName="istio-proxy" Jul 13 15:20:48.234133 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.234037 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b2f35a1-06aa-4494-bbfe-850970365063" containerName="main" Jul 13 15:20:48.234133 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.234042 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c05396ca-6ed2-467e-a3fd-87a295191d92" containerName="main" Jul 13 15:20:48.236964 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.236947 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8" Jul 13 15:20:48.240481 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.240456 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0a6921\"/\"default-dockercfg-4j9wp\"" Jul 13 15:20:48.240634 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.240456 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0a6921\"/\"openshift-service-ca.crt\"" Jul 13 15:20:48.240634 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.240508 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0a6921\"/\"kube-root-ca.crt\"" Jul 13 15:20:48.240634 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.240516 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0a6921\"/\"auth-test-model-0a6921-kserve-self-signed-certs\"" Jul 13 15:20:48.248150 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.248127 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8"] Jul 13 15:20:48.285407 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.285378 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2498f14-bf70-442d-991e-1e0eeed48da1-tmp-dir\") pod \"auth-test-model-0a6921-kserve-f9bfb659f-bxtn8\" (UID: \"b2498f14-bf70-442d-991e-1e0eeed48da1\") " pod="ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8" Jul 13 15:20:48.285585 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.285412 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2498f14-bf70-442d-991e-1e0eeed48da1-dshm\") pod \"auth-test-model-0a6921-kserve-f9bfb659f-bxtn8\" (UID: \"b2498f14-bf70-442d-991e-1e0eeed48da1\") " pod="ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8" Jul 13 15:20:48.285585 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.285444 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2498f14-bf70-442d-991e-1e0eeed48da1-home\") pod \"auth-test-model-0a6921-kserve-f9bfb659f-bxtn8\" (UID: \"b2498f14-bf70-442d-991e-1e0eeed48da1\") " pod="ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8" Jul 13 15:20:48.285585 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.285503 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnngx\" (UniqueName: \"kubernetes.io/projected/b2498f14-bf70-442d-991e-1e0eeed48da1-kube-api-access-nnngx\") pod \"auth-test-model-0a6921-kserve-f9bfb659f-bxtn8\" (UID: \"b2498f14-bf70-442d-991e-1e0eeed48da1\") " pod="ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8" Jul 13 15:20:48.285585 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.285526 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2498f14-bf70-442d-991e-1e0eeed48da1-model-cache\") pod \"auth-test-model-0a6921-kserve-f9bfb659f-bxtn8\" (UID: \"b2498f14-bf70-442d-991e-1e0eeed48da1\") " pod="ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8" Jul 13 15:20:48.285585 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.285561 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2498f14-bf70-442d-991e-1e0eeed48da1-tls-certs\") pod \"auth-test-model-0a6921-kserve-f9bfb659f-bxtn8\" (UID: \"b2498f14-bf70-442d-991e-1e0eeed48da1\") " pod="ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8" Jul 13 15:20:48.346832 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.346793 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:48.386597 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.386540 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnngx\" (UniqueName: \"kubernetes.io/projected/b2498f14-bf70-442d-991e-1e0eeed48da1-kube-api-access-nnngx\") pod \"auth-test-model-0a6921-kserve-f9bfb659f-bxtn8\" (UID: \"b2498f14-bf70-442d-991e-1e0eeed48da1\") " pod="ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8" Jul 13 15:20:48.386769 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.386613 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2498f14-bf70-442d-991e-1e0eeed48da1-model-cache\") pod \"auth-test-model-0a6921-kserve-f9bfb659f-bxtn8\" (UID: \"b2498f14-bf70-442d-991e-1e0eeed48da1\") " pod="ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8" Jul 13 15:20:48.386769 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.386673 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2498f14-bf70-442d-991e-1e0eeed48da1-tls-certs\") pod \"auth-test-model-0a6921-kserve-f9bfb659f-bxtn8\" (UID: \"b2498f14-bf70-442d-991e-1e0eeed48da1\") " pod="ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8" Jul 13 15:20:48.386769 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.386739 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2498f14-bf70-442d-991e-1e0eeed48da1-tmp-dir\") pod \"auth-test-model-0a6921-kserve-f9bfb659f-bxtn8\" (UID: \"b2498f14-bf70-442d-991e-1e0eeed48da1\") " pod="ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8" Jul 13 15:20:48.386938 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.386770 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2498f14-bf70-442d-991e-1e0eeed48da1-dshm\") pod \"auth-test-model-0a6921-kserve-f9bfb659f-bxtn8\" (UID: \"b2498f14-bf70-442d-991e-1e0eeed48da1\") " pod="ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8" Jul 13 15:20:48.386938 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.386814 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2498f14-bf70-442d-991e-1e0eeed48da1-home\") pod \"auth-test-model-0a6921-kserve-f9bfb659f-bxtn8\" (UID: \"b2498f14-bf70-442d-991e-1e0eeed48da1\") " pod="ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8" Jul 13 15:20:48.387104 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.387082 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2498f14-bf70-442d-991e-1e0eeed48da1-model-cache\") pod \"auth-test-model-0a6921-kserve-f9bfb659f-bxtn8\" (UID: \"b2498f14-bf70-442d-991e-1e0eeed48da1\") " pod="ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8" Jul 13 15:20:48.387143 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.387101 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2498f14-bf70-442d-991e-1e0eeed48da1-tmp-dir\") pod \"auth-test-model-0a6921-kserve-f9bfb659f-bxtn8\" (UID: \"b2498f14-bf70-442d-991e-1e0eeed48da1\") " pod="ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8" Jul 13 15:20:48.387235 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.387217 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2498f14-bf70-442d-991e-1e0eeed48da1-home\") pod \"auth-test-model-0a6921-kserve-f9bfb659f-bxtn8\" (UID: \"b2498f14-bf70-442d-991e-1e0eeed48da1\") " pod="ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8" Jul 13 15:20:48.388926 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.388908 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2498f14-bf70-442d-991e-1e0eeed48da1-dshm\") pod \"auth-test-model-0a6921-kserve-f9bfb659f-bxtn8\" (UID: \"b2498f14-bf70-442d-991e-1e0eeed48da1\") " pod="ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8" Jul 13 15:20:48.389281 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.389264 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2498f14-bf70-442d-991e-1e0eeed48da1-tls-certs\") pod \"auth-test-model-0a6921-kserve-f9bfb659f-bxtn8\" (UID: \"b2498f14-bf70-442d-991e-1e0eeed48da1\") " pod="ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8" Jul 13 15:20:48.398989 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.398960 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnngx\" (UniqueName: \"kubernetes.io/projected/b2498f14-bf70-442d-991e-1e0eeed48da1-kube-api-access-nnngx\") pod \"auth-test-model-0a6921-kserve-f9bfb659f-bxtn8\" (UID: \"b2498f14-bf70-442d-991e-1e0eeed48da1\") " pod="ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8" Jul 13 15:20:48.447093 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.447057 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:48.546810 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.546773 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8" Jul 13 15:20:48.687273 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:48.687249 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8"] Jul 13 15:20:48.689976 ip-10-0-136-96 kubenswrapper[2568]: W0713 15:20:48.689950 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb2498f14_bf70_442d_991e_1e0eeed48da1.slice/crio-fa054d7c3e85b429e086028908815836423a4d8fe288a956dea1da7e29ea7b6c WatchSource:0}: Error finding container fa054d7c3e85b429e086028908815836423a4d8fe288a956dea1da7e29ea7b6c: Status 404 returned error can't find the container with id fa054d7c3e85b429e086028908815836423a4d8fe288a956dea1da7e29ea7b6c Jul 13 15:20:49.676171 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:49.676136 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8" event={"ID":"b2498f14-bf70-442d-991e-1e0eeed48da1","Type":"ContainerStarted","Data":"d8e69ab3263b71a81125c6e8dd05d4d8840e0a247b0eada47e2d006234074cd8"} Jul 13 15:20:49.676171 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:49.676178 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8" event={"ID":"b2498f14-bf70-442d-991e-1e0eeed48da1","Type":"ContainerStarted","Data":"fa054d7c3e85b429e086028908815836423a4d8fe288a956dea1da7e29ea7b6c"} Jul 13 15:20:49.700118 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:49.700067 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8" podStartSLOduration=1.7000512589999999 podStartE2EDuration="1.700051259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 15:20:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 15:20:49.697948829 +0000 UTC m=+2061.938226690" watchObservedRunningTime="2026-07-13 15:20:49.700051259 +0000 UTC m=+2061.940329108" Jul 13 15:20:55.388677 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:55.388631 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:55.487725 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:55.487686 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:20:58.547810 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:58.547780 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8" Jul 13 15:20:58.548275 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:58.547859 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8" Jul 13 15:20:58.560279 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:58.560251 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8" Jul 13 15:20:58.730741 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:20:58.730708 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8" Jul 13 15:21:09.104003 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:09.103963 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh"] Jul 13 15:21:09.107810 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:09.107791 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh" Jul 13 15:21:09.111261 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:09.111238 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-215a0a\"/\"openshift-service-ca.crt\"" Jul 13 15:21:09.111457 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:09.111300 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-215a0a\"/\"kube-root-ca.crt\"" Jul 13 15:21:09.111457 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:09.111311 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-215a0a\"/\"default-dockercfg-z2n9k\"" Jul 13 15:21:09.111651 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:09.111637 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-215a0a\"/\"auth-test-model-215a0a-kserve-self-signed-certs\"" Jul 13 15:21:09.118860 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:09.118837 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh"] Jul 13 15:21:09.138359 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:09.138321 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2333788a-6250-488e-9598-c24a0bf83520-home\") pod \"auth-test-model-215a0a-kserve-6859ff9dc-ld2qh\" (UID: \"2333788a-6250-488e-9598-c24a0bf83520\") " pod="ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh" Jul 13 15:21:09.138584 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:09.138382 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2333788a-6250-488e-9598-c24a0bf83520-model-cache\") pod \"auth-test-model-215a0a-kserve-6859ff9dc-ld2qh\" (UID: \"2333788a-6250-488e-9598-c24a0bf83520\") " pod="ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh" Jul 13 15:21:09.138584 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:09.138430 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2333788a-6250-488e-9598-c24a0bf83520-dshm\") pod \"auth-test-model-215a0a-kserve-6859ff9dc-ld2qh\" (UID: \"2333788a-6250-488e-9598-c24a0bf83520\") " pod="ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh" Jul 13 15:21:09.138584 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:09.138513 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2333788a-6250-488e-9598-c24a0bf83520-tls-certs\") pod \"auth-test-model-215a0a-kserve-6859ff9dc-ld2qh\" (UID: \"2333788a-6250-488e-9598-c24a0bf83520\") " pod="ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh" Jul 13 15:21:09.138747 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:09.138558 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlhfl\" (UniqueName: \"kubernetes.io/projected/2333788a-6250-488e-9598-c24a0bf83520-kube-api-access-tlhfl\") pod \"auth-test-model-215a0a-kserve-6859ff9dc-ld2qh\" (UID: \"2333788a-6250-488e-9598-c24a0bf83520\") " pod="ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh" Jul 13 15:21:09.138747 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:09.138671 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2333788a-6250-488e-9598-c24a0bf83520-tmp-dir\") pod \"auth-test-model-215a0a-kserve-6859ff9dc-ld2qh\" (UID: \"2333788a-6250-488e-9598-c24a0bf83520\") " pod="ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh" Jul 13 15:21:09.147770 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:09.147735 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:21:09.239755 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:09.239712 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2333788a-6250-488e-9598-c24a0bf83520-tls-certs\") pod \"auth-test-model-215a0a-kserve-6859ff9dc-ld2qh\" (UID: \"2333788a-6250-488e-9598-c24a0bf83520\") " pod="ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh" Jul 13 15:21:09.239959 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:09.239773 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlhfl\" (UniqueName: \"kubernetes.io/projected/2333788a-6250-488e-9598-c24a0bf83520-kube-api-access-tlhfl\") pod \"auth-test-model-215a0a-kserve-6859ff9dc-ld2qh\" (UID: \"2333788a-6250-488e-9598-c24a0bf83520\") " pod="ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh" Jul 13 15:21:09.239959 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:09.239843 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2333788a-6250-488e-9598-c24a0bf83520-tmp-dir\") pod \"auth-test-model-215a0a-kserve-6859ff9dc-ld2qh\" (UID: \"2333788a-6250-488e-9598-c24a0bf83520\") " pod="ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh" Jul 13 15:21:09.239959 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:09.239890 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2333788a-6250-488e-9598-c24a0bf83520-home\") pod \"auth-test-model-215a0a-kserve-6859ff9dc-ld2qh\" (UID: \"2333788a-6250-488e-9598-c24a0bf83520\") " pod="ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh" Jul 13 15:21:09.239959 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:09.239922 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2333788a-6250-488e-9598-c24a0bf83520-model-cache\") pod \"auth-test-model-215a0a-kserve-6859ff9dc-ld2qh\" (UID: \"2333788a-6250-488e-9598-c24a0bf83520\") " pod="ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh" Jul 13 15:21:09.240153 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:09.239960 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2333788a-6250-488e-9598-c24a0bf83520-dshm\") pod \"auth-test-model-215a0a-kserve-6859ff9dc-ld2qh\" (UID: \"2333788a-6250-488e-9598-c24a0bf83520\") " pod="ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh" Jul 13 15:21:09.240311 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:09.240289 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2333788a-6250-488e-9598-c24a0bf83520-tmp-dir\") pod \"auth-test-model-215a0a-kserve-6859ff9dc-ld2qh\" (UID: \"2333788a-6250-488e-9598-c24a0bf83520\") " pod="ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh" Jul 13 15:21:09.240384 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:09.240346 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2333788a-6250-488e-9598-c24a0bf83520-home\") pod \"auth-test-model-215a0a-kserve-6859ff9dc-ld2qh\" (UID: \"2333788a-6250-488e-9598-c24a0bf83520\") " pod="ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh" Jul 13 15:21:09.240384 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:09.240358 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2333788a-6250-488e-9598-c24a0bf83520-model-cache\") pod \"auth-test-model-215a0a-kserve-6859ff9dc-ld2qh\" (UID: \"2333788a-6250-488e-9598-c24a0bf83520\") " pod="ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh" Jul 13 15:21:09.242177 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:09.242140 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2333788a-6250-488e-9598-c24a0bf83520-dshm\") pod \"auth-test-model-215a0a-kserve-6859ff9dc-ld2qh\" (UID: \"2333788a-6250-488e-9598-c24a0bf83520\") " pod="ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh" Jul 13 15:21:09.242362 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:09.242342 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2333788a-6250-488e-9598-c24a0bf83520-tls-certs\") pod \"auth-test-model-215a0a-kserve-6859ff9dc-ld2qh\" (UID: \"2333788a-6250-488e-9598-c24a0bf83520\") " pod="ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh" Jul 13 15:21:09.250486 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:09.250466 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlhfl\" (UniqueName: \"kubernetes.io/projected/2333788a-6250-488e-9598-c24a0bf83520-kube-api-access-tlhfl\") pod \"auth-test-model-215a0a-kserve-6859ff9dc-ld2qh\" (UID: \"2333788a-6250-488e-9598-c24a0bf83520\") " pod="ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh" Jul 13 15:21:09.418674 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:09.418546 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh" Jul 13 15:21:09.554102 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:09.554071 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh"] Jul 13 15:21:09.555064 ip-10-0-136-96 kubenswrapper[2568]: W0713 15:21:09.555041 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2333788a_6250_488e_9598_c24a0bf83520.slice/crio-38c8eebd7b5a584a6860046a7f56bf68e1cc5bdf88d3f58789aef3b216f31b8f WatchSource:0}: Error finding container 38c8eebd7b5a584a6860046a7f56bf68e1cc5bdf88d3f58789aef3b216f31b8f: Status 404 returned error can't find the container with id 38c8eebd7b5a584a6860046a7f56bf68e1cc5bdf88d3f58789aef3b216f31b8f Jul 13 15:21:09.775264 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:09.775231 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh" event={"ID":"2333788a-6250-488e-9598-c24a0bf83520","Type":"ContainerStarted","Data":"15218efada0d91f26a8f6a871030d982971b1aa80ed06849d7eabd9f79549766"} Jul 13 15:21:09.775438 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:09.775281 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh" event={"ID":"2333788a-6250-488e-9598-c24a0bf83520","Type":"ContainerStarted","Data":"38c8eebd7b5a584a6860046a7f56bf68e1cc5bdf88d3f58789aef3b216f31b8f"} Jul 13 15:21:09.812293 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:09.812235 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh" podStartSLOduration=0.812219976 podStartE2EDuration="812.219976ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 15:21:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 15:21:09.810510778 +0000 UTC m=+2082.050788628" watchObservedRunningTime="2026-07-13 15:21:09.812219976 +0000 UTC m=+2082.052497825" Jul 13 15:21:19.419441 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:19.419384 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh" Jul 13 15:21:19.419441 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:19.419456 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh" Jul 13 15:21:19.431897 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:19.431859 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh" Jul 13 15:21:19.834314 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:19.834284 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh" Jul 13 15:21:28.340091 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:28.340059 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xgqvx_0bd25f5c-138c-4c60-9d20-525c8b12ebb1/ovn-acl-logging/0.log" Jul 13 15:21:28.349299 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:21:28.349278 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xgqvx_0bd25f5c-138c-4c60-9d20-525c8b12ebb1/ovn-acl-logging/0.log" Jul 13 15:22:26.182964 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:22:26.182848 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:23:00.355968 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:00.355919 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8"] Jul 13 15:23:00.367727 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:00.367689 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:23:01.208520 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:01.208488 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh"] Jul 13 15:23:01.229730 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:01.229697 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:23:09.215042 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:09.215007 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw"] Jul 13 15:23:09.218292 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:09.218268 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw" Jul 13 15:23:09.220963 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:09.220942 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0a6921\"/\"sub-test-model-0a6921-kserve-self-signed-certs\"" Jul 13 15:23:09.231328 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:09.231305 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw"] Jul 13 15:23:09.243461 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:09.243428 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:23:09.327217 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:09.327175 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-tls-certs\") pod \"sub-test-model-0a6921-kserve-7cf7588669-jcdgw\" (UID: \"3dd33868-8fc2-452f-ac4a-4c02b2bc44cc\") " pod="ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw" Jul 13 15:23:09.327217 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:09.327221 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-model-cache\") pod \"sub-test-model-0a6921-kserve-7cf7588669-jcdgw\" (UID: \"3dd33868-8fc2-452f-ac4a-4c02b2bc44cc\") " pod="ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw" Jul 13 15:23:09.327446 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:09.327285 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nb97q\" (UniqueName: \"kubernetes.io/projected/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-kube-api-access-nb97q\") pod \"sub-test-model-0a6921-kserve-7cf7588669-jcdgw\" (UID: \"3dd33868-8fc2-452f-ac4a-4c02b2bc44cc\") " pod="ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw" Jul 13 15:23:09.327446 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:09.327340 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-home\") pod \"sub-test-model-0a6921-kserve-7cf7588669-jcdgw\" (UID: \"3dd33868-8fc2-452f-ac4a-4c02b2bc44cc\") " pod="ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw" Jul 13 15:23:09.327446 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:09.327371 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-dshm\") pod \"sub-test-model-0a6921-kserve-7cf7588669-jcdgw\" (UID: \"3dd33868-8fc2-452f-ac4a-4c02b2bc44cc\") " pod="ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw" Jul 13 15:23:09.327560 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:09.327448 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-tmp-dir\") pod \"sub-test-model-0a6921-kserve-7cf7588669-jcdgw\" (UID: \"3dd33868-8fc2-452f-ac4a-4c02b2bc44cc\") " pod="ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw" Jul 13 15:23:09.428346 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:09.428310 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-dshm\") pod \"sub-test-model-0a6921-kserve-7cf7588669-jcdgw\" (UID: \"3dd33868-8fc2-452f-ac4a-4c02b2bc44cc\") " pod="ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw" Jul 13 15:23:09.428544 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:09.428372 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-tmp-dir\") pod \"sub-test-model-0a6921-kserve-7cf7588669-jcdgw\" (UID: \"3dd33868-8fc2-452f-ac4a-4c02b2bc44cc\") " pod="ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw" Jul 13 15:23:09.428544 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:09.428399 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-tls-certs\") pod \"sub-test-model-0a6921-kserve-7cf7588669-jcdgw\" (UID: \"3dd33868-8fc2-452f-ac4a-4c02b2bc44cc\") " pod="ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw" Jul 13 15:23:09.428544 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:09.428416 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-model-cache\") pod \"sub-test-model-0a6921-kserve-7cf7588669-jcdgw\" (UID: \"3dd33868-8fc2-452f-ac4a-4c02b2bc44cc\") " pod="ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw" Jul 13 15:23:09.428544 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:09.428441 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nb97q\" (UniqueName: \"kubernetes.io/projected/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-kube-api-access-nb97q\") pod \"sub-test-model-0a6921-kserve-7cf7588669-jcdgw\" (UID: \"3dd33868-8fc2-452f-ac4a-4c02b2bc44cc\") " pod="ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw" Jul 13 15:23:09.428829 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:09.428686 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-home\") pod \"sub-test-model-0a6921-kserve-7cf7588669-jcdgw\" (UID: \"3dd33868-8fc2-452f-ac4a-4c02b2bc44cc\") " pod="ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw" Jul 13 15:23:09.428886 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:09.428868 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-tmp-dir\") pod \"sub-test-model-0a6921-kserve-7cf7588669-jcdgw\" (UID: \"3dd33868-8fc2-452f-ac4a-4c02b2bc44cc\") " pod="ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw" Jul 13 15:23:09.429063 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:09.429040 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-home\") pod \"sub-test-model-0a6921-kserve-7cf7588669-jcdgw\" (UID: \"3dd33868-8fc2-452f-ac4a-4c02b2bc44cc\") " pod="ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw" Jul 13 15:23:09.429196 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:09.429165 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-model-cache\") pod \"sub-test-model-0a6921-kserve-7cf7588669-jcdgw\" (UID: \"3dd33868-8fc2-452f-ac4a-4c02b2bc44cc\") " pod="ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw" Jul 13 15:23:09.430782 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:09.430758 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-dshm\") pod \"sub-test-model-0a6921-kserve-7cf7588669-jcdgw\" (UID: \"3dd33868-8fc2-452f-ac4a-4c02b2bc44cc\") " pod="ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw" Jul 13 15:23:09.430970 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:09.430952 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-tls-certs\") pod \"sub-test-model-0a6921-kserve-7cf7588669-jcdgw\" (UID: \"3dd33868-8fc2-452f-ac4a-4c02b2bc44cc\") " pod="ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw" Jul 13 15:23:09.439067 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:09.439041 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nb97q\" (UniqueName: \"kubernetes.io/projected/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-kube-api-access-nb97q\") pod \"sub-test-model-0a6921-kserve-7cf7588669-jcdgw\" (UID: \"3dd33868-8fc2-452f-ac4a-4c02b2bc44cc\") " pod="ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw" Jul 13 15:23:09.527791 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:09.527760 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw" Jul 13 15:23:09.663785 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:09.663753 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw"] Jul 13 15:23:09.666471 ip-10-0-136-96 kubenswrapper[2568]: W0713 15:23:09.666420 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3dd33868_8fc2_452f_ac4a_4c02b2bc44cc.slice/crio-b76a61db0e868d4e43dfaf8842f466d020713d6794080be146509c4891be00ee WatchSource:0}: Error finding container b76a61db0e868d4e43dfaf8842f466d020713d6794080be146509c4891be00ee: Status 404 returned error can't find the container with id b76a61db0e868d4e43dfaf8842f466d020713d6794080be146509c4891be00ee Jul 13 15:23:10.373703 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:10.373662 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw" event={"ID":"3dd33868-8fc2-452f-ac4a-4c02b2bc44cc","Type":"ContainerStarted","Data":"a965908fbb9f8d3c3d9956abbfc5cde3e65574055906a1aff80551becf52fc90"} Jul 13 15:23:10.374096 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:10.373711 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw" event={"ID":"3dd33868-8fc2-452f-ac4a-4c02b2bc44cc","Type":"ContainerStarted","Data":"b76a61db0e868d4e43dfaf8842f466d020713d6794080be146509c4891be00ee"} Jul 13 15:23:10.411126 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:10.411000 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw" podStartSLOduration=1.4109817900000001 podStartE2EDuration="1.41098179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 15:23:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 15:23:10.409599759 +0000 UTC m=+2202.649877600" watchObservedRunningTime="2026-07-13 15:23:10.41098179 +0000 UTC m=+2202.651259639" Jul 13 15:23:15.376379 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:15.376336 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8" podUID="b2498f14-bf70-442d-991e-1e0eeed48da1" containerName="main" containerID="cri-o://d8e69ab3263b71a81125c6e8dd05d4d8840e0a247b0eada47e2d006234074cd8" gracePeriod=45 Jul 13 15:23:15.633547 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:15.633479 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8" Jul 13 15:23:15.785718 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:15.785669 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2498f14-bf70-442d-991e-1e0eeed48da1-tmp-dir\") pod \"b2498f14-bf70-442d-991e-1e0eeed48da1\" (UID: \"b2498f14-bf70-442d-991e-1e0eeed48da1\") " Jul 13 15:23:15.785924 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:15.785743 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2498f14-bf70-442d-991e-1e0eeed48da1-home\") pod \"b2498f14-bf70-442d-991e-1e0eeed48da1\" (UID: \"b2498f14-bf70-442d-991e-1e0eeed48da1\") " Jul 13 15:23:15.785924 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:15.785771 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nnngx\" (UniqueName: \"kubernetes.io/projected/b2498f14-bf70-442d-991e-1e0eeed48da1-kube-api-access-nnngx\") pod \"b2498f14-bf70-442d-991e-1e0eeed48da1\" (UID: \"b2498f14-bf70-442d-991e-1e0eeed48da1\") " Jul 13 15:23:15.785924 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:15.785820 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2498f14-bf70-442d-991e-1e0eeed48da1-tls-certs\") pod \"b2498f14-bf70-442d-991e-1e0eeed48da1\" (UID: \"b2498f14-bf70-442d-991e-1e0eeed48da1\") " Jul 13 15:23:15.785924 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:15.785867 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2498f14-bf70-442d-991e-1e0eeed48da1-dshm\") pod \"b2498f14-bf70-442d-991e-1e0eeed48da1\" (UID: \"b2498f14-bf70-442d-991e-1e0eeed48da1\") " Jul 13 15:23:15.786130 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:15.785948 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2498f14-bf70-442d-991e-1e0eeed48da1-model-cache\") pod \"b2498f14-bf70-442d-991e-1e0eeed48da1\" (UID: \"b2498f14-bf70-442d-991e-1e0eeed48da1\") " Jul 13 15:23:15.786130 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:15.786032 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2498f14-bf70-442d-991e-1e0eeed48da1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b2498f14-bf70-442d-991e-1e0eeed48da1" (UID: "b2498f14-bf70-442d-991e-1e0eeed48da1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:23:15.786235 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:15.786138 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2498f14-bf70-442d-991e-1e0eeed48da1-home" (OuterVolumeSpecName: "home") pod "b2498f14-bf70-442d-991e-1e0eeed48da1" (UID: "b2498f14-bf70-442d-991e-1e0eeed48da1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:23:15.786275 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:15.786230 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2498f14-bf70-442d-991e-1e0eeed48da1-model-cache" (OuterVolumeSpecName: "model-cache") pod "b2498f14-bf70-442d-991e-1e0eeed48da1" (UID: "b2498f14-bf70-442d-991e-1e0eeed48da1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:23:15.786339 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:15.786294 2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2498f14-bf70-442d-991e-1e0eeed48da1-model-cache\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:23:15.786339 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:15.786303 2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2498f14-bf70-442d-991e-1e0eeed48da1-tmp-dir\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:23:15.786339 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:15.786312 2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2498f14-bf70-442d-991e-1e0eeed48da1-home\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:23:15.788089 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:15.788062 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2498f14-bf70-442d-991e-1e0eeed48da1-dshm" (OuterVolumeSpecName: "dshm") pod "b2498f14-bf70-442d-991e-1e0eeed48da1" (UID: "b2498f14-bf70-442d-991e-1e0eeed48da1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:23:15.788089 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:15.788072 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2498f14-bf70-442d-991e-1e0eeed48da1-kube-api-access-nnngx" (OuterVolumeSpecName: "kube-api-access-nnngx") pod "b2498f14-bf70-442d-991e-1e0eeed48da1" (UID: "b2498f14-bf70-442d-991e-1e0eeed48da1"). InnerVolumeSpecName "kube-api-access-nnngx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 15:23:15.788243 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:15.788109 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2498f14-bf70-442d-991e-1e0eeed48da1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b2498f14-bf70-442d-991e-1e0eeed48da1" (UID: "b2498f14-bf70-442d-991e-1e0eeed48da1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 15:23:15.887946 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:15.887762 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nnngx\" (UniqueName: \"kubernetes.io/projected/b2498f14-bf70-442d-991e-1e0eeed48da1-kube-api-access-nnngx\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:23:15.887946 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:15.887808 2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2498f14-bf70-442d-991e-1e0eeed48da1-tls-certs\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:23:15.887946 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:15.887823 2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2498f14-bf70-442d-991e-1e0eeed48da1-dshm\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:23:16.228673 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:16.228528 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh" podUID="2333788a-6250-488e-9598-c24a0bf83520" containerName="main" containerID="cri-o://15218efada0d91f26a8f6a871030d982971b1aa80ed06849d7eabd9f79549766" gracePeriod=45 Jul 13 15:23:16.406009 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:16.405973 2568 generic.go:356] "Generic (PLEG): container finished" podID="b2498f14-bf70-442d-991e-1e0eeed48da1" containerID="d8e69ab3263b71a81125c6e8dd05d4d8840e0a247b0eada47e2d006234074cd8" exitCode=0 Jul 13 15:23:16.406440 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:16.406022 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8" Jul 13 15:23:16.406440 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:16.406021 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8" event={"ID":"b2498f14-bf70-442d-991e-1e0eeed48da1","Type":"ContainerDied","Data":"d8e69ab3263b71a81125c6e8dd05d4d8840e0a247b0eada47e2d006234074cd8"} Jul 13 15:23:16.406440 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:16.406081 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8" event={"ID":"b2498f14-bf70-442d-991e-1e0eeed48da1","Type":"ContainerDied","Data":"fa054d7c3e85b429e086028908815836423a4d8fe288a956dea1da7e29ea7b6c"} Jul 13 15:23:16.406440 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:16.406098 2568 scope.go:117] "RemoveContainer" containerID="d8e69ab3263b71a81125c6e8dd05d4d8840e0a247b0eada47e2d006234074cd8" Jul 13 15:23:16.409866 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:16.409644 2568 generic.go:356] "Generic (PLEG): container finished" podID="2333788a-6250-488e-9598-c24a0bf83520" containerID="15218efada0d91f26a8f6a871030d982971b1aa80ed06849d7eabd9f79549766" exitCode=0 Jul 13 15:23:16.409866 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:16.409764 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh" event={"ID":"2333788a-6250-488e-9598-c24a0bf83520","Type":"ContainerDied","Data":"15218efada0d91f26a8f6a871030d982971b1aa80ed06849d7eabd9f79549766"} Jul 13 15:23:16.420491 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:16.420465 2568 scope.go:117] "RemoveContainer" containerID="d8e69ab3263b71a81125c6e8dd05d4d8840e0a247b0eada47e2d006234074cd8" Jul 13 15:23:16.420862 ip-10-0-136-96 kubenswrapper[2568]: E0713 15:23:16.420841 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8e69ab3263b71a81125c6e8dd05d4d8840e0a247b0eada47e2d006234074cd8\": container with ID starting with d8e69ab3263b71a81125c6e8dd05d4d8840e0a247b0eada47e2d006234074cd8 not found: ID does not exist" containerID="d8e69ab3263b71a81125c6e8dd05d4d8840e0a247b0eada47e2d006234074cd8" Jul 13 15:23:16.421005 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:16.420870 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8e69ab3263b71a81125c6e8dd05d4d8840e0a247b0eada47e2d006234074cd8"} err="failed to get container status \"d8e69ab3263b71a81125c6e8dd05d4d8840e0a247b0eada47e2d006234074cd8\": rpc error: code = NotFound desc = could not find container \"d8e69ab3263b71a81125c6e8dd05d4d8840e0a247b0eada47e2d006234074cd8\": container with ID starting with d8e69ab3263b71a81125c6e8dd05d4d8840e0a247b0eada47e2d006234074cd8 not found: ID does not exist" Jul 13 15:23:16.428939 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:16.428913 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8"] Jul 13 15:23:16.435866 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:16.435840 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-0a6921/auth-test-model-0a6921-kserve-f9bfb659f-bxtn8"] Jul 13 15:23:16.479811 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:16.479743 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh" Jul 13 15:23:16.593391 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:16.593354 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2333788a-6250-488e-9598-c24a0bf83520-home\") pod \"2333788a-6250-488e-9598-c24a0bf83520\" (UID: \"2333788a-6250-488e-9598-c24a0bf83520\") " Jul 13 15:23:16.593391 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:16.593398 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2333788a-6250-488e-9598-c24a0bf83520-model-cache\") pod \"2333788a-6250-488e-9598-c24a0bf83520\" (UID: \"2333788a-6250-488e-9598-c24a0bf83520\") " Jul 13 15:23:16.593702 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:16.593451 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2333788a-6250-488e-9598-c24a0bf83520-dshm\") pod \"2333788a-6250-488e-9598-c24a0bf83520\" (UID: \"2333788a-6250-488e-9598-c24a0bf83520\") " Jul 13 15:23:16.593702 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:16.593502 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlhfl\" (UniqueName: \"kubernetes.io/projected/2333788a-6250-488e-9598-c24a0bf83520-kube-api-access-tlhfl\") pod \"2333788a-6250-488e-9598-c24a0bf83520\" (UID: \"2333788a-6250-488e-9598-c24a0bf83520\") " Jul 13 15:23:16.593702 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:16.593630 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2333788a-6250-488e-9598-c24a0bf83520-tls-certs\") pod \"2333788a-6250-488e-9598-c24a0bf83520\" (UID: \"2333788a-6250-488e-9598-c24a0bf83520\") " Jul 13 15:23:16.593871 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:16.593721 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2333788a-6250-488e-9598-c24a0bf83520-tmp-dir\") pod \"2333788a-6250-488e-9598-c24a0bf83520\" (UID: \"2333788a-6250-488e-9598-c24a0bf83520\") " Jul 13 15:23:16.593871 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:16.593729 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2333788a-6250-488e-9598-c24a0bf83520-home" (OuterVolumeSpecName: "home") pod "2333788a-6250-488e-9598-c24a0bf83520" (UID: "2333788a-6250-488e-9598-c24a0bf83520"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:23:16.593871 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:16.593738 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2333788a-6250-488e-9598-c24a0bf83520-model-cache" (OuterVolumeSpecName: "model-cache") pod "2333788a-6250-488e-9598-c24a0bf83520" (UID: "2333788a-6250-488e-9598-c24a0bf83520"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:23:16.594019 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:16.593989 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2333788a-6250-488e-9598-c24a0bf83520-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2333788a-6250-488e-9598-c24a0bf83520" (UID: "2333788a-6250-488e-9598-c24a0bf83520"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:23:16.594019 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:16.594006 2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2333788a-6250-488e-9598-c24a0bf83520-home\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:23:16.594122 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:16.594025 2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2333788a-6250-488e-9598-c24a0bf83520-model-cache\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:23:16.595832 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:16.595809 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2333788a-6250-488e-9598-c24a0bf83520-dshm" (OuterVolumeSpecName: "dshm") pod "2333788a-6250-488e-9598-c24a0bf83520" (UID: "2333788a-6250-488e-9598-c24a0bf83520"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:23:16.595966 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:16.595848 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2333788a-6250-488e-9598-c24a0bf83520-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2333788a-6250-488e-9598-c24a0bf83520" (UID: "2333788a-6250-488e-9598-c24a0bf83520"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 15:23:16.595966 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:16.595912 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2333788a-6250-488e-9598-c24a0bf83520-kube-api-access-tlhfl" (OuterVolumeSpecName: "kube-api-access-tlhfl") pod "2333788a-6250-488e-9598-c24a0bf83520" (UID: "2333788a-6250-488e-9598-c24a0bf83520"). InnerVolumeSpecName "kube-api-access-tlhfl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 15:23:16.695150 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:16.695106 2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2333788a-6250-488e-9598-c24a0bf83520-dshm\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:23:16.695150 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:16.695143 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlhfl\" (UniqueName: \"kubernetes.io/projected/2333788a-6250-488e-9598-c24a0bf83520-kube-api-access-tlhfl\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:23:16.695150 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:16.695158 2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2333788a-6250-488e-9598-c24a0bf83520-tls-certs\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:23:16.695405 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:16.695172 2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2333788a-6250-488e-9598-c24a0bf83520-tmp-dir\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:23:17.417400 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:17.417369 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh" Jul 13 15:23:17.417887 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:17.417408 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh" event={"ID":"2333788a-6250-488e-9598-c24a0bf83520","Type":"ContainerDied","Data":"38c8eebd7b5a584a6860046a7f56bf68e1cc5bdf88d3f58789aef3b216f31b8f"} Jul 13 15:23:17.417887 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:17.417459 2568 scope.go:117] "RemoveContainer" containerID="15218efada0d91f26a8f6a871030d982971b1aa80ed06849d7eabd9f79549766" Jul 13 15:23:17.439645 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:17.439615 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh"] Jul 13 15:23:17.448630 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:17.448339 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-215a0a/auth-test-model-215a0a-kserve-6859ff9dc-ld2qh"] Jul 13 15:23:18.273063 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:18.273028 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2333788a-6250-488e-9598-c24a0bf83520" path="/var/lib/kubelet/pods/2333788a-6250-488e-9598-c24a0bf83520/volumes" Jul 13 15:23:18.273432 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:18.273419 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2498f14-bf70-442d-991e-1e0eeed48da1" path="/var/lib/kubelet/pods/b2498f14-bf70-442d-991e-1e0eeed48da1/volumes" Jul 13 15:23:19.528620 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:19.528557 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw" Jul 13 15:23:19.529016 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:19.528664 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw" Jul 13 15:23:19.541049 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:19.541019 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw" Jul 13 15:23:20.446021 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:20.445989 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw" Jul 13 15:23:47.603650 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:47.603611 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7"] Jul 13 15:23:47.604259 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:47.604102 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2333788a-6250-488e-9598-c24a0bf83520" containerName="main" Jul 13 15:23:47.604259 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:47.604125 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2333788a-6250-488e-9598-c24a0bf83520" containerName="main" Jul 13 15:23:47.604259 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:47.604144 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2498f14-bf70-442d-991e-1e0eeed48da1" containerName="main" Jul 13 15:23:47.604259 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:47.604152 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2498f14-bf70-442d-991e-1e0eeed48da1" containerName="main" Jul 13 15:23:47.604259 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:47.604238 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2333788a-6250-488e-9598-c24a0bf83520" containerName="main" Jul 13 15:23:47.604259 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:47.604248 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2498f14-bf70-442d-991e-1e0eeed48da1" containerName="main" Jul 13 15:23:47.607227 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:47.607206 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7" Jul 13 15:23:47.610489 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:47.610466 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-215a0a\"/\"kube-root-ca.crt\"" Jul 13 15:23:47.610489 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:47.610473 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-215a0a\"/\"sub-test-model-215a0a-kserve-self-signed-certs\"" Jul 13 15:23:47.610689 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:47.610473 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-215a0a\"/\"openshift-service-ca.crt\"" Jul 13 15:23:47.610689 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:47.610476 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-215a0a\"/\"default-dockercfg-z2n9k\"" Jul 13 15:23:47.619406 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:47.619388 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7"] Jul 13 15:23:47.630422 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:47.630388 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:23:47.672231 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:47.672194 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72496cd6-8add-49d4-aa93-9e3a60640689-dshm\") pod \"sub-test-model-215a0a-kserve-6b45c8668c-8xmj7\" (UID: \"72496cd6-8add-49d4-aa93-9e3a60640689\") " pod="ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7" Jul 13 15:23:47.672231 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:47.672232 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhwvc\" (UniqueName: \"kubernetes.io/projected/72496cd6-8add-49d4-aa93-9e3a60640689-kube-api-access-rhwvc\") pod \"sub-test-model-215a0a-kserve-6b45c8668c-8xmj7\" (UID: \"72496cd6-8add-49d4-aa93-9e3a60640689\") " pod="ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7" Jul 13 15:23:47.672443 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:47.672263 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72496cd6-8add-49d4-aa93-9e3a60640689-model-cache\") pod \"sub-test-model-215a0a-kserve-6b45c8668c-8xmj7\" (UID: \"72496cd6-8add-49d4-aa93-9e3a60640689\") " pod="ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7" Jul 13 15:23:47.672443 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:47.672344 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72496cd6-8add-49d4-aa93-9e3a60640689-home\") pod \"sub-test-model-215a0a-kserve-6b45c8668c-8xmj7\" (UID: \"72496cd6-8add-49d4-aa93-9e3a60640689\") " pod="ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7" Jul 13 15:23:47.672443 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:47.672404 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72496cd6-8add-49d4-aa93-9e3a60640689-tls-certs\") pod \"sub-test-model-215a0a-kserve-6b45c8668c-8xmj7\" (UID: \"72496cd6-8add-49d4-aa93-9e3a60640689\") " pod="ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7" Jul 13 15:23:47.672443 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:47.672440 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72496cd6-8add-49d4-aa93-9e3a60640689-tmp-dir\") pod \"sub-test-model-215a0a-kserve-6b45c8668c-8xmj7\" (UID: \"72496cd6-8add-49d4-aa93-9e3a60640689\") " pod="ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7" Jul 13 15:23:47.773580 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:47.773546 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72496cd6-8add-49d4-aa93-9e3a60640689-model-cache\") pod \"sub-test-model-215a0a-kserve-6b45c8668c-8xmj7\" (UID: \"72496cd6-8add-49d4-aa93-9e3a60640689\") " pod="ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7" Jul 13 15:23:47.773772 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:47.773616 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72496cd6-8add-49d4-aa93-9e3a60640689-home\") pod \"sub-test-model-215a0a-kserve-6b45c8668c-8xmj7\" (UID: \"72496cd6-8add-49d4-aa93-9e3a60640689\") " pod="ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7" Jul 13 15:23:47.773772 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:47.773660 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72496cd6-8add-49d4-aa93-9e3a60640689-tls-certs\") pod \"sub-test-model-215a0a-kserve-6b45c8668c-8xmj7\" (UID: \"72496cd6-8add-49d4-aa93-9e3a60640689\") " pod="ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7" Jul 13 15:23:47.773772 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:47.773686 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72496cd6-8add-49d4-aa93-9e3a60640689-tmp-dir\") pod \"sub-test-model-215a0a-kserve-6b45c8668c-8xmj7\" (UID: \"72496cd6-8add-49d4-aa93-9e3a60640689\") " pod="ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7" Jul 13 15:23:47.773772 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:47.773716 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72496cd6-8add-49d4-aa93-9e3a60640689-dshm\") pod \"sub-test-model-215a0a-kserve-6b45c8668c-8xmj7\" (UID: \"72496cd6-8add-49d4-aa93-9e3a60640689\") " pod="ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7" Jul 13 15:23:47.773772 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:47.773741 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhwvc\" (UniqueName: \"kubernetes.io/projected/72496cd6-8add-49d4-aa93-9e3a60640689-kube-api-access-rhwvc\") pod \"sub-test-model-215a0a-kserve-6b45c8668c-8xmj7\" (UID: \"72496cd6-8add-49d4-aa93-9e3a60640689\") " pod="ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7" Jul 13 15:23:47.774046 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:47.774020 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72496cd6-8add-49d4-aa93-9e3a60640689-model-cache\") pod \"sub-test-model-215a0a-kserve-6b45c8668c-8xmj7\" (UID: \"72496cd6-8add-49d4-aa93-9e3a60640689\") " pod="ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7" Jul 13 15:23:47.774124 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:47.774106 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72496cd6-8add-49d4-aa93-9e3a60640689-home\") pod \"sub-test-model-215a0a-kserve-6b45c8668c-8xmj7\" (UID: \"72496cd6-8add-49d4-aa93-9e3a60640689\") " pod="ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7" Jul 13 15:23:47.774183 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:47.774149 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72496cd6-8add-49d4-aa93-9e3a60640689-tmp-dir\") pod \"sub-test-model-215a0a-kserve-6b45c8668c-8xmj7\" (UID: \"72496cd6-8add-49d4-aa93-9e3a60640689\") " pod="ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7" Jul 13 15:23:47.775886 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:47.775870 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72496cd6-8add-49d4-aa93-9e3a60640689-dshm\") pod \"sub-test-model-215a0a-kserve-6b45c8668c-8xmj7\" (UID: \"72496cd6-8add-49d4-aa93-9e3a60640689\") " pod="ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7" Jul 13 15:23:47.776299 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:47.776278 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72496cd6-8add-49d4-aa93-9e3a60640689-tls-certs\") pod \"sub-test-model-215a0a-kserve-6b45c8668c-8xmj7\" (UID: \"72496cd6-8add-49d4-aa93-9e3a60640689\") " pod="ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7" Jul 13 15:23:47.789991 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:47.789967 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhwvc\" (UniqueName: \"kubernetes.io/projected/72496cd6-8add-49d4-aa93-9e3a60640689-kube-api-access-rhwvc\") pod \"sub-test-model-215a0a-kserve-6b45c8668c-8xmj7\" (UID: \"72496cd6-8add-49d4-aa93-9e3a60640689\") " pod="ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7" Jul 13 15:23:47.916758 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:47.916644 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7" Jul 13 15:23:48.053126 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:48.053100 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7"] Jul 13 15:23:48.055092 ip-10-0-136-96 kubenswrapper[2568]: W0713 15:23:48.055066 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod72496cd6_8add_49d4_aa93_9e3a60640689.slice/crio-3390b228dd158bc2be8ce56638e9142befcfa4870cb5cdd3f74bc86d27313877 WatchSource:0}: Error finding container 3390b228dd158bc2be8ce56638e9142befcfa4870cb5cdd3f74bc86d27313877: Status 404 returned error can't find the container with id 3390b228dd158bc2be8ce56638e9142befcfa4870cb5cdd3f74bc86d27313877 Jul 13 15:23:48.586549 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:48.586504 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7" event={"ID":"72496cd6-8add-49d4-aa93-9e3a60640689","Type":"ContainerStarted","Data":"caa58df16abbe838857fb42bdc7ab81c79c8c6d0fa73aca1767102ed74a1b3c3"} Jul 13 15:23:48.586549 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:48.586546 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7" event={"ID":"72496cd6-8add-49d4-aa93-9e3a60640689","Type":"ContainerStarted","Data":"3390b228dd158bc2be8ce56638e9142befcfa4870cb5cdd3f74bc86d27313877"} Jul 13 15:23:48.608060 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:48.608004 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7" podStartSLOduration=1.607985077 podStartE2EDuration="1.607985077s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 15:23:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 15:23:48.605665964 +0000 UTC m=+2240.845943827" watchObservedRunningTime="2026-07-13 15:23:48.607985077 +0000 UTC m=+2240.848262961" Jul 13 15:23:57.917501 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:57.917412 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7" Jul 13 15:23:57.917501 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:57.917458 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7" Jul 13 15:23:57.930098 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:57.930073 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7" Jul 13 15:23:58.647741 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:23:58.647713 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7" Jul 13 15:24:08.262372 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:24:08.262329 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:24:08.325866 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:24:08.325828 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:24:26.595674 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:24:26.595635 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:24:26.696154 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:24:26.696116 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:24:44.725385 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:24:44.725347 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw"] Jul 13 15:24:44.735789 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:24:44.735761 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:24:45.528970 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:24:45.528935 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7"] Jul 13 15:24:46.587598 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:24:46.587539 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:24:59.745410 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:24:59.745366 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw" podUID="3dd33868-8fc2-452f-ac4a-4c02b2bc44cc" containerName="main" containerID="cri-o://a965908fbb9f8d3c3d9956abbfc5cde3e65574055906a1aff80551becf52fc90" gracePeriod=45 Jul 13 15:24:59.950252 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:24:59.950222 2568 generic.go:356] "Generic (PLEG): container finished" podID="3dd33868-8fc2-452f-ac4a-4c02b2bc44cc" containerID="a965908fbb9f8d3c3d9956abbfc5cde3e65574055906a1aff80551becf52fc90" exitCode=0 Jul 13 15:24:59.950416 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:24:59.950296 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw" event={"ID":"3dd33868-8fc2-452f-ac4a-4c02b2bc44cc","Type":"ContainerDied","Data":"a965908fbb9f8d3c3d9956abbfc5cde3e65574055906a1aff80551becf52fc90"} Jul 13 15:24:59.994853 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:24:59.994828 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw" Jul 13 15:25:00.108383 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.108330 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-tmp-dir\") pod \"3dd33868-8fc2-452f-ac4a-4c02b2bc44cc\" (UID: \"3dd33868-8fc2-452f-ac4a-4c02b2bc44cc\") " Jul 13 15:25:00.108615 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.108437 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nb97q\" (UniqueName: \"kubernetes.io/projected/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-kube-api-access-nb97q\") pod \"3dd33868-8fc2-452f-ac4a-4c02b2bc44cc\" (UID: \"3dd33868-8fc2-452f-ac4a-4c02b2bc44cc\") " Jul 13 15:25:00.108615 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.108501 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-dshm\") pod \"3dd33868-8fc2-452f-ac4a-4c02b2bc44cc\" (UID: \"3dd33868-8fc2-452f-ac4a-4c02b2bc44cc\") " Jul 13 15:25:00.108615 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.108532 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-tls-certs\") pod \"3dd33868-8fc2-452f-ac4a-4c02b2bc44cc\" (UID: \"3dd33868-8fc2-452f-ac4a-4c02b2bc44cc\") " Jul 13 15:25:00.108615 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.108556 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-model-cache\") pod \"3dd33868-8fc2-452f-ac4a-4c02b2bc44cc\" (UID: \"3dd33868-8fc2-452f-ac4a-4c02b2bc44cc\") " Jul 13 15:25:00.108863 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.108681 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-home\") pod \"3dd33868-8fc2-452f-ac4a-4c02b2bc44cc\" (UID: \"3dd33868-8fc2-452f-ac4a-4c02b2bc44cc\") " Jul 13 15:25:00.108863 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.108678 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3dd33868-8fc2-452f-ac4a-4c02b2bc44cc" (UID: "3dd33868-8fc2-452f-ac4a-4c02b2bc44cc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:25:00.108961 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.108900 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-model-cache" (OuterVolumeSpecName: "model-cache") pod "3dd33868-8fc2-452f-ac4a-4c02b2bc44cc" (UID: "3dd33868-8fc2-452f-ac4a-4c02b2bc44cc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:25:00.108961 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.108916 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-home" (OuterVolumeSpecName: "home") pod "3dd33868-8fc2-452f-ac4a-4c02b2bc44cc" (UID: "3dd33868-8fc2-452f-ac4a-4c02b2bc44cc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:25:00.109073 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.109041 2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-model-cache\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:25:00.109073 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.109056 2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-home\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:25:00.109073 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.109065 2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-tmp-dir\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:25:00.110630 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.110605 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3dd33868-8fc2-452f-ac4a-4c02b2bc44cc" (UID: "3dd33868-8fc2-452f-ac4a-4c02b2bc44cc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 15:25:00.110737 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.110647 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-kube-api-access-nb97q" (OuterVolumeSpecName: "kube-api-access-nb97q") pod "3dd33868-8fc2-452f-ac4a-4c02b2bc44cc" (UID: "3dd33868-8fc2-452f-ac4a-4c02b2bc44cc"). InnerVolumeSpecName "kube-api-access-nb97q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 15:25:00.110737 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.110705 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-dshm" (OuterVolumeSpecName: "dshm") pod "3dd33868-8fc2-452f-ac4a-4c02b2bc44cc" (UID: "3dd33868-8fc2-452f-ac4a-4c02b2bc44cc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:25:00.209998 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.209946 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nb97q\" (UniqueName: \"kubernetes.io/projected/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-kube-api-access-nb97q\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:25:00.209998 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.209991 2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-dshm\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:25:00.209998 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.210004 2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc-tls-certs\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:25:00.548903 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.548858 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7" podUID="72496cd6-8add-49d4-aa93-9e3a60640689" containerName="main" containerID="cri-o://caa58df16abbe838857fb42bdc7ab81c79c8c6d0fa73aca1767102ed74a1b3c3" gracePeriod=45 Jul 13 15:25:00.797430 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.797403 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7" Jul 13 15:25:00.916300 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.916212 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72496cd6-8add-49d4-aa93-9e3a60640689-model-cache\") pod \"72496cd6-8add-49d4-aa93-9e3a60640689\" (UID: \"72496cd6-8add-49d4-aa93-9e3a60640689\") " Jul 13 15:25:00.916300 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.916264 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72496cd6-8add-49d4-aa93-9e3a60640689-dshm\") pod \"72496cd6-8add-49d4-aa93-9e3a60640689\" (UID: \"72496cd6-8add-49d4-aa93-9e3a60640689\") " Jul 13 15:25:00.916300 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.916304 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72496cd6-8add-49d4-aa93-9e3a60640689-tmp-dir\") pod \"72496cd6-8add-49d4-aa93-9e3a60640689\" (UID: \"72496cd6-8add-49d4-aa93-9e3a60640689\") " Jul 13 15:25:00.916627 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.916342 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72496cd6-8add-49d4-aa93-9e3a60640689-tls-certs\") pod \"72496cd6-8add-49d4-aa93-9e3a60640689\" (UID: \"72496cd6-8add-49d4-aa93-9e3a60640689\") " Jul 13 15:25:00.916627 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.916376 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rhwvc\" (UniqueName: \"kubernetes.io/projected/72496cd6-8add-49d4-aa93-9e3a60640689-kube-api-access-rhwvc\") pod \"72496cd6-8add-49d4-aa93-9e3a60640689\" (UID: \"72496cd6-8add-49d4-aa93-9e3a60640689\") " Jul 13 15:25:00.916627 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.916405 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72496cd6-8add-49d4-aa93-9e3a60640689-home\") pod \"72496cd6-8add-49d4-aa93-9e3a60640689\" (UID: \"72496cd6-8add-49d4-aa93-9e3a60640689\") " Jul 13 15:25:00.916627 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.916524 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72496cd6-8add-49d4-aa93-9e3a60640689-model-cache" (OuterVolumeSpecName: "model-cache") pod "72496cd6-8add-49d4-aa93-9e3a60640689" (UID: "72496cd6-8add-49d4-aa93-9e3a60640689"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:25:00.916627 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.916599 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72496cd6-8add-49d4-aa93-9e3a60640689-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "72496cd6-8add-49d4-aa93-9e3a60640689" (UID: "72496cd6-8add-49d4-aa93-9e3a60640689"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:25:00.916859 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.916676 2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72496cd6-8add-49d4-aa93-9e3a60640689-model-cache\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:25:00.916859 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.916708 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72496cd6-8add-49d4-aa93-9e3a60640689-home" (OuterVolumeSpecName: "home") pod "72496cd6-8add-49d4-aa93-9e3a60640689" (UID: "72496cd6-8add-49d4-aa93-9e3a60640689"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:25:00.918503 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.918482 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72496cd6-8add-49d4-aa93-9e3a60640689-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "72496cd6-8add-49d4-aa93-9e3a60640689" (UID: "72496cd6-8add-49d4-aa93-9e3a60640689"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 15:25:00.918584 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.918529 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72496cd6-8add-49d4-aa93-9e3a60640689-dshm" (OuterVolumeSpecName: "dshm") pod "72496cd6-8add-49d4-aa93-9e3a60640689" (UID: "72496cd6-8add-49d4-aa93-9e3a60640689"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:25:00.918633 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.918566 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72496cd6-8add-49d4-aa93-9e3a60640689-kube-api-access-rhwvc" (OuterVolumeSpecName: "kube-api-access-rhwvc") pod "72496cd6-8add-49d4-aa93-9e3a60640689" (UID: "72496cd6-8add-49d4-aa93-9e3a60640689"). InnerVolumeSpecName "kube-api-access-rhwvc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 15:25:00.957324 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.957289 2568 generic.go:356] "Generic (PLEG): container finished" podID="72496cd6-8add-49d4-aa93-9e3a60640689" containerID="caa58df16abbe838857fb42bdc7ab81c79c8c6d0fa73aca1767102ed74a1b3c3" exitCode=0 Jul 13 15:25:00.957501 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.957333 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7" Jul 13 15:25:00.957501 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.957382 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7" event={"ID":"72496cd6-8add-49d4-aa93-9e3a60640689","Type":"ContainerDied","Data":"caa58df16abbe838857fb42bdc7ab81c79c8c6d0fa73aca1767102ed74a1b3c3"} Jul 13 15:25:00.957501 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.957432 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7" event={"ID":"72496cd6-8add-49d4-aa93-9e3a60640689","Type":"ContainerDied","Data":"3390b228dd158bc2be8ce56638e9142befcfa4870cb5cdd3f74bc86d27313877"} Jul 13 15:25:00.957501 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.957456 2568 scope.go:117] "RemoveContainer" containerID="caa58df16abbe838857fb42bdc7ab81c79c8c6d0fa73aca1767102ed74a1b3c3" Jul 13 15:25:00.958828 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.958807 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw" event={"ID":"3dd33868-8fc2-452f-ac4a-4c02b2bc44cc","Type":"ContainerDied","Data":"b76a61db0e868d4e43dfaf8842f466d020713d6794080be146509c4891be00ee"} Jul 13 15:25:00.958934 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.958863 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw" Jul 13 15:25:00.966987 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.966972 2568 scope.go:117] "RemoveContainer" containerID="caa58df16abbe838857fb42bdc7ab81c79c8c6d0fa73aca1767102ed74a1b3c3" Jul 13 15:25:00.967221 ip-10-0-136-96 kubenswrapper[2568]: E0713 15:25:00.967202 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"caa58df16abbe838857fb42bdc7ab81c79c8c6d0fa73aca1767102ed74a1b3c3\": container with ID starting with caa58df16abbe838857fb42bdc7ab81c79c8c6d0fa73aca1767102ed74a1b3c3 not found: ID does not exist" containerID="caa58df16abbe838857fb42bdc7ab81c79c8c6d0fa73aca1767102ed74a1b3c3" Jul 13 15:25:00.967278 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.967228 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caa58df16abbe838857fb42bdc7ab81c79c8c6d0fa73aca1767102ed74a1b3c3"} err="failed to get container status \"caa58df16abbe838857fb42bdc7ab81c79c8c6d0fa73aca1767102ed74a1b3c3\": rpc error: code = NotFound desc = could not find container \"caa58df16abbe838857fb42bdc7ab81c79c8c6d0fa73aca1767102ed74a1b3c3\": container with ID starting with caa58df16abbe838857fb42bdc7ab81c79c8c6d0fa73aca1767102ed74a1b3c3 not found: ID does not exist" Jul 13 15:25:00.967278 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.967246 2568 scope.go:117] "RemoveContainer" containerID="a965908fbb9f8d3c3d9956abbfc5cde3e65574055906a1aff80551becf52fc90" Jul 13 15:25:00.980389 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.980362 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw"] Jul 13 15:25:00.986696 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:00.986673 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-0a6921/sub-test-model-0a6921-kserve-7cf7588669-jcdgw"] Jul 13 15:25:01.000892 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.000864 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7"] Jul 13 15:25:01.009082 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.009059 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-215a0a/sub-test-model-215a0a-kserve-6b45c8668c-8xmj7"] Jul 13 15:25:01.017652 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.017634 2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72496cd6-8add-49d4-aa93-9e3a60640689-tls-certs\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:25:01.017708 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.017653 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhwvc\" (UniqueName: \"kubernetes.io/projected/72496cd6-8add-49d4-aa93-9e3a60640689-kube-api-access-rhwvc\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:25:01.017708 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.017663 2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72496cd6-8add-49d4-aa93-9e3a60640689-home\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:25:01.017708 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.017673 2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72496cd6-8add-49d4-aa93-9e3a60640689-dshm\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:25:01.017708 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.017682 2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72496cd6-8add-49d4-aa93-9e3a60640689-tmp-dir\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:25:01.731436 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.731406 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299"] Jul 13 15:25:01.731771 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.731756 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72496cd6-8add-49d4-aa93-9e3a60640689" containerName="main" Jul 13 15:25:01.731824 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.731773 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="72496cd6-8add-49d4-aa93-9e3a60640689" containerName="main" Jul 13 15:25:01.731824 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.731796 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3dd33868-8fc2-452f-ac4a-4c02b2bc44cc" containerName="main" Jul 13 15:25:01.731824 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.731802 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3dd33868-8fc2-452f-ac4a-4c02b2bc44cc" containerName="main" Jul 13 15:25:01.731921 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.731860 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3dd33868-8fc2-452f-ac4a-4c02b2bc44cc" containerName="main" Jul 13 15:25:01.731921 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.731873 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="72496cd6-8add-49d4-aa93-9e3a60640689" containerName="main" Jul 13 15:25:01.736329 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.736311 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299" Jul 13 15:25:01.741111 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.741090 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0a6921\"/\"kube-root-ca.crt\"" Jul 13 15:25:01.741252 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.741093 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0a6921\"/\"rate-test-model-0a6921-kserve-self-signed-certs\"" Jul 13 15:25:01.741252 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.741227 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0a6921\"/\"default-dockercfg-4j9wp\"" Jul 13 15:25:01.741366 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.741280 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0a6921\"/\"openshift-service-ca.crt\"" Jul 13 15:25:01.748858 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.748836 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299"] Jul 13 15:25:01.795083 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.795042 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:25:01.825503 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.825470 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9f15846-9978-4beb-9a32-33144979617c-tmp-dir\") pod \"rate-test-model-0a6921-kserve-79f9676ff9-tp299\" (UID: \"c9f15846-9978-4beb-9a32-33144979617c\") " pod="ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299" Jul 13 15:25:01.825912 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.825538 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9f15846-9978-4beb-9a32-33144979617c-home\") pod \"rate-test-model-0a6921-kserve-79f9676ff9-tp299\" (UID: \"c9f15846-9978-4beb-9a32-33144979617c\") " pod="ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299" Jul 13 15:25:01.825912 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.825618 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdbh4\" (UniqueName: \"kubernetes.io/projected/c9f15846-9978-4beb-9a32-33144979617c-kube-api-access-xdbh4\") pod \"rate-test-model-0a6921-kserve-79f9676ff9-tp299\" (UID: \"c9f15846-9978-4beb-9a32-33144979617c\") " pod="ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299" Jul 13 15:25:01.825912 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.825687 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9f15846-9978-4beb-9a32-33144979617c-model-cache\") pod \"rate-test-model-0a6921-kserve-79f9676ff9-tp299\" (UID: \"c9f15846-9978-4beb-9a32-33144979617c\") " pod="ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299" Jul 13 15:25:01.825912 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.825765 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9f15846-9978-4beb-9a32-33144979617c-tls-certs\") pod \"rate-test-model-0a6921-kserve-79f9676ff9-tp299\" (UID: \"c9f15846-9978-4beb-9a32-33144979617c\") " pod="ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299" Jul 13 15:25:01.825912 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.825795 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9f15846-9978-4beb-9a32-33144979617c-dshm\") pod \"rate-test-model-0a6921-kserve-79f9676ff9-tp299\" (UID: \"c9f15846-9978-4beb-9a32-33144979617c\") " pod="ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299" Jul 13 15:25:01.926849 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.926811 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9f15846-9978-4beb-9a32-33144979617c-tmp-dir\") pod \"rate-test-model-0a6921-kserve-79f9676ff9-tp299\" (UID: \"c9f15846-9978-4beb-9a32-33144979617c\") " pod="ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299" Jul 13 15:25:01.927031 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.926880 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9f15846-9978-4beb-9a32-33144979617c-home\") pod \"rate-test-model-0a6921-kserve-79f9676ff9-tp299\" (UID: \"c9f15846-9978-4beb-9a32-33144979617c\") " pod="ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299" Jul 13 15:25:01.927031 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.926911 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdbh4\" (UniqueName: \"kubernetes.io/projected/c9f15846-9978-4beb-9a32-33144979617c-kube-api-access-xdbh4\") pod \"rate-test-model-0a6921-kserve-79f9676ff9-tp299\" (UID: \"c9f15846-9978-4beb-9a32-33144979617c\") " pod="ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299" Jul 13 15:25:01.927152 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.927065 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9f15846-9978-4beb-9a32-33144979617c-model-cache\") pod \"rate-test-model-0a6921-kserve-79f9676ff9-tp299\" (UID: \"c9f15846-9978-4beb-9a32-33144979617c\") " pod="ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299" Jul 13 15:25:01.927211 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.927167 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9f15846-9978-4beb-9a32-33144979617c-tls-certs\") pod \"rate-test-model-0a6921-kserve-79f9676ff9-tp299\" (UID: \"c9f15846-9978-4beb-9a32-33144979617c\") " pod="ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299" Jul 13 15:25:01.927211 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.927203 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9f15846-9978-4beb-9a32-33144979617c-dshm\") pod \"rate-test-model-0a6921-kserve-79f9676ff9-tp299\" (UID: \"c9f15846-9978-4beb-9a32-33144979617c\") " pod="ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299" Jul 13 15:25:01.927329 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.927311 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9f15846-9978-4beb-9a32-33144979617c-home\") pod \"rate-test-model-0a6921-kserve-79f9676ff9-tp299\" (UID: \"c9f15846-9978-4beb-9a32-33144979617c\") " pod="ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299" Jul 13 15:25:01.927390 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.927344 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9f15846-9978-4beb-9a32-33144979617c-tmp-dir\") pod \"rate-test-model-0a6921-kserve-79f9676ff9-tp299\" (UID: \"c9f15846-9978-4beb-9a32-33144979617c\") " pod="ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299" Jul 13 15:25:01.927454 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.927392 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9f15846-9978-4beb-9a32-33144979617c-model-cache\") pod \"rate-test-model-0a6921-kserve-79f9676ff9-tp299\" (UID: \"c9f15846-9978-4beb-9a32-33144979617c\") " pod="ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299" Jul 13 15:25:01.929482 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.929459 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9f15846-9978-4beb-9a32-33144979617c-dshm\") pod \"rate-test-model-0a6921-kserve-79f9676ff9-tp299\" (UID: \"c9f15846-9978-4beb-9a32-33144979617c\") " pod="ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299" Jul 13 15:25:01.929624 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.929609 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9f15846-9978-4beb-9a32-33144979617c-tls-certs\") pod \"rate-test-model-0a6921-kserve-79f9676ff9-tp299\" (UID: \"c9f15846-9978-4beb-9a32-33144979617c\") " pod="ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299" Jul 13 15:25:01.938119 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:01.938099 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdbh4\" (UniqueName: \"kubernetes.io/projected/c9f15846-9978-4beb-9a32-33144979617c-kube-api-access-xdbh4\") pod \"rate-test-model-0a6921-kserve-79f9676ff9-tp299\" (UID: \"c9f15846-9978-4beb-9a32-33144979617c\") " pod="ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299" Jul 13 15:25:02.046105 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:02.046069 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299" Jul 13 15:25:02.184279 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:02.184251 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299"] Jul 13 15:25:02.185619 ip-10-0-136-96 kubenswrapper[2568]: W0713 15:25:02.185586 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc9f15846_9978_4beb_9a32_33144979617c.slice/crio-5bd381a9ee6b8cd5362e76250bec7115fa9971707560b53d5925e4e4f34efec8 WatchSource:0}: Error finding container 5bd381a9ee6b8cd5362e76250bec7115fa9971707560b53d5925e4e4f34efec8: Status 404 returned error can't find the container with id 5bd381a9ee6b8cd5362e76250bec7115fa9971707560b53d5925e4e4f34efec8 Jul 13 15:25:02.272742 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:02.272708 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3dd33868-8fc2-452f-ac4a-4c02b2bc44cc" path="/var/lib/kubelet/pods/3dd33868-8fc2-452f-ac4a-4c02b2bc44cc/volumes" Jul 13 15:25:02.273142 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:02.273129 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72496cd6-8add-49d4-aa93-9e3a60640689" path="/var/lib/kubelet/pods/72496cd6-8add-49d4-aa93-9e3a60640689/volumes" Jul 13 15:25:02.971329 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:02.971283 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299" event={"ID":"c9f15846-9978-4beb-9a32-33144979617c","Type":"ContainerStarted","Data":"05636a3e1e38a043ffc85e26f02f0cd2e21e7b8188dfdbadee9202a2562e7f7a"} Jul 13 15:25:02.971329 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:02.971337 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299" event={"ID":"c9f15846-9978-4beb-9a32-33144979617c","Type":"ContainerStarted","Data":"5bd381a9ee6b8cd5362e76250bec7115fa9971707560b53d5925e4e4f34efec8"} Jul 13 15:25:03.002806 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:03.002741 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299" podStartSLOduration=2.002724228 podStartE2EDuration="2.002724228s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 15:25:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 15:25:03.002021812 +0000 UTC m=+2315.242299662" watchObservedRunningTime="2026-07-13 15:25:03.002724228 +0000 UTC m=+2315.243002077" Jul 13 15:25:12.046401 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:12.046361 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299" Jul 13 15:25:12.046935 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:12.046451 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299" Jul 13 15:25:12.058816 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:12.058790 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299" Jul 13 15:25:13.033985 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:13.033952 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299" Jul 13 15:25:21.139849 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:21.139770 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85"] Jul 13 15:25:21.142190 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:21.142172 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85" Jul 13 15:25:21.145837 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:21.145812 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-215a0a\"/\"default-dockercfg-z2n9k\"" Jul 13 15:25:21.146560 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:21.146542 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-215a0a\"/\"kube-root-ca.crt\"" Jul 13 15:25:21.146709 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:21.146649 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-215a0a\"/\"openshift-service-ca.crt\"" Jul 13 15:25:21.146779 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:21.146761 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-215a0a\"/\"rate-test-model-215a0a-kserve-self-signed-certs\"" Jul 13 15:25:21.152634 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:21.152612 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85"] Jul 13 15:25:21.188668 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:21.188628 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58f0cf32-eb88-4433-a699-43c51182565c-dshm\") pod \"rate-test-model-215a0a-kserve-644b7d88c5-tgx85\" (UID: \"58f0cf32-eb88-4433-a699-43c51182565c\") " pod="ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85" Jul 13 15:25:21.188842 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:21.188676 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58f0cf32-eb88-4433-a699-43c51182565c-tmp-dir\") pod \"rate-test-model-215a0a-kserve-644b7d88c5-tgx85\" (UID: \"58f0cf32-eb88-4433-a699-43c51182565c\") " pod="ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85" Jul 13 15:25:21.188842 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:21.188706 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58f0cf32-eb88-4433-a699-43c51182565c-model-cache\") pod \"rate-test-model-215a0a-kserve-644b7d88c5-tgx85\" (UID: \"58f0cf32-eb88-4433-a699-43c51182565c\") " pod="ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85" Jul 13 15:25:21.188842 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:21.188736 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jxh5\" (UniqueName: \"kubernetes.io/projected/58f0cf32-eb88-4433-a699-43c51182565c-kube-api-access-6jxh5\") pod \"rate-test-model-215a0a-kserve-644b7d88c5-tgx85\" (UID: \"58f0cf32-eb88-4433-a699-43c51182565c\") " pod="ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85" Jul 13 15:25:21.188842 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:21.188774 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58f0cf32-eb88-4433-a699-43c51182565c-tls-certs\") pod \"rate-test-model-215a0a-kserve-644b7d88c5-tgx85\" (UID: \"58f0cf32-eb88-4433-a699-43c51182565c\") " pod="ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85" Jul 13 15:25:21.189001 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:21.188862 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58f0cf32-eb88-4433-a699-43c51182565c-home\") pod \"rate-test-model-215a0a-kserve-644b7d88c5-tgx85\" (UID: \"58f0cf32-eb88-4433-a699-43c51182565c\") " pod="ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85" Jul 13 15:25:21.189791 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:21.189765 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:25:21.290100 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:21.290064 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jxh5\" (UniqueName: \"kubernetes.io/projected/58f0cf32-eb88-4433-a699-43c51182565c-kube-api-access-6jxh5\") pod \"rate-test-model-215a0a-kserve-644b7d88c5-tgx85\" (UID: \"58f0cf32-eb88-4433-a699-43c51182565c\") " pod="ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85" Jul 13 15:25:21.290312 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:21.290111 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58f0cf32-eb88-4433-a699-43c51182565c-tls-certs\") pod \"rate-test-model-215a0a-kserve-644b7d88c5-tgx85\" (UID: \"58f0cf32-eb88-4433-a699-43c51182565c\") " pod="ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85" Jul 13 15:25:21.290312 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:21.290150 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58f0cf32-eb88-4433-a699-43c51182565c-home\") pod \"rate-test-model-215a0a-kserve-644b7d88c5-tgx85\" (UID: \"58f0cf32-eb88-4433-a699-43c51182565c\") " pod="ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85" Jul 13 15:25:21.290312 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:21.290185 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58f0cf32-eb88-4433-a699-43c51182565c-dshm\") pod \"rate-test-model-215a0a-kserve-644b7d88c5-tgx85\" (UID: \"58f0cf32-eb88-4433-a699-43c51182565c\") " pod="ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85" Jul 13 15:25:21.290312 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:21.290206 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58f0cf32-eb88-4433-a699-43c51182565c-tmp-dir\") pod \"rate-test-model-215a0a-kserve-644b7d88c5-tgx85\" (UID: \"58f0cf32-eb88-4433-a699-43c51182565c\") " pod="ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85" Jul 13 15:25:21.290312 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:21.290225 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58f0cf32-eb88-4433-a699-43c51182565c-model-cache\") pod \"rate-test-model-215a0a-kserve-644b7d88c5-tgx85\" (UID: \"58f0cf32-eb88-4433-a699-43c51182565c\") " pod="ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85" Jul 13 15:25:21.290678 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:21.290648 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58f0cf32-eb88-4433-a699-43c51182565c-home\") pod \"rate-test-model-215a0a-kserve-644b7d88c5-tgx85\" (UID: \"58f0cf32-eb88-4433-a699-43c51182565c\") " pod="ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85" Jul 13 15:25:21.290804 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:21.290690 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58f0cf32-eb88-4433-a699-43c51182565c-model-cache\") pod \"rate-test-model-215a0a-kserve-644b7d88c5-tgx85\" (UID: \"58f0cf32-eb88-4433-a699-43c51182565c\") " pod="ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85" Jul 13 15:25:21.290804 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:21.290716 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58f0cf32-eb88-4433-a699-43c51182565c-tmp-dir\") pod \"rate-test-model-215a0a-kserve-644b7d88c5-tgx85\" (UID: \"58f0cf32-eb88-4433-a699-43c51182565c\") " pod="ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85" Jul 13 15:25:21.292471 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:21.292440 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58f0cf32-eb88-4433-a699-43c51182565c-dshm\") pod \"rate-test-model-215a0a-kserve-644b7d88c5-tgx85\" (UID: \"58f0cf32-eb88-4433-a699-43c51182565c\") " pod="ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85" Jul 13 15:25:21.292680 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:21.292652 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58f0cf32-eb88-4433-a699-43c51182565c-tls-certs\") pod \"rate-test-model-215a0a-kserve-644b7d88c5-tgx85\" (UID: \"58f0cf32-eb88-4433-a699-43c51182565c\") " pod="ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85" Jul 13 15:25:21.300782 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:21.300760 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jxh5\" (UniqueName: \"kubernetes.io/projected/58f0cf32-eb88-4433-a699-43c51182565c-kube-api-access-6jxh5\") pod \"rate-test-model-215a0a-kserve-644b7d88c5-tgx85\" (UID: \"58f0cf32-eb88-4433-a699-43c51182565c\") " pod="ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85" Jul 13 15:25:21.452322 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:21.452223 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85" Jul 13 15:25:21.592127 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:21.592099 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85"] Jul 13 15:25:21.593632 ip-10-0-136-96 kubenswrapper[2568]: W0713 15:25:21.593603 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod58f0cf32_eb88_4433_a699_43c51182565c.slice/crio-78493ab3c39075de05f775b86fde45650e782bd13211fcf1fd4c32df755f71d3 WatchSource:0}: Error finding container 78493ab3c39075de05f775b86fde45650e782bd13211fcf1fd4c32df755f71d3: Status 404 returned error can't find the container with id 78493ab3c39075de05f775b86fde45650e782bd13211fcf1fd4c32df755f71d3 Jul 13 15:25:22.079467 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:22.079429 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85" event={"ID":"58f0cf32-eb88-4433-a699-43c51182565c","Type":"ContainerStarted","Data":"04b21a3b170a0e0507401f19dddbe4653b0ffce38c613246beee84b7b81acd5c"} Jul 13 15:25:22.079702 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:22.079475 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85" event={"ID":"58f0cf32-eb88-4433-a699-43c51182565c","Type":"ContainerStarted","Data":"78493ab3c39075de05f775b86fde45650e782bd13211fcf1fd4c32df755f71d3"} Jul 13 15:25:22.103404 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:22.103349 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85" podStartSLOduration=1.1033330239999999 podStartE2EDuration="1.103333024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 15:25:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 15:25:22.101411867 +0000 UTC m=+2334.341689719" watchObservedRunningTime="2026-07-13 15:25:22.103333024 +0000 UTC m=+2334.343610872" Jul 13 15:25:31.452907 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:31.452872 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85" Jul 13 15:25:31.453432 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:31.452928 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85" Jul 13 15:25:31.465684 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:31.465656 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85" Jul 13 15:25:32.142126 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:32.142093 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85" Jul 13 15:25:49.133024 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:49.132982 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:25:49.232803 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:49.232767 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:25:56.284002 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:56.283963 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:25:56.386116 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:25:56.386081 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:26:00.237762 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:00.237732 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299"] Jul 13 15:26:00.262218 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:00.262176 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:26:01.226139 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:01.226102 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85"] Jul 13 15:26:01.260941 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:01.260903 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:26:02.324551 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:02.324514 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:26:02.732781 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:02.732685 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:26:02.830448 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:02.830408 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:26:02.934505 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:02.934458 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:26:03.277247 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:03.277215 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:26:07.871151 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:07.871106 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:26:15.257297 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:15.257258 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299" podUID="c9f15846-9978-4beb-9a32-33144979617c" containerName="main" containerID="cri-o://05636a3e1e38a043ffc85e26f02f0cd2e21e7b8188dfdbadee9202a2562e7f7a" gracePeriod=45 Jul 13 15:26:15.506903 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:15.506876 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299" Jul 13 15:26:15.576464 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:15.576428 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xdbh4\" (UniqueName: \"kubernetes.io/projected/c9f15846-9978-4beb-9a32-33144979617c-kube-api-access-xdbh4\") pod \"c9f15846-9978-4beb-9a32-33144979617c\" (UID: \"c9f15846-9978-4beb-9a32-33144979617c\") " Jul 13 15:26:15.576688 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:15.576503 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9f15846-9978-4beb-9a32-33144979617c-tmp-dir\") pod \"c9f15846-9978-4beb-9a32-33144979617c\" (UID: \"c9f15846-9978-4beb-9a32-33144979617c\") " Jul 13 15:26:15.576688 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:15.576527 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9f15846-9978-4beb-9a32-33144979617c-dshm\") pod \"c9f15846-9978-4beb-9a32-33144979617c\" (UID: \"c9f15846-9978-4beb-9a32-33144979617c\") " Jul 13 15:26:15.576688 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:15.576564 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9f15846-9978-4beb-9a32-33144979617c-model-cache\") pod \"c9f15846-9978-4beb-9a32-33144979617c\" (UID: \"c9f15846-9978-4beb-9a32-33144979617c\") " Jul 13 15:26:15.576688 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:15.576628 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9f15846-9978-4beb-9a32-33144979617c-home\") pod \"c9f15846-9978-4beb-9a32-33144979617c\" (UID: \"c9f15846-9978-4beb-9a32-33144979617c\") " Jul 13 15:26:15.576688 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:15.576658 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9f15846-9978-4beb-9a32-33144979617c-tls-certs\") pod \"c9f15846-9978-4beb-9a32-33144979617c\" (UID: \"c9f15846-9978-4beb-9a32-33144979617c\") " Jul 13 15:26:15.576961 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:15.576837 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9f15846-9978-4beb-9a32-33144979617c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c9f15846-9978-4beb-9a32-33144979617c" (UID: "c9f15846-9978-4beb-9a32-33144979617c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:26:15.576961 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:15.576880 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9f15846-9978-4beb-9a32-33144979617c-model-cache" (OuterVolumeSpecName: "model-cache") pod "c9f15846-9978-4beb-9a32-33144979617c" (UID: "c9f15846-9978-4beb-9a32-33144979617c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:26:15.576961 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:15.576896 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9f15846-9978-4beb-9a32-33144979617c-home" (OuterVolumeSpecName: "home") pod "c9f15846-9978-4beb-9a32-33144979617c" (UID: "c9f15846-9978-4beb-9a32-33144979617c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:26:15.577109 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:15.576987 2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9f15846-9978-4beb-9a32-33144979617c-tmp-dir\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:26:15.577109 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:15.577005 2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9f15846-9978-4beb-9a32-33144979617c-model-cache\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:26:15.577109 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:15.577019 2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9f15846-9978-4beb-9a32-33144979617c-home\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:26:15.578719 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:15.578688 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9f15846-9978-4beb-9a32-33144979617c-kube-api-access-xdbh4" (OuterVolumeSpecName: "kube-api-access-xdbh4") pod "c9f15846-9978-4beb-9a32-33144979617c" (UID: "c9f15846-9978-4beb-9a32-33144979617c"). InnerVolumeSpecName "kube-api-access-xdbh4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 15:26:15.578719 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:15.578703 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9f15846-9978-4beb-9a32-33144979617c-dshm" (OuterVolumeSpecName: "dshm") pod "c9f15846-9978-4beb-9a32-33144979617c" (UID: "c9f15846-9978-4beb-9a32-33144979617c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:26:15.578719 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:15.578709 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9f15846-9978-4beb-9a32-33144979617c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c9f15846-9978-4beb-9a32-33144979617c" (UID: "c9f15846-9978-4beb-9a32-33144979617c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 15:26:15.677431 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:15.677385 2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9f15846-9978-4beb-9a32-33144979617c-tls-certs\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:26:15.677431 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:15.677428 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdbh4\" (UniqueName: \"kubernetes.io/projected/c9f15846-9978-4beb-9a32-33144979617c-kube-api-access-xdbh4\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:26:15.677431 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:15.677438 2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9f15846-9978-4beb-9a32-33144979617c-dshm\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:26:16.245712 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:16.245654 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85" podUID="58f0cf32-eb88-4433-a699-43c51182565c" containerName="main" containerID="cri-o://04b21a3b170a0e0507401f19dddbe4653b0ffce38c613246beee84b7b81acd5c" gracePeriod=45 Jul 13 15:26:16.367846 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:16.367818 2568 generic.go:356] "Generic (PLEG): container finished" podID="c9f15846-9978-4beb-9a32-33144979617c" containerID="05636a3e1e38a043ffc85e26f02f0cd2e21e7b8188dfdbadee9202a2562e7f7a" exitCode=0 Jul 13 15:26:16.368275 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:16.367863 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299" Jul 13 15:26:16.368275 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:16.367913 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299" event={"ID":"c9f15846-9978-4beb-9a32-33144979617c","Type":"ContainerDied","Data":"05636a3e1e38a043ffc85e26f02f0cd2e21e7b8188dfdbadee9202a2562e7f7a"} Jul 13 15:26:16.368275 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:16.367971 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299" event={"ID":"c9f15846-9978-4beb-9a32-33144979617c","Type":"ContainerDied","Data":"5bd381a9ee6b8cd5362e76250bec7115fa9971707560b53d5925e4e4f34efec8"} Jul 13 15:26:16.368275 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:16.367999 2568 scope.go:117] "RemoveContainer" containerID="05636a3e1e38a043ffc85e26f02f0cd2e21e7b8188dfdbadee9202a2562e7f7a" Jul 13 15:26:16.369788 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:16.369764 2568 generic.go:356] "Generic (PLEG): container finished" podID="58f0cf32-eb88-4433-a699-43c51182565c" containerID="04b21a3b170a0e0507401f19dddbe4653b0ffce38c613246beee84b7b81acd5c" exitCode=0 Jul 13 15:26:16.369904 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:16.369819 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85" event={"ID":"58f0cf32-eb88-4433-a699-43c51182565c","Type":"ContainerDied","Data":"04b21a3b170a0e0507401f19dddbe4653b0ffce38c613246beee84b7b81acd5c"} Jul 13 15:26:16.377513 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:16.377357 2568 scope.go:117] "RemoveContainer" containerID="05636a3e1e38a043ffc85e26f02f0cd2e21e7b8188dfdbadee9202a2562e7f7a" Jul 13 15:26:16.377748 ip-10-0-136-96 kubenswrapper[2568]: E0713 15:26:16.377728 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05636a3e1e38a043ffc85e26f02f0cd2e21e7b8188dfdbadee9202a2562e7f7a\": container with ID starting with 05636a3e1e38a043ffc85e26f02f0cd2e21e7b8188dfdbadee9202a2562e7f7a not found: ID does not exist" containerID="05636a3e1e38a043ffc85e26f02f0cd2e21e7b8188dfdbadee9202a2562e7f7a" Jul 13 15:26:16.377802 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:16.377758 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05636a3e1e38a043ffc85e26f02f0cd2e21e7b8188dfdbadee9202a2562e7f7a"} err="failed to get container status \"05636a3e1e38a043ffc85e26f02f0cd2e21e7b8188dfdbadee9202a2562e7f7a\": rpc error: code = NotFound desc = could not find container \"05636a3e1e38a043ffc85e26f02f0cd2e21e7b8188dfdbadee9202a2562e7f7a\": container with ID starting with 05636a3e1e38a043ffc85e26f02f0cd2e21e7b8188dfdbadee9202a2562e7f7a not found: ID does not exist" Jul 13 15:26:16.390692 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:16.390663 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299"] Jul 13 15:26:16.397641 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:16.397613 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-0a6921/rate-test-model-0a6921-kserve-79f9676ff9-tp299"] Jul 13 15:26:16.499729 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:16.499669 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85" Jul 13 15:26:16.580598 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:16.580542 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:26:16.586456 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:16.586435 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58f0cf32-eb88-4433-a699-43c51182565c-home\") pod \"58f0cf32-eb88-4433-a699-43c51182565c\" (UID: \"58f0cf32-eb88-4433-a699-43c51182565c\") " Jul 13 15:26:16.586582 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:16.586523 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58f0cf32-eb88-4433-a699-43c51182565c-tmp-dir\") pod \"58f0cf32-eb88-4433-a699-43c51182565c\" (UID: \"58f0cf32-eb88-4433-a699-43c51182565c\") " Jul 13 15:26:16.586582 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:16.586560 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58f0cf32-eb88-4433-a699-43c51182565c-model-cache\") pod \"58f0cf32-eb88-4433-a699-43c51182565c\" (UID: \"58f0cf32-eb88-4433-a699-43c51182565c\") " Jul 13 15:26:16.586698 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:16.586594 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58f0cf32-eb88-4433-a699-43c51182565c-tls-certs\") pod \"58f0cf32-eb88-4433-a699-43c51182565c\" (UID: \"58f0cf32-eb88-4433-a699-43c51182565c\") " Jul 13 15:26:16.586698 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:16.586638 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58f0cf32-eb88-4433-a699-43c51182565c-dshm\") pod \"58f0cf32-eb88-4433-a699-43c51182565c\" (UID: \"58f0cf32-eb88-4433-a699-43c51182565c\") " Jul 13 15:26:16.586698 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:16.586672 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6jxh5\" (UniqueName: \"kubernetes.io/projected/58f0cf32-eb88-4433-a699-43c51182565c-kube-api-access-6jxh5\") pod \"58f0cf32-eb88-4433-a699-43c51182565c\" (UID: \"58f0cf32-eb88-4433-a699-43c51182565c\") " Jul 13 15:26:16.586850 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:16.586767 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58f0cf32-eb88-4433-a699-43c51182565c-home" (OuterVolumeSpecName: "home") pod "58f0cf32-eb88-4433-a699-43c51182565c" (UID: "58f0cf32-eb88-4433-a699-43c51182565c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:26:16.586850 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:16.586833 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58f0cf32-eb88-4433-a699-43c51182565c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "58f0cf32-eb88-4433-a699-43c51182565c" (UID: "58f0cf32-eb88-4433-a699-43c51182565c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:26:16.586962 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:16.586854 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58f0cf32-eb88-4433-a699-43c51182565c-model-cache" (OuterVolumeSpecName: "model-cache") pod "58f0cf32-eb88-4433-a699-43c51182565c" (UID: "58f0cf32-eb88-4433-a699-43c51182565c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:26:16.586962 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:16.586937 2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58f0cf32-eb88-4433-a699-43c51182565c-home\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:26:16.586962 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:16.586951 2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58f0cf32-eb88-4433-a699-43c51182565c-tmp-dir\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:26:16.586962 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:16.586959 2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58f0cf32-eb88-4433-a699-43c51182565c-model-cache\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:26:16.588790 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:16.588767 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58f0cf32-eb88-4433-a699-43c51182565c-dshm" (OuterVolumeSpecName: "dshm") pod "58f0cf32-eb88-4433-a699-43c51182565c" (UID: "58f0cf32-eb88-4433-a699-43c51182565c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 15:26:16.588790 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:16.588782 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58f0cf32-eb88-4433-a699-43c51182565c-kube-api-access-6jxh5" (OuterVolumeSpecName: "kube-api-access-6jxh5") pod "58f0cf32-eb88-4433-a699-43c51182565c" (UID: "58f0cf32-eb88-4433-a699-43c51182565c"). InnerVolumeSpecName "kube-api-access-6jxh5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 15:26:16.588923 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:16.588857 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58f0cf32-eb88-4433-a699-43c51182565c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "58f0cf32-eb88-4433-a699-43c51182565c" (UID: "58f0cf32-eb88-4433-a699-43c51182565c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 15:26:16.688167 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:16.688126 2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58f0cf32-eb88-4433-a699-43c51182565c-tls-certs\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:26:16.688167 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:16.688159 2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58f0cf32-eb88-4433-a699-43c51182565c-dshm\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:26:16.688167 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:16.688169 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jxh5\" (UniqueName: \"kubernetes.io/projected/58f0cf32-eb88-4433-a699-43c51182565c-kube-api-access-6jxh5\") on node \"ip-10-0-136-96.ec2.internal\" DevicePath \"\"" Jul 13 15:26:17.375544 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:17.375510 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85" Jul 13 15:26:17.376034 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:17.375537 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85" event={"ID":"58f0cf32-eb88-4433-a699-43c51182565c","Type":"ContainerDied","Data":"78493ab3c39075de05f775b86fde45650e782bd13211fcf1fd4c32df755f71d3"} Jul 13 15:26:17.376034 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:17.375608 2568 scope.go:117] "RemoveContainer" containerID="04b21a3b170a0e0507401f19dddbe4653b0ffce38c613246beee84b7b81acd5c" Jul 13 15:26:17.402914 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:17.402887 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85"] Jul 13 15:26:17.408965 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:17.408939 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-215a0a/rate-test-model-215a0a-kserve-644b7d88c5-tgx85"] Jul 13 15:26:18.272076 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:18.272040 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58f0cf32-eb88-4433-a699-43c51182565c" path="/var/lib/kubelet/pods/58f0cf32-eb88-4433-a699-43c51182565c/volumes" Jul 13 15:26:18.272444 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:18.272431 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9f15846-9978-4beb-9a32-33144979617c" path="/var/lib/kubelet/pods/c9f15846-9978-4beb-9a32-33144979617c/volumes" Jul 13 15:26:28.377837 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:28.377804 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xgqvx_0bd25f5c-138c-4c60-9d20-525c8b12ebb1/ovn-acl-logging/0.log" Jul 13 15:26:28.387949 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:28.387925 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xgqvx_0bd25f5c-138c-4c60-9d20-525c8b12ebb1/ovn-acl-logging/0.log" Jul 13 15:26:45.517597 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:45.517537 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:26:45.763796 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:45.763747 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:26:45.990631 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:45.990595 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:26:46.171735 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:46.171679 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:26:46.581164 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:46.581126 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:26:46.697769 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:46.697731 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:26:46.781849 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:46.781811 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:26:46.951914 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:46.951815 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:26:53.336975 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:53.336880 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:26:53.751010 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:53.750914 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:26:54.687550 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:54.687512 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:26:54.878622 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:54.878563 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:26:58.444328 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:26:58.444291 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:27:00.976945 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:00.976915 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-4n846_b559bca0-d4cc-4a33-b63a-1c03a6c54a84/server/0.log" Jul 13 15:27:01.355095 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:01.355063 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6496967d77-f5ptc_94eff779-7543-4a52-af77-debeeb23f1f9/copy-manifests/0.log" Jul 13 15:27:01.364248 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:01.364216 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6496967d77-f5ptc_94eff779-7543-4a52-af77-debeeb23f1f9/manager/0.log" Jul 13 15:27:02.450215 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:02.450186 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw_b7f81017-83f8-4937-b033-67823029c79b/pull/0.log" Jul 13 15:27:02.462261 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:02.462236 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw_b7f81017-83f8-4937-b033-67823029c79b/extract/0.log" Jul 13 15:27:02.470740 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:02.470719 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw_b7f81017-83f8-4937-b033-67823029c79b/util/0.log" Jul 13 15:27:02.582935 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:02.582909 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc_04986b2b-a511-4c87-907e-45105dd3222a/util/0.log" Jul 13 15:27:02.590417 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:02.590388 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc_04986b2b-a511-4c87-907e-45105dd3222a/pull/0.log" Jul 13 15:27:02.599149 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:02.599124 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc_04986b2b-a511-4c87-907e-45105dd3222a/extract/0.log" Jul 13 15:27:02.717634 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:02.717527 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk_9a82918c-2764-47f5-aa3d-85096e2524b1/util/0.log" Jul 13 15:27:02.726022 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:02.725996 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk_9a82918c-2764-47f5-aa3d-85096e2524b1/pull/0.log" Jul 13 15:27:02.737184 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:02.737161 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk_9a82918c-2764-47f5-aa3d-85096e2524b1/extract/0.log" Jul 13 15:27:02.854915 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:02.854882 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk_7360a6d7-1bce-4ccd-a3e3-111b9bcf051b/util/0.log" Jul 13 15:27:02.863450 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:02.863426 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk_7360a6d7-1bce-4ccd-a3e3-111b9bcf051b/pull/0.log" Jul 13 15:27:02.872472 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:02.872432 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk_7360a6d7-1bce-4ccd-a3e3-111b9bcf051b/extract/0.log" Jul 13 15:27:04.237965 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:04.237938 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678zbt4t_dfe27dc5-31d2-4bda-bddd-a1995f68e78b/istio-proxy/0.log" Jul 13 15:27:04.489635 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:04.489535 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-84c9577b7-772ld_d6122581-51d2-48cc-b2f3-8ee6223e7bd7/kube-auth-proxy/0.log" Jul 13 15:27:05.475243 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:05.475210 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-jgbnv_a08294c3-7fc6-4c60-8bd7-1e2d297ca3ab/main/0.log" Jul 13 15:27:05.854366 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:05.854327 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4c25_dac9630f-90ed-4882-8465-b3954148c835/main/0.log" Jul 13 15:27:06.120817 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:06.120739 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-q4z2n_2d0ab6f4-cec5-48c7-a369-deb499405d34/main/0.log" Jul 13 15:27:10.667437 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:10.667340 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-g54mc/must-gather-j8lxw"] Jul 13 15:27:10.667874 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:10.667689 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9f15846-9978-4beb-9a32-33144979617c" containerName="main" Jul 13 15:27:10.667874 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:10.667702 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9f15846-9978-4beb-9a32-33144979617c" containerName="main" Jul 13 15:27:10.667874 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:10.667712 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58f0cf32-eb88-4433-a699-43c51182565c" containerName="main" Jul 13 15:27:10.667874 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:10.667718 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="58f0cf32-eb88-4433-a699-43c51182565c" containerName="main" Jul 13 15:27:10.667874 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:10.667782 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9f15846-9978-4beb-9a32-33144979617c" containerName="main" Jul 13 15:27:10.667874 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:10.667790 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="58f0cf32-eb88-4433-a699-43c51182565c" containerName="main" Jul 13 15:27:10.670936 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:10.670913 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g54mc/must-gather-j8lxw" Jul 13 15:27:10.674204 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:10.674169 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-g54mc\"/\"default-dockercfg-f7xfv\"" Jul 13 15:27:10.675055 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:10.675038 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g54mc\"/\"kube-root-ca.crt\"" Jul 13 15:27:10.675055 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:10.675050 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g54mc\"/\"openshift-service-ca.crt\"" Jul 13 15:27:10.680720 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:10.680697 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g54mc/must-gather-j8lxw"] Jul 13 15:27:10.700711 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:10.700657 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:27:10.757348 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:10.757310 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7n4dn\" (UniqueName: \"kubernetes.io/projected/483ed7f1-4e02-4080-8f65-b0227b3bc04f-kube-api-access-7n4dn\") pod \"must-gather-j8lxw\" (UID: \"483ed7f1-4e02-4080-8f65-b0227b3bc04f\") " pod="openshift-must-gather-g54mc/must-gather-j8lxw" Jul 13 15:27:10.757519 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:10.757444 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/483ed7f1-4e02-4080-8f65-b0227b3bc04f-must-gather-output\") pod \"must-gather-j8lxw\" (UID: \"483ed7f1-4e02-4080-8f65-b0227b3bc04f\") " pod="openshift-must-gather-g54mc/must-gather-j8lxw" Jul 13 15:27:10.803303 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:10.803258 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" limit=1024 Jul 13 15:27:10.858441 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:10.858400 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/483ed7f1-4e02-4080-8f65-b0227b3bc04f-must-gather-output\") pod \"must-gather-j8lxw\" (UID: \"483ed7f1-4e02-4080-8f65-b0227b3bc04f\") " pod="openshift-must-gather-g54mc/must-gather-j8lxw" Jul 13 15:27:10.858692 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:10.858451 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7n4dn\" (UniqueName: \"kubernetes.io/projected/483ed7f1-4e02-4080-8f65-b0227b3bc04f-kube-api-access-7n4dn\") pod \"must-gather-j8lxw\" (UID: \"483ed7f1-4e02-4080-8f65-b0227b3bc04f\") " pod="openshift-must-gather-g54mc/must-gather-j8lxw" Jul 13 15:27:10.858855 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:10.858834 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/483ed7f1-4e02-4080-8f65-b0227b3bc04f-must-gather-output\") pod \"must-gather-j8lxw\" (UID: \"483ed7f1-4e02-4080-8f65-b0227b3bc04f\") " pod="openshift-must-gather-g54mc/must-gather-j8lxw" Jul 13 15:27:10.867805 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:10.867780 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7n4dn\" (UniqueName: \"kubernetes.io/projected/483ed7f1-4e02-4080-8f65-b0227b3bc04f-kube-api-access-7n4dn\") pod \"must-gather-j8lxw\" (UID: \"483ed7f1-4e02-4080-8f65-b0227b3bc04f\") " pod="openshift-must-gather-g54mc/must-gather-j8lxw" Jul 13 15:27:10.980829 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:10.980733 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g54mc/must-gather-j8lxw" Jul 13 15:27:11.114030 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:11.113998 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g54mc/must-gather-j8lxw"] Jul 13 15:27:11.115459 ip-10-0-136-96 kubenswrapper[2568]: W0713 15:27:11.115433 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod483ed7f1_4e02_4080_8f65_b0227b3bc04f.slice/crio-d0eaa0ecac84d269205d57e6067cc40fef6ac57103ed20909ca0b159073b258b WatchSource:0}: Error finding container d0eaa0ecac84d269205d57e6067cc40fef6ac57103ed20909ca0b159073b258b: Status 404 returned error can't find the container with id d0eaa0ecac84d269205d57e6067cc40fef6ac57103ed20909ca0b159073b258b Jul 13 15:27:11.117233 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:11.117212 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 15:27:11.647642 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:11.647603 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g54mc/must-gather-j8lxw" event={"ID":"483ed7f1-4e02-4080-8f65-b0227b3bc04f","Type":"ContainerStarted","Data":"d0eaa0ecac84d269205d57e6067cc40fef6ac57103ed20909ca0b159073b258b"} Jul 13 15:27:13.662914 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:13.662832 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g54mc/must-gather-j8lxw" event={"ID":"483ed7f1-4e02-4080-8f65-b0227b3bc04f","Type":"ContainerStarted","Data":"1ae9639dea0856f19bb08bea485099295ad0a4b20384ae6fe5f920a6d05b0237"} Jul 13 15:27:13.662914 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:13.662886 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g54mc/must-gather-j8lxw" event={"ID":"483ed7f1-4e02-4080-8f65-b0227b3bc04f","Type":"ContainerStarted","Data":"a2946e2218826a40745808757ffc6532e0b87b836096ffafc398887126f0c019"} Jul 13 15:27:13.693543 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:13.693480 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-g54mc/must-gather-j8lxw" podStartSLOduration=2.1937023 podStartE2EDuration="3.693457767s" totalImagesPullingTime="1.499755467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 15:27:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 15:27:13.691156336 +0000 UTC m=+2445.931434187" watchObservedRunningTime="2026-07-13 15:27:13.693457767 +0000 UTC m=+2445.933735616" Jul 13 15:27:14.412497 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:14.412437 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-km6w9_764568c4-93b4-4bf4-a627-a26e55610c81/global-pull-secret-syncer/0.log" Jul 13 15:27:14.524500 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:14.524464 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-7v6qt_5d5298a8-b2b8-4f8f-80ea-e4e66ba7c822/konnectivity-agent/0.log" Jul 13 15:27:14.682872 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:14.682791 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-96.ec2.internal_4161d8f2b036cc0e991f7a67c69322ec/haproxy/0.log" Jul 13 15:27:18.775856 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:18.775822 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw_b7f81017-83f8-4937-b033-67823029c79b/extract/0.log" Jul 13 15:27:18.812271 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:18.812242 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw_b7f81017-83f8-4937-b033-67823029c79b/util/0.log" Jul 13 15:27:18.857747 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:18.857712 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zbctw_b7f81017-83f8-4937-b033-67823029c79b/pull/0.log" Jul 13 15:27:18.908871 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:18.908839 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc_04986b2b-a511-4c87-907e-45105dd3222a/extract/0.log" Jul 13 15:27:18.947385 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:18.947354 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc_04986b2b-a511-4c87-907e-45105dd3222a/util/0.log" Jul 13 15:27:18.983543 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:18.983503 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06qlmc_04986b2b-a511-4c87-907e-45105dd3222a/pull/0.log" Jul 13 15:27:19.029458 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:19.029377 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk_9a82918c-2764-47f5-aa3d-85096e2524b1/extract/0.log" Jul 13 15:27:19.064425 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:19.064392 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk_9a82918c-2764-47f5-aa3d-85096e2524b1/util/0.log" Jul 13 15:27:19.097890 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:19.097857 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zvsbk_9a82918c-2764-47f5-aa3d-85096e2524b1/pull/0.log" Jul 13 15:27:19.135539 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:19.135511 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk_7360a6d7-1bce-4ccd-a3e3-111b9bcf051b/extract/0.log" Jul 13 15:27:19.167945 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:19.167918 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk_7360a6d7-1bce-4ccd-a3e3-111b9bcf051b/util/0.log" Jul 13 15:27:19.199497 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:19.199463 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pjlmk_7360a6d7-1bce-4ccd-a3e3-111b9bcf051b/pull/0.log" Jul 13 15:27:22.055950 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:22.055915 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6c65bfbc8f-kj8xv_09033cc5-16e0-488e-bf10-a0e7a5838d94/metrics-server/0.log" Jul 13 15:27:22.097492 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:22.097462 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-ghh7k_88947007-7a57-4987-93e7-91531f043125/monitoring-plugin/0.log" Jul 13 15:27:22.276842 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:22.276807 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q748j_a3feb036-72ac-407b-9847-5af7a59377f8/node-exporter/0.log" Jul 13 15:27:22.321449 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:22.321363 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q748j_a3feb036-72ac-407b-9847-5af7a59377f8/kube-rbac-proxy/0.log" Jul 13 15:27:22.359619 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:22.359559 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q748j_a3feb036-72ac-407b-9847-5af7a59377f8/init-textfile/0.log" Jul 13 15:27:23.020839 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:23.020811 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-674b978496-25nld_8de88c3c-89f8-46bc-9486-d12804f06f6d/telemeter-client/0.log" Jul 13 15:27:23.050702 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:23.050672 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-674b978496-25nld_8de88c3c-89f8-46bc-9486-d12804f06f6d/reload/0.log" Jul 13 15:27:23.085671 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:23.085621 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-674b978496-25nld_8de88c3c-89f8-46bc-9486-d12804f06f6d/kube-rbac-proxy/0.log" Jul 13 15:27:24.508596 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:24.508545 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-g54mc/perf-node-gather-daemonset-7slz7"] Jul 13 15:27:24.516991 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:24.516959 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g54mc/perf-node-gather-daemonset-7slz7" Jul 13 15:27:24.524114 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:24.524087 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g54mc/perf-node-gather-daemonset-7slz7"] Jul 13 15:27:24.588166 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:24.588129 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-4frwd_6c8d99ef-1d66-4af6-bc92-e65658a50b36/networking-console-plugin/0.log" Jul 13 15:27:24.604226 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:24.604187 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4p2vr\" (UniqueName: \"kubernetes.io/projected/010ecbfb-1013-4218-8077-956ebbd65542-kube-api-access-4p2vr\") pod \"perf-node-gather-daemonset-7slz7\" (UID: \"010ecbfb-1013-4218-8077-956ebbd65542\") " pod="openshift-must-gather-g54mc/perf-node-gather-daemonset-7slz7" Jul 13 15:27:24.604413 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:24.604241 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/010ecbfb-1013-4218-8077-956ebbd65542-podres\") pod \"perf-node-gather-daemonset-7slz7\" (UID: \"010ecbfb-1013-4218-8077-956ebbd65542\") " pod="openshift-must-gather-g54mc/perf-node-gather-daemonset-7slz7" Jul 13 15:27:24.604413 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:24.604313 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/010ecbfb-1013-4218-8077-956ebbd65542-proc\") pod \"perf-node-gather-daemonset-7slz7\" (UID: \"010ecbfb-1013-4218-8077-956ebbd65542\") " pod="openshift-must-gather-g54mc/perf-node-gather-daemonset-7slz7" Jul 13 15:27:24.604413 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:24.604354 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/010ecbfb-1013-4218-8077-956ebbd65542-sys\") pod \"perf-node-gather-daemonset-7slz7\" (UID: \"010ecbfb-1013-4218-8077-956ebbd65542\") " pod="openshift-must-gather-g54mc/perf-node-gather-daemonset-7slz7" Jul 13 15:27:24.604413 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:24.604409 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/010ecbfb-1013-4218-8077-956ebbd65542-lib-modules\") pod \"perf-node-gather-daemonset-7slz7\" (UID: \"010ecbfb-1013-4218-8077-956ebbd65542\") " pod="openshift-must-gather-g54mc/perf-node-gather-daemonset-7slz7" Jul 13 15:27:24.705818 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:24.705781 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p2vr\" (UniqueName: \"kubernetes.io/projected/010ecbfb-1013-4218-8077-956ebbd65542-kube-api-access-4p2vr\") pod \"perf-node-gather-daemonset-7slz7\" (UID: \"010ecbfb-1013-4218-8077-956ebbd65542\") " pod="openshift-must-gather-g54mc/perf-node-gather-daemonset-7slz7" Jul 13 15:27:24.706089 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:24.706062 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/010ecbfb-1013-4218-8077-956ebbd65542-podres\") pod \"perf-node-gather-daemonset-7slz7\" (UID: \"010ecbfb-1013-4218-8077-956ebbd65542\") " pod="openshift-must-gather-g54mc/perf-node-gather-daemonset-7slz7" Jul 13 15:27:24.706199 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:24.706153 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/010ecbfb-1013-4218-8077-956ebbd65542-proc\") pod \"perf-node-gather-daemonset-7slz7\" (UID: \"010ecbfb-1013-4218-8077-956ebbd65542\") " pod="openshift-must-gather-g54mc/perf-node-gather-daemonset-7slz7" Jul 13 15:27:24.706199 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:24.706181 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/010ecbfb-1013-4218-8077-956ebbd65542-sys\") pod \"perf-node-gather-daemonset-7slz7\" (UID: \"010ecbfb-1013-4218-8077-956ebbd65542\") " pod="openshift-must-gather-g54mc/perf-node-gather-daemonset-7slz7" Jul 13 15:27:24.706278 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:24.706240 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/010ecbfb-1013-4218-8077-956ebbd65542-lib-modules\") pod \"perf-node-gather-daemonset-7slz7\" (UID: \"010ecbfb-1013-4218-8077-956ebbd65542\") " pod="openshift-must-gather-g54mc/perf-node-gather-daemonset-7slz7" Jul 13 15:27:24.706313 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:24.706269 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/010ecbfb-1013-4218-8077-956ebbd65542-podres\") pod \"perf-node-gather-daemonset-7slz7\" (UID: \"010ecbfb-1013-4218-8077-956ebbd65542\") " pod="openshift-must-gather-g54mc/perf-node-gather-daemonset-7slz7" Jul 13 15:27:24.706348 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:24.706291 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/010ecbfb-1013-4218-8077-956ebbd65542-proc\") pod \"perf-node-gather-daemonset-7slz7\" (UID: \"010ecbfb-1013-4218-8077-956ebbd65542\") " pod="openshift-must-gather-g54mc/perf-node-gather-daemonset-7slz7" Jul 13 15:27:24.706348 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:24.706290 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/010ecbfb-1013-4218-8077-956ebbd65542-sys\") pod \"perf-node-gather-daemonset-7slz7\" (UID: \"010ecbfb-1013-4218-8077-956ebbd65542\") " pod="openshift-must-gather-g54mc/perf-node-gather-daemonset-7slz7" Jul 13 15:27:24.706417 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:24.706378 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/010ecbfb-1013-4218-8077-956ebbd65542-lib-modules\") pod \"perf-node-gather-daemonset-7slz7\" (UID: \"010ecbfb-1013-4218-8077-956ebbd65542\") " pod="openshift-must-gather-g54mc/perf-node-gather-daemonset-7slz7" Jul 13 15:27:24.716936 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:24.716891 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4p2vr\" (UniqueName: \"kubernetes.io/projected/010ecbfb-1013-4218-8077-956ebbd65542-kube-api-access-4p2vr\") pod \"perf-node-gather-daemonset-7slz7\" (UID: \"010ecbfb-1013-4218-8077-956ebbd65542\") " pod="openshift-must-gather-g54mc/perf-node-gather-daemonset-7slz7" Jul 13 15:27:24.834783 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:24.834746 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g54mc/perf-node-gather-daemonset-7slz7" Jul 13 15:27:25.028633 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:25.028598 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g54mc/perf-node-gather-daemonset-7slz7"] Jul 13 15:27:25.032116 ip-10-0-136-96 kubenswrapper[2568]: W0713 15:27:25.032065 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod010ecbfb_1013_4218_8077_956ebbd65542.slice/crio-5c3ccd581cd96d431f143ba0385a5c0111b365f6ffef5cb3f3ea91d5eb0ddea6 WatchSource:0}: Error finding container 5c3ccd581cd96d431f143ba0385a5c0111b365f6ffef5cb3f3ea91d5eb0ddea6: Status 404 returned error can't find the container with id 5c3ccd581cd96d431f143ba0385a5c0111b365f6ffef5cb3f3ea91d5eb0ddea6 Jul 13 15:27:25.751075 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:25.751032 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g54mc/perf-node-gather-daemonset-7slz7" event={"ID":"010ecbfb-1013-4218-8077-956ebbd65542","Type":"ContainerStarted","Data":"614a35f1a82aed706ead8dc715d2863eadbbd00675325ffa97dce17db3190476"} Jul 13 15:27:25.751585 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:25.751104 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g54mc/perf-node-gather-daemonset-7slz7" event={"ID":"010ecbfb-1013-4218-8077-956ebbd65542","Type":"ContainerStarted","Data":"5c3ccd581cd96d431f143ba0385a5c0111b365f6ffef5cb3f3ea91d5eb0ddea6"} Jul 13 15:27:25.751585 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:25.751312 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-g54mc/perf-node-gather-daemonset-7slz7" Jul 13 15:27:25.778715 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:25.778661 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-g54mc/perf-node-gather-daemonset-7slz7" podStartSLOduration=1.77863864 podStartE2EDuration="1.77863864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 15:27:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 15:27:25.776842353 +0000 UTC m=+2458.017120228" watchObservedRunningTime="2026-07-13 15:27:25.77863864 +0000 UTC m=+2458.018916490" Jul 13 15:27:27.948256 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:27.948221 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b47lx_6c60d2d4-243b-4952-83b9-bb2c462d9e2c/dns/0.log" Jul 13 15:27:27.978957 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:27.978909 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b47lx_6c60d2d4-243b-4952-83b9-bb2c462d9e2c/kube-rbac-proxy/0.log" Jul 13 15:27:28.184843 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:28.184810 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xz7wd_d05cd493-5f17-4976-b2e5-404431d3389a/dns-node-resolver/0.log" Jul 13 15:27:28.858365 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:28.858321 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qfs4l_df5174e4-503e-475b-ab01-20dd9f45124d/node-ca/0.log" Jul 13 15:27:30.031248 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:30.031210 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678zbt4t_dfe27dc5-31d2-4bda-bddd-a1995f68e78b/istio-proxy/0.log" Jul 13 15:27:30.258300 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:30.258266 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-84c9577b7-772ld_d6122581-51d2-48cc-b2f3-8ee6223e7bd7/kube-auth-proxy/0.log" Jul 13 15:27:31.292558 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:31.292527 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mgkcj_d8061d1f-15e7-4f20-a3c6-d3b9b865cb0e/serve-healthcheck-canary/0.log" Jul 13 15:27:31.771483 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:31.771395 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-g54mc/perf-node-gather-daemonset-7slz7" Jul 13 15:27:32.029609 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:32.029558 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fqm8q_a3ae3942-f36d-405f-8c8c-124b903d6273/kube-rbac-proxy/0.log" Jul 13 15:27:32.055420 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:32.055389 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fqm8q_a3ae3942-f36d-405f-8c8c-124b903d6273/exporter/0.log" Jul 13 15:27:32.084643 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:32.084608 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fqm8q_a3ae3942-f36d-405f-8c8c-124b903d6273/extractor/0.log" Jul 13 15:27:35.027956 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:35.027921 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678zbt4t_dfe27dc5-31d2-4bda-bddd-a1995f68e78b/istio-proxy/0.log" Jul 13 15:27:35.147492 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:35.147454 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-4n846_b559bca0-d4cc-4a33-b63a-1c03a6c54a84/server/0.log" Jul 13 15:27:35.248220 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:35.248189 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-84c9577b7-772ld_d6122581-51d2-48cc-b2f3-8ee6223e7bd7/kube-auth-proxy/0.log" Jul 13 15:27:35.299507 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:35.299412 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6496967d77-f5ptc_94eff779-7543-4a52-af77-debeeb23f1f9/manager/0.log" Jul 13 15:27:35.325460 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:35.325411 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6496967d77-f5ptc_94eff779-7543-4a52-af77-debeeb23f1f9/copy-manifests/0.log" Jul 13 15:27:36.624628 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:36.624527 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678zbt4t_dfe27dc5-31d2-4bda-bddd-a1995f68e78b/istio-proxy/0.log" Jul 13 15:27:36.828185 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:36.828145 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-84c9577b7-772ld_d6122581-51d2-48cc-b2f3-8ee6223e7bd7/kube-auth-proxy/0.log" Jul 13 15:27:36.883148 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:36.883060 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-d4d649b46-9stcp_2e347796-e195-4186-ac37-c22c554570c0/manager/0.log" Jul 13 15:27:36.942079 ip-10-0-136-96 kubenswrapper[2568]: I0713 15:27:36.942007 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-n9fx8_e643ce43-0221-45f9-b421-c9320445f31b/openshift-lws-operator/0.log"