Jul 15 16:54:12.978167 ip-10-0-140-149 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 15 16:54:12.978179 ip-10-0-140-149 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 15 16:54:12.978187 ip-10-0-140-149 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 15 16:54:12.978561 ip-10-0-140-149 systemd[1]: Failed to start Kubernetes Kubelet. Jul 15 16:54:22.984794 ip-10-0-140-149 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 15 16:54:22.984814 ip-10-0-140-149 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot ab8e778637554d61be8d41c856ef8f4c -- Jul 15 16:56:20.738913 ip-10-0-140-149 systemd[1]: Starting Kubernetes Kubelet... Jul 15 16:56:21.146170 ip-10-0-140-149 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 16:56:21.146170 ip-10-0-140-149 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 15 16:56:21.146170 ip-10-0-140-149 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 16:56:21.146170 ip-10-0-140-149 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 15 16:56:21.146170 ip-10-0-140-149 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 16:56:21.146170 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.115743 2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 15 16:56:21.146170 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121139 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 16:56:21.146170 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121151 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 16:56:21.146170 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121155 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 16:56:21.146170 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121160 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 16:56:21.146170 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121164 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 16:56:21.146170 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121167 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 16:56:21.146170 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121170 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 16:56:21.147336 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121173 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 16:56:21.147336 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121175 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 16:56:21.147336 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121178 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 16:56:21.147336 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121181 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 16:56:21.147336 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121183 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 16:56:21.147336 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121186 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 16:56:21.147336 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121189 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 15 16:56:21.147336 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121191 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 16:56:21.147336 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121194 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 16:56:21.147336 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121197 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 16:56:21.147336 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121199 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 16:56:21.147336 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121202 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 16:56:21.147336 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121204 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 16:56:21.147336 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121207 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 16:56:21.147336 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121209 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 16:56:21.147336 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121212 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 16:56:21.147336 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121214 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 16:56:21.147336 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121217 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 16:56:21.147336 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121219 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 16:56:21.147336 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121226 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 16:56:21.147862 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121229 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 16:56:21.147862 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121231 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 16:56:21.147862 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121234 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 16:56:21.147862 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121237 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 16:56:21.147862 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121240 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 16:56:21.147862 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121242 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 16:56:21.147862 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121245 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 16:56:21.147862 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121247 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 16:56:21.147862 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121250 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 16:56:21.147862 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121253 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 16:56:21.147862 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121256 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 16:56:21.147862 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121259 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 16:56:21.147862 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121261 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 16:56:21.147862 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121264 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 16:56:21.147862 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121266 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 16:56:21.147862 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121269 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 16:56:21.147862 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121272 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 16:56:21.147862 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121274 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 16:56:21.147862 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121277 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 16:56:21.148472 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121279 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 16:56:21.148472 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121282 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 16:56:21.148472 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121286 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 16:56:21.148472 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121288 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 16:56:21.148472 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121291 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 16:56:21.148472 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121295 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 16:56:21.148472 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121299 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 16:56:21.148472 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121302 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 16:56:21.148472 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121305 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 16:56:21.148472 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121308 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 16:56:21.148472 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121311 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 16:56:21.148472 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121313 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 16:56:21.148472 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121316 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 16:56:21.148472 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121318 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 16:56:21.148472 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121321 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 16:56:21.148472 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121323 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 16:56:21.148472 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121326 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 16:56:21.148472 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121329 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 16:56:21.148472 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121331 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 16:56:21.148472 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121334 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 16:56:21.149061 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121338 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 16:56:21.149061 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121343 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 16:56:21.149061 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121346 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 16:56:21.149061 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121348 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 16:56:21.149061 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121351 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 16:56:21.149061 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121353 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 16:56:21.149061 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121356 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 16:56:21.149061 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121358 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 16:56:21.149061 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121361 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 16:56:21.149061 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121364 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 16:56:21.149061 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121367 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 16:56:21.149061 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121370 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 16:56:21.149061 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121372 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 16:56:21.149061 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121375 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 16:56:21.149061 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121378 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 16:56:21.149061 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121380 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 16:56:21.149061 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121383 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 16:56:21.149061 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121385 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 16:56:21.149061 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121388 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 16:56:21.149619 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121390 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 16:56:21.149619 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121393 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 16:56:21.149619 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121396 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 16:56:21.149619 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121398 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 16:56:21.149619 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121400 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 16:56:21.149619 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121403 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 16:56:21.149619 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121405 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 16:56:21.149619 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121408 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 16:56:21.149619 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121410 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 16:56:21.149619 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121413 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 16:56:21.149619 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121416 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 16:56:21.149619 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121418 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 16:56:21.149619 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121421 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 16:56:21.149619 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.121423 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 16:56:21.149619 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122100 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 16:56:21.149619 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122107 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 16:56:21.149619 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122110 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 16:56:21.149619 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122112 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 16:56:21.149619 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122115 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 16:56:21.149619 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122118 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 16:56:21.343831 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122120 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 16:56:21.343831 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122123 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 16:56:21.343831 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122126 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 16:56:21.343831 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122129 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 16:56:21.343831 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122131 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 16:56:21.343831 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122134 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 16:56:21.343831 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122137 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 16:56:21.343831 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122140 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 16:56:21.343831 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122142 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 16:56:21.343831 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122145 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 16:56:21.343831 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122147 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 16:56:21.343831 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122150 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 16:56:21.343831 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122152 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 16:56:21.343831 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122155 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 16:56:21.343831 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122159 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 16:56:21.343831 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122163 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 16:56:21.343831 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122166 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 16:56:21.343831 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122169 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 16:56:21.343831 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122172 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 16:56:21.183997 ip-10-0-140-149 systemd[1]: Started Kubernetes Kubelet. Jul 15 16:56:21.359169 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122175 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 16:56:21.359169 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122178 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 16:56:21.359169 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122181 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 16:56:21.359169 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122184 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 16:56:21.359169 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122186 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 16:56:21.359169 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122189 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 16:56:21.359169 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122191 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 16:56:21.359169 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122194 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 16:56:21.359169 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122197 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 16:56:21.359169 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122200 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 16:56:21.359169 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122202 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 16:56:21.359169 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122205 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 16:56:21.359169 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122207 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 16:56:21.359169 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122210 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 16:56:21.359169 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122213 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 16:56:21.359169 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122215 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 16:56:21.359169 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122218 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 16:56:21.359169 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122220 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 16:56:21.359169 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122223 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 16:56:21.359169 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122225 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 16:56:21.360045 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122228 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 16:56:21.360045 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122231 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 16:56:21.360045 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122234 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 16:56:21.360045 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122236 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 16:56:21.360045 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122240 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 16:56:21.360045 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122244 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 16:56:21.360045 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122247 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 16:56:21.360045 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122250 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 16:56:21.360045 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122253 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 16:56:21.360045 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122256 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 16:56:21.360045 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122259 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 16:56:21.360045 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122262 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 16:56:21.360045 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122265 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 16:56:21.360045 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122268 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 16:56:21.360045 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122273 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 16:56:21.360045 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122275 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 16:56:21.360045 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122278 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 16:56:21.360045 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122280 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 16:56:21.360045 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122283 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 16:56:21.360045 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122285 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 16:56:21.360838 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122288 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 16:56:21.360838 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122290 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 16:56:21.360838 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122293 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 16:56:21.360838 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122295 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 16:56:21.360838 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122298 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 16:56:21.360838 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122301 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 16:56:21.360838 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122303 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 16:56:21.360838 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122306 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 16:56:21.360838 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122308 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 16:56:21.360838 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122311 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 16:56:21.360838 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122314 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 16:56:21.360838 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122316 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 16:56:21.360838 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122319 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 16:56:21.360838 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122323 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 16:56:21.360838 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122326 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 16:56:21.360838 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122329 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 16:56:21.360838 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122331 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 16:56:21.360838 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122334 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 15 16:56:21.360838 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122336 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 16:56:21.360838 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122339 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 16:56:21.361464 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122341 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 16:56:21.361464 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122344 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 16:56:21.361464 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122346 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 16:56:21.361464 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122349 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 16:56:21.361464 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122351 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 16:56:21.361464 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122353 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 16:56:21.361464 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122356 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 16:56:21.361464 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122359 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 16:56:21.361464 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122362 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 16:56:21.361464 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122364 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 16:56:21.361464 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122367 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 16:56:21.361464 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122369 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 16:56:21.361464 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122372 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 16:56:21.361464 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122374 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 16:56:21.361464 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122775 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 16:56:21.361464 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122780 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 16:56:21.361464 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122783 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 16:56:21.361464 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122787 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 16:56:21.361464 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122790 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 16:56:21.362107 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122792 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 16:56:21.362107 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122795 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 16:56:21.362107 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122798 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 16:56:21.362107 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122801 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 16:56:21.362107 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122804 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 16:56:21.362107 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122806 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 16:56:21.362107 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122809 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 16:56:21.362107 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122812 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 16:56:21.362107 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122817 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 16:56:21.362107 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122821 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 16:56:21.362107 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122824 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 16:56:21.362107 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122826 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 16:56:21.362107 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122829 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 16:56:21.362107 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122832 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 16:56:21.362107 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122834 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 16:56:21.362107 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122837 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 16:56:21.362107 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122839 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 16:56:21.362107 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122843 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 16:56:21.362107 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122847 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 16:56:21.362107 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122850 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 16:56:21.362993 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122852 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 16:56:21.362993 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122855 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 16:56:21.362993 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122858 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 16:56:21.362993 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122861 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 16:56:21.362993 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122863 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 16:56:21.362993 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122866 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 16:56:21.362993 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122869 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 16:56:21.362993 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122871 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 16:56:21.362993 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122874 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 16:56:21.362993 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122876 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 16:56:21.362993 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122879 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 16:56:21.362993 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122882 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 16:56:21.362993 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122886 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 16:56:21.362993 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122888 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 16:56:21.362993 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122891 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 16:56:21.362993 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122896 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 16:56:21.362993 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122900 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 16:56:21.362993 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122903 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 16:56:21.362993 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122906 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 16:56:21.363596 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122909 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 16:56:21.363596 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122913 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 16:56:21.363596 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122915 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 16:56:21.363596 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122918 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 16:56:21.363596 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122921 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 16:56:21.363596 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122923 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 16:56:21.363596 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122926 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 16:56:21.363596 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122928 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 16:56:21.363596 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122931 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 16:56:21.363596 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122934 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 16:56:21.363596 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122936 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 16:56:21.363596 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122939 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 16:56:21.363596 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122941 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 16:56:21.363596 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122944 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 16:56:21.363596 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122946 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 16:56:21.363596 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122949 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 16:56:21.363596 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122951 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 16:56:21.363596 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122954 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 16:56:21.363596 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122956 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 16:56:21.363596 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122959 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 16:56:21.364332 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122962 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 16:56:21.364332 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122964 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 16:56:21.364332 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122966 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 15 16:56:21.364332 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122969 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 16:56:21.364332 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122972 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 16:56:21.364332 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122974 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 16:56:21.364332 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122976 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 16:56:21.364332 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122979 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 16:56:21.364332 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122982 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 16:56:21.364332 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122985 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 16:56:21.364332 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122988 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 16:56:21.364332 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122991 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 16:56:21.364332 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122993 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 16:56:21.364332 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.122998 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 16:56:21.364332 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.123001 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 16:56:21.364332 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.123004 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 16:56:21.364332 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.123006 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 16:56:21.364332 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.123008 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 16:56:21.364332 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.123011 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 16:56:21.364332 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.123014 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 16:56:21.365170 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.123016 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 16:56:21.365170 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.123019 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 16:56:21.365170 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.123021 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 16:56:21.365170 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.123024 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 16:56:21.365170 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.123026 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 16:56:21.365170 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.123029 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 16:56:21.365170 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.123031 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 16:56:21.365170 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.123034 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 16:56:21.365170 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.123036 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 16:56:21.365170 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.123039 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 16:56:21.365170 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.123041 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 16:56:21.365170 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.123045 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 16:56:21.365170 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.123049 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 16:56:21.365170 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.123052 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 16:56:21.365170 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.123054 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 16:56:21.365170 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123130 2578 flags.go:64] FLAG: --address="0.0.0.0" Jul 15 16:56:21.365170 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123141 2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 15 16:56:21.365170 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123812 2578 flags.go:64] FLAG: --anonymous-auth="true" Jul 15 16:56:21.365170 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123817 2578 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 15 16:56:21.365170 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123821 2578 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 15 16:56:21.365874 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123826 2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 15 16:56:21.365874 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123831 2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 15 16:56:21.365874 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123835 2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 15 16:56:21.365874 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123840 2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 15 16:56:21.365874 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123843 2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 15 16:56:21.365874 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123848 2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 15 16:56:21.365874 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123852 2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 15 16:56:21.365874 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123855 2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 15 16:56:21.365874 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123858 2578 flags.go:64] FLAG: --cgroup-root="" Jul 15 16:56:21.365874 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123861 2578 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 15 16:56:21.365874 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123864 2578 flags.go:64] FLAG: --client-ca-file="" Jul 15 16:56:21.365874 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123867 2578 flags.go:64] FLAG: --cloud-provider="external" Jul 15 16:56:21.365874 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123870 2578 flags.go:64] FLAG: --cluster-dns="[]" Jul 15 16:56:21.365874 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123874 2578 flags.go:64] FLAG: --cluster-domain="" Jul 15 16:56:21.365874 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123877 2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 15 16:56:21.365874 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123880 2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 15 16:56:21.365874 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123883 2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 15 16:56:21.365874 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123887 2578 flags.go:64] FLAG: --container-log-max-files="5" Jul 15 16:56:21.365874 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123890 2578 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 15 16:56:21.365874 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123893 2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 15 16:56:21.365874 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123896 2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 15 16:56:21.365874 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123900 2578 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 15 16:56:21.365874 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123903 2578 flags.go:64] FLAG: --contention-profiling="false" Jul 15 16:56:21.365874 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123906 2578 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 15 16:56:21.366579 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123909 2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 15 16:56:21.366579 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123912 2578 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 15 16:56:21.366579 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123919 2578 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 15 16:56:21.366579 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123923 2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 15 16:56:21.366579 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123926 2578 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 15 16:56:21.366579 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123929 2578 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 15 16:56:21.366579 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123932 2578 flags.go:64] FLAG: --enable-load-reader="false" Jul 15 16:56:21.366579 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123935 2578 flags.go:64] FLAG: --enable-server="true" Jul 15 16:56:21.366579 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123940 2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 15 16:56:21.366579 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123944 2578 flags.go:64] FLAG: --event-burst="100" Jul 15 16:56:21.366579 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123947 2578 flags.go:64] FLAG: --event-qps="50" Jul 15 16:56:21.366579 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123950 2578 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 15 16:56:21.366579 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123953 2578 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 15 16:56:21.366579 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123958 2578 flags.go:64] FLAG: --eviction-hard="" Jul 15 16:56:21.366579 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123962 2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 15 16:56:21.366579 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123965 2578 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 15 16:56:21.366579 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123968 2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 15 16:56:21.366579 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123971 2578 flags.go:64] FLAG: --eviction-soft="" Jul 15 16:56:21.366579 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123974 2578 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 15 16:56:21.366579 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123977 2578 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 15 16:56:21.366579 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123979 2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 15 16:56:21.366579 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123982 2578 flags.go:64] FLAG: --experimental-mounter-path="" Jul 15 16:56:21.366579 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123985 2578 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 15 16:56:21.366579 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123988 2578 flags.go:64] FLAG: --fail-swap-on="true" Jul 15 16:56:21.366579 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123991 2578 flags.go:64] FLAG: --feature-gates="" Jul 15 16:56:21.367355 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123994 2578 flags.go:64] FLAG: --file-check-frequency="20s" Jul 15 16:56:21.367355 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.123998 2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 15 16:56:21.367355 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124001 2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 15 16:56:21.367355 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124004 2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 15 16:56:21.367355 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124007 2578 flags.go:64] FLAG: --healthz-port="10248" Jul 15 16:56:21.367355 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124010 2578 flags.go:64] FLAG: --help="false" Jul 15 16:56:21.367355 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124013 2578 flags.go:64] FLAG: --hostname-override="ip-10-0-140-149.ec2.internal" Jul 15 16:56:21.367355 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124016 2578 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 15 16:56:21.367355 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124020 2578 flags.go:64] FLAG: --http-check-frequency="20s" Jul 15 16:56:21.367355 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124023 2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 15 16:56:21.367355 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124026 2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 15 16:56:21.367355 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124029 2578 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 15 16:56:21.367355 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124032 2578 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 15 16:56:21.367355 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124035 2578 flags.go:64] FLAG: --image-service-endpoint="" Jul 15 16:56:21.367355 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124038 2578 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 15 16:56:21.367355 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124042 2578 flags.go:64] FLAG: --kube-api-burst="100" Jul 15 16:56:21.367355 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124045 2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 15 16:56:21.367355 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124048 2578 flags.go:64] FLAG: --kube-api-qps="50" Jul 15 16:56:21.367355 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124051 2578 flags.go:64] FLAG: --kube-reserved="" Jul 15 16:56:21.367355 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124054 2578 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 15 16:56:21.367355 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124059 2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 15 16:56:21.367355 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124062 2578 flags.go:64] FLAG: --kubelet-cgroups="" Jul 15 16:56:21.367355 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124065 2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 15 16:56:21.367355 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124068 2578 flags.go:64] FLAG: --lock-file="" Jul 15 16:56:21.368040 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124071 2578 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 15 16:56:21.368040 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124074 2578 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 15 16:56:21.368040 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124077 2578 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 15 16:56:21.368040 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124082 2578 flags.go:64] FLAG: --log-json-split-stream="false" Jul 15 16:56:21.368040 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124085 2578 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 15 16:56:21.368040 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124088 2578 flags.go:64] FLAG: --log-text-split-stream="false" Jul 15 16:56:21.368040 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124090 2578 flags.go:64] FLAG: --logging-format="text" Jul 15 16:56:21.368040 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124093 2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 15 16:56:21.368040 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124097 2578 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 15 16:56:21.368040 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124099 2578 flags.go:64] FLAG: --manifest-url="" Jul 15 16:56:21.368040 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124102 2578 flags.go:64] FLAG: --manifest-url-header="" Jul 15 16:56:21.368040 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124106 2578 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 15 16:56:21.368040 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124110 2578 flags.go:64] FLAG: --max-open-files="1000000" Jul 15 16:56:21.368040 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124113 2578 flags.go:64] FLAG: --max-pods="110" Jul 15 16:56:21.368040 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124116 2578 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 15 16:56:21.368040 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124119 2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 15 16:56:21.368040 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124122 2578 flags.go:64] FLAG: --memory-manager-policy="None" Jul 15 16:56:21.368040 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124125 2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 15 16:56:21.368040 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124128 2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 15 16:56:21.368040 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124131 2578 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 15 16:56:21.368040 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124134 2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 15 16:56:21.368040 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124142 2578 flags.go:64] FLAG: --node-status-max-images="50" Jul 15 16:56:21.368040 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124144 2578 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 15 16:56:21.368040 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124149 2578 flags.go:64] FLAG: --oom-score-adj="-999" Jul 15 16:56:21.368040 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124152 2578 flags.go:64] FLAG: --pod-cidr="" Jul 15 16:56:21.369026 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124155 2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 15 16:56:21.369026 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124160 2578 flags.go:64] FLAG: --pod-manifest-path="" Jul 15 16:56:21.369026 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124163 2578 flags.go:64] FLAG: --pod-max-pids="-1" Jul 15 16:56:21.369026 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124167 2578 flags.go:64] FLAG: --pods-per-core="0" Jul 15 16:56:21.369026 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124171 2578 flags.go:64] FLAG: --port="10250" Jul 15 16:56:21.369026 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124174 2578 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 15 16:56:21.369026 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124178 2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c7fb0294ce905ef1" Jul 15 16:56:21.369026 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124181 2578 flags.go:64] FLAG: --qos-reserved="" Jul 15 16:56:21.369026 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124184 2578 flags.go:64] FLAG: --read-only-port="10255" Jul 15 16:56:21.369026 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124187 2578 flags.go:64] FLAG: --register-node="true" Jul 15 16:56:21.369026 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124189 2578 flags.go:64] FLAG: --register-with-taints="" Jul 15 16:56:21.369026 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124196 2578 flags.go:64] FLAG: --registry-burst="10" Jul 15 16:56:21.369026 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124198 2578 flags.go:64] FLAG: --registry-qps="5" Jul 15 16:56:21.369026 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124201 2578 flags.go:64] FLAG: --reserved-cpus="" Jul 15 16:56:21.369026 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124204 2578 flags.go:64] FLAG: --reserved-memory="" Jul 15 16:56:21.369026 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124208 2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 15 16:56:21.369026 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124211 2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 15 16:56:21.369026 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124214 2578 flags.go:64] FLAG: --rotate-certificates="false" Jul 15 16:56:21.369026 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124217 2578 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 15 16:56:21.369026 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124219 2578 flags.go:64] FLAG: --runonce="false" Jul 15 16:56:21.369026 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124222 2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 15 16:56:21.369026 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124225 2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 15 16:56:21.369026 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124228 2578 flags.go:64] FLAG: --seccomp-default="false" Jul 15 16:56:21.369026 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124231 2578 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 15 16:56:21.369026 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124234 2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 15 16:56:21.370116 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124237 2578 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 15 16:56:21.370116 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124240 2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 15 16:56:21.370116 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124243 2578 flags.go:64] FLAG: --storage-driver-password="root" Jul 15 16:56:21.370116 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124245 2578 flags.go:64] FLAG: --storage-driver-secure="false" Jul 15 16:56:21.370116 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124248 2578 flags.go:64] FLAG: --storage-driver-table="stats" Jul 15 16:56:21.370116 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124253 2578 flags.go:64] FLAG: --storage-driver-user="root" Jul 15 16:56:21.370116 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124256 2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 15 16:56:21.370116 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124259 2578 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 15 16:56:21.370116 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124262 2578 flags.go:64] FLAG: --system-cgroups="" Jul 15 16:56:21.370116 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124273 2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 15 16:56:21.370116 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124278 2578 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 15 16:56:21.370116 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124283 2578 flags.go:64] FLAG: --tls-cert-file="" Jul 15 16:56:21.370116 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124286 2578 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 15 16:56:21.370116 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124291 2578 flags.go:64] FLAG: --tls-min-version="" Jul 15 16:56:21.370116 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124293 2578 flags.go:64] FLAG: --tls-private-key-file="" Jul 15 16:56:21.370116 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124296 2578 flags.go:64] FLAG: --topology-manager-policy="none" Jul 15 16:56:21.370116 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124299 2578 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 15 16:56:21.370116 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124302 2578 flags.go:64] FLAG: --topology-manager-scope="container" Jul 15 16:56:21.370116 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124305 2578 flags.go:64] FLAG: --v="2" Jul 15 16:56:21.370116 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124309 2578 flags.go:64] FLAG: --version="false" Jul 15 16:56:21.370116 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124313 2578 flags.go:64] FLAG: --vmodule="" Jul 15 16:56:21.370116 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124316 2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 15 16:56:21.370116 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124320 2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 15 16:56:21.370116 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124435 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 16:56:21.370116 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124439 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 16:56:21.371213 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124442 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 16:56:21.371213 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124445 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 16:56:21.371213 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124449 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 16:56:21.371213 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124452 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 16:56:21.371213 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124455 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 16:56:21.371213 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124457 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 16:56:21.371213 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124460 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 16:56:21.371213 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124462 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 16:56:21.371213 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124465 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 16:56:21.371213 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124468 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 16:56:21.371213 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124470 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 16:56:21.371213 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124473 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 16:56:21.371213 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124477 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 16:56:21.371213 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124480 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 16:56:21.371213 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124482 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 16:56:21.371213 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124485 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 16:56:21.371213 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124487 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 16:56:21.371213 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124502 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 16:56:21.371213 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124508 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 16:56:21.371213 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124511 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 16:56:21.372109 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124515 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 16:56:21.372109 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124518 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 16:56:21.372109 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124520 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 16:56:21.372109 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124523 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 16:56:21.372109 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124526 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 16:56:21.372109 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124528 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 16:56:21.372109 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124531 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 16:56:21.372109 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124533 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 16:56:21.372109 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124537 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 16:56:21.372109 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124541 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 16:56:21.372109 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124544 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 16:56:21.372109 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124547 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 16:56:21.372109 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124549 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 16:56:21.372109 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124552 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 16:56:21.372109 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124555 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 16:56:21.372109 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124557 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 16:56:21.372109 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124560 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 16:56:21.372109 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124562 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 16:56:21.372109 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124565 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 16:56:21.372109 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124567 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 16:56:21.373019 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124570 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 16:56:21.373019 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124572 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 16:56:21.373019 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124575 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 16:56:21.373019 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124577 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 16:56:21.373019 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124581 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 16:56:21.373019 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124583 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 16:56:21.373019 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124586 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 16:56:21.373019 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124588 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 16:56:21.373019 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124592 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 16:56:21.373019 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124596 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 16:56:21.373019 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124600 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 16:56:21.373019 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124603 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 16:56:21.373019 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124605 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 16:56:21.373019 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124608 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 16:56:21.373019 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124611 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 16:56:21.373019 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124613 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 16:56:21.373019 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124616 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 16:56:21.373019 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124618 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 16:56:21.373019 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124621 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 16:56:21.373019 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124623 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 16:56:21.373902 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124627 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 16:56:21.373902 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124629 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 16:56:21.373902 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124632 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 15 16:56:21.373902 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124634 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 16:56:21.373902 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124637 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 16:56:21.373902 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124651 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 16:56:21.373902 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124654 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 16:56:21.373902 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124657 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 16:56:21.373902 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124660 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 16:56:21.373902 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124662 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 16:56:21.373902 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124665 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 16:56:21.373902 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124668 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 16:56:21.373902 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124671 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 16:56:21.373902 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124673 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 16:56:21.373902 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124676 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 16:56:21.373902 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124678 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 16:56:21.373902 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124682 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 16:56:21.373902 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124685 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 16:56:21.373902 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124687 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 16:56:21.373902 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124690 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 16:56:21.374795 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124692 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 16:56:21.374795 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124695 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 16:56:21.374795 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124699 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 16:56:21.374795 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124702 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 16:56:21.374795 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124705 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 16:56:21.374795 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124707 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 16:56:21.374795 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124710 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 16:56:21.374795 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124713 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 16:56:21.374795 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124716 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 16:56:21.374795 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124718 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 16:56:21.374795 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124721 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 16:56:21.374795 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124723 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 16:56:21.374795 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124726 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 16:56:21.374795 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124729 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 16:56:21.374795 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124733 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 16:56:21.374795 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124736 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 16:56:21.374795 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.124739 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 16:56:21.375660 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.124744 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 16:56:21.375660 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.131436 2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 15 16:56:21.375660 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.131450 2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 15 16:56:21.375660 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.131467 2578 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 15 16:56:21.375660 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.131475 2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 15 16:56:21.375660 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131530 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 16:56:21.375660 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131536 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 16:56:21.375660 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131540 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 16:56:21.375660 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131544 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 16:56:21.375660 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131547 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 16:56:21.375660 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131550 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 16:56:21.375660 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131553 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 16:56:21.375660 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131556 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 16:56:21.375660 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131559 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 16:56:21.377135 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131562 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 16:56:21.377135 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131565 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 16:56:21.377135 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131568 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 16:56:21.377135 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131570 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 16:56:21.377135 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131573 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 16:56:21.377135 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131575 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 16:56:21.377135 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131578 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 16:56:21.377135 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131581 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 16:56:21.377135 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131583 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 16:56:21.377135 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131586 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 16:56:21.377135 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131588 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 16:56:21.377135 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131591 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 16:56:21.377135 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131593 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 16:56:21.377135 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131595 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 16:56:21.377135 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131599 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 16:56:21.377135 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131602 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 16:56:21.377135 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131605 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 16:56:21.377135 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131607 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 16:56:21.377135 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131610 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 16:56:21.377135 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131612 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 16:56:21.378000 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131616 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 16:56:21.378000 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131620 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 16:56:21.378000 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131623 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 16:56:21.378000 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131626 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 16:56:21.378000 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131629 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 16:56:21.378000 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131632 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 16:56:21.378000 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131635 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 16:56:21.378000 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131637 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 16:56:21.378000 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131656 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 16:56:21.378000 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131659 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 16:56:21.378000 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131661 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 16:56:21.378000 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131664 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 16:56:21.378000 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131667 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 16:56:21.378000 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131670 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 16:56:21.378000 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131672 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 16:56:21.378000 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131675 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 16:56:21.378000 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131678 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 16:56:21.378000 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131681 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 16:56:21.378000 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131683 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 16:56:21.379893 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131686 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 16:56:21.379893 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131689 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 16:56:21.379893 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131691 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 16:56:21.379893 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131694 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 16:56:21.379893 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131696 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 16:56:21.379893 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131699 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 16:56:21.379893 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131701 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 16:56:21.379893 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131703 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 16:56:21.379893 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131706 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 16:56:21.379893 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131709 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 16:56:21.379893 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131711 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 16:56:21.379893 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131714 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 16:56:21.379893 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131717 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 16:56:21.379893 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131719 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 16:56:21.379893 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131723 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 16:56:21.379893 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131726 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 16:56:21.379893 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131728 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 16:56:21.379893 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131731 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 16:56:21.379893 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131734 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 16:56:21.379893 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131737 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 16:56:21.380763 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131739 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 16:56:21.380763 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131742 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 16:56:21.380763 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131744 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 16:56:21.380763 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131747 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 16:56:21.380763 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131749 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 16:56:21.380763 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131753 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 16:56:21.380763 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131757 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 16:56:21.380763 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131760 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 16:56:21.380763 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131763 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 15 16:56:21.380763 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131765 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 16:56:21.380763 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131768 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 16:56:21.380763 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131770 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 16:56:21.380763 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131773 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 16:56:21.380763 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131776 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 16:56:21.380763 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131778 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 16:56:21.380763 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131781 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 16:56:21.380763 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131783 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 16:56:21.380763 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131786 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 16:56:21.380763 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131788 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 16:56:21.380763 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131791 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 16:56:23.157347 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131794 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 16:56:23.157347 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131796 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 16:56:23.157347 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131801 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 16:56:23.157347 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131804 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 16:56:23.157347 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131806 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 16:56:23.157347 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131809 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 16:56:23.157347 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131811 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 16:56:23.157347 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131814 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 16:56:23.157347 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131817 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 16:56:23.157347 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131819 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 16:56:23.157347 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131822 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 16:56:23.157347 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.131829 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 16:56:23.157347 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131939 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 16:56:23.157347 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131944 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 16:56:23.159137 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131947 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 16:56:23.159137 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131950 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 16:56:23.159137 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131954 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 16:56:23.159137 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131957 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 16:56:23.159137 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131960 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 16:56:23.159137 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131963 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 16:56:23.159137 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131966 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 16:56:23.159137 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131968 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 16:56:23.159137 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131971 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 16:56:23.159137 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131974 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 16:56:23.159137 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131977 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 16:56:23.159137 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131979 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 16:56:23.159137 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131982 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 16:56:23.159137 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131984 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 16:56:23.159137 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131987 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 16:56:23.159137 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131989 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 16:56:23.159137 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131993 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 16:56:23.159137 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131996 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 16:56:23.159137 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.131999 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 16:56:23.160205 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132002 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 16:56:23.160205 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132007 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 16:56:23.160205 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132009 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 16:56:23.160205 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132012 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 16:56:23.160205 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132014 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 16:56:23.160205 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132017 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 16:56:23.160205 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132019 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 16:56:23.160205 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132022 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 16:56:23.160205 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132025 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 16:56:23.160205 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132028 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 16:56:23.160205 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132030 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 16:56:23.160205 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132034 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 16:56:23.160205 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132037 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 16:56:23.160205 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132039 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 16:56:23.160205 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132042 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 16:56:23.160205 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132045 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 16:56:23.160205 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132048 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 16:56:23.160205 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132050 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 16:56:23.160205 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132053 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 16:56:23.160205 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132056 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 16:56:23.161287 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132058 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 16:56:23.161287 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132061 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 16:56:23.161287 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132063 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 16:56:23.161287 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132066 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 16:56:23.161287 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132069 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 16:56:23.161287 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132071 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 16:56:23.161287 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132074 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 16:56:23.161287 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132077 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 16:56:23.161287 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132079 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 16:56:23.161287 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132082 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 16:56:23.161287 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132084 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 16:56:23.161287 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132087 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 16:56:23.161287 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132089 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 16:56:23.161287 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132093 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 16:56:23.161287 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132095 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 16:56:23.161287 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132098 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 16:56:23.161287 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132100 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 16:56:23.161287 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132103 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 16:56:23.161287 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132106 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 16:56:23.162088 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132108 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 16:56:23.162088 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132111 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 16:56:23.162088 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132114 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 16:56:23.162088 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132116 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 16:56:23.162088 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132120 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 16:56:23.162088 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132122 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 16:56:23.162088 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132125 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 16:56:23.162088 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132128 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 16:56:23.162088 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132130 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 16:56:23.162088 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132133 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 16:56:23.162088 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132135 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 16:56:23.162088 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132138 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 16:56:23.162088 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132140 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 16:56:23.162088 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132143 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 16:56:23.162088 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132147 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 16:56:23.162088 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132150 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 16:56:23.162088 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132152 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 16:56:23.162088 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132155 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 16:56:23.162088 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132158 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 16:56:23.162088 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132160 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 16:56:23.163587 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132163 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 16:56:23.163587 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132165 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 16:56:23.163587 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132168 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 16:56:23.163587 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132171 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 16:56:23.163587 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132173 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 16:56:23.163587 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132176 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 16:56:23.163587 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132179 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 16:56:23.163587 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132182 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 16:56:23.163587 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132185 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 16:56:23.163587 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132187 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 16:56:23.163587 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132190 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 16:56:23.163587 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132192 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 16:56:23.163587 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132195 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 16:56:23.163587 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132198 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 16:56:23.163587 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132202 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 16:56:23.163587 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132204 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 15 16:56:23.163587 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132210 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 16:56:23.163587 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132212 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 16:56:23.163587 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:21.132215 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 16:56:23.164466 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.132220 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 16:56:23.164466 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.132348 2578 server.go:962] "Client rotation is on, will bootstrap in background" Jul 15 16:56:23.164466 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.134323 2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 15 16:56:23.164466 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.135430 2578 server.go:1019] "Starting client certificate rotation" Jul 15 16:56:23.164466 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.135524 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 16:56:23.164466 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.135553 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 16:56:23.164466 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.139442 2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 16:56:23.164466 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.144834 2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 16:56:23.164466 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.155322 2578 log.go:25] "Validated CRI v1 runtime API" Jul 15 16:56:23.164466 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.157459 2578 log.go:25] "Validated CRI v1 image API" Jul 15 16:56:23.164466 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.159033 2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 15 16:56:23.164466 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.161497 2578 fs.go:135] Filesystem UUIDs: map[3d392fd5-cfcc-48b6-9ed7-c1a16e9e5557:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 ef11e0f4-d603-4b15-9db1-1cb37219db1e:/dev/nvme0n1p3] Jul 15 16:56:23.165116 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.161519 2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 15 16:56:23.165326 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.166706 2578 manager.go:216] Machine: {Timestamp:2026-07-15 16:56:21.166224427 +0000 UTC m=+0.318308609 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3071570 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ed8f5c36165ca357550ced33de7c1 SystemUUID:ec2ed8f5-c361-65ca-3575-50ced33de7c1 BootID:ab8e7786-3755-4d61-be8d-41c856ef8f4c Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:00:6a:9f:00:b5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:00:6a:9f:00:b5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:32:55:22:b8:31:1c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 15 16:56:23.165326 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.166801 2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 15 16:56:23.165326 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.166899 2578 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 15 16:56:23.165326 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.167891 2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 15 16:56:23.165326 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.167913 2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-149.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 15 16:56:23.165326 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.168148 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 16:56:23.165326 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.168515 2578 topology_manager.go:138] "Creating topology manager with none policy" Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.168526 2578 container_manager_linux.go:309] "Creating device plugin manager" Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.168534 2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.168545 2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.169756 2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.172380 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.172493 2578 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.172516 2578 kubelet.go:486] "Attempting to sync node with API server" Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.172529 2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.172543 2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.172551 2578 kubelet.go:398] "Adding apiserver pod source" Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.172559 2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.172883 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.172904 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.176970 2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.178689 2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.178717 2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.178865 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.178877 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.178883 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.178889 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.178897 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.178903 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.179371 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.179377 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.179384 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.179389 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.179397 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.179404 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.180094 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.180101 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.183070 2578 server.go:1268] "Started kubelet" Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.183132 2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.183170 2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.183201 2578 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:21.183532 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.183603 2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-149.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:21.183696 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-149.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.184295 2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 15 16:56:23.166155 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.185512 2578 server.go:310] "Adding debug handlers to kubelet server" Jul 15 16:56:23.167881 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:21.187275 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-149.ec2.internal.18c284f2e6e7da41 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-149.ec2.internal,UID:ip-10-0-140-149.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-149.ec2.internal,},FirstTimestamp:2026-07-15 16:56:21.183044161 +0000 UTC m=+0.335128324,LastTimestamp:2026-07-15 16:56:21.183044161 +0000 UTC m=+0.335128324,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-149.ec2.internal,}" Jul 15 16:56:23.167881 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:21.188892 2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 15 16:56:23.167881 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.189518 2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 15 16:56:23.167881 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.189531 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 15 16:56:23.167881 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:21.189587 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-149.ec2.internal\" not found" Jul 15 16:56:23.167881 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.190231 2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 15 16:56:23.167881 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.190360 2578 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 15 16:56:23.167881 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.190373 2578 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 15 16:56:23.167881 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.190428 2578 reconstruct.go:97] "Volume reconstruction finished" Jul 15 16:56:23.167881 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.190438 2578 reconciler.go:29] "Reconciler: start to sync state" Jul 15 16:56:23.167881 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.190472 2578 factory.go:55] Registering systemd factory Jul 15 16:56:23.167881 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.190485 2578 factory.go:223] Registration of the systemd container factory successfully Jul 15 16:56:23.167881 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.190701 2578 factory.go:153] Registering CRI-O factory Jul 15 16:56:23.167881 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.190714 2578 factory.go:223] Registration of the crio container factory successfully Jul 15 16:56:23.167881 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.190760 2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 15 16:56:23.167881 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.190780 2578 factory.go:103] Registering Raw factory Jul 15 16:56:23.167881 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.190794 2578 manager.go:1196] Started watching for new ooms in manager Jul 15 16:56:23.167881 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.191143 2578 manager.go:349] Starting recovery of all containers Jul 15 16:56:23.167881 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:21.198408 2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-149.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 15 16:56:23.167881 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:21.198484 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 15 16:56:23.167881 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.198486 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 15 16:56:23.167881 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.205737 2578 manager.go:354] Recovery completed Jul 15 16:56:23.167881 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.210067 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:56:23.167881 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.211755 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-149.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:56:23.167881 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.211785 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-149.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:56:23.167881 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.211797 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-149.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:56:23.167881 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.212236 2578 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 15 16:56:23.167881 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.212246 2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 15 16:56:23.167881 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.212261 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 15 16:56:23.167881 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.212287 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:56:23.167881 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.213025 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-149.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:56:23.167881 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.213046 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-149.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:56:23.167881 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.213056 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-149.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:56:23.169605 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.213317 2578 policy_none.go:49] "None policy: Start" Jul 15 16:56:23.169605 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.213334 2578 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 15 16:56:23.169605 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.213348 2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 15 16:56:23.169605 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:21.213827 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-149.ec2.internal.18c284f2e89e29f5 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-149.ec2.internal,UID:ip-10-0-140-149.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-149.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-149.ec2.internal,},FirstTimestamp:2026-07-15 16:56:21.211769333 +0000 UTC m=+0.363853498,LastTimestamp:2026-07-15 16:56:21.211769333 +0000 UTC m=+0.363853498,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-149.ec2.internal,}" Jul 15 16:56:23.169605 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.214289 2578 policy_none.go:47] "Start" Jul 15 16:56:23.169605 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.216727 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-s6hrw" Jul 15 16:56:23.169605 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:21.221840 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-149.ec2.internal.18c284f2e89e7f4f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-149.ec2.internal,UID:ip-10-0-140-149.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-140-149.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-140-149.ec2.internal,},FirstTimestamp:2026-07-15 16:56:21.211791183 +0000 UTC m=+0.363875343,LastTimestamp:2026-07-15 16:56:21.211791183 +0000 UTC m=+0.363875343,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-149.ec2.internal,}" Jul 15 16:56:23.169605 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.225025 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-s6hrw" Jul 15 16:56:23.169605 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.250189 2578 manager.go:337] "Starting Device Plugin manager" Jul 15 16:56:23.169605 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:21.250356 2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 15 16:56:23.169605 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.250368 2578 server.go:87] "Starting device plugin registration server" Jul 15 16:56:23.169605 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.250608 2578 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 15 16:56:23.169605 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.250620 2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 15 16:56:23.169605 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.250708 2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 15 16:56:23.169605 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.250789 2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 15 16:56:23.169605 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.250797 2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 15 16:56:23.169605 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:21.251211 2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 15 16:56:23.169605 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:21.251259 2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-149.ec2.internal\" not found" Jul 15 16:56:23.169605 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.351106 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:56:23.169605 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.352083 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-149.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:56:23.169605 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.352112 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-149.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:56:23.169605 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.352124 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-149.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:56:23.169605 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.352148 2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-149.ec2.internal" Jul 15 16:56:23.169605 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.361794 2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-149.ec2.internal" Jul 15 16:56:23.169605 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:21.361814 2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-149.ec2.internal\": node \"ip-10-0-140-149.ec2.internal\" not found" Jul 15 16:56:23.169605 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:21.384411 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-149.ec2.internal\" not found" Jul 15 16:56:23.170886 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.385080 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 15 16:56:23.170886 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.385119 2578 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 15 16:56:23.170886 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.385143 2578 kubelet.go:2448] "Starting kubelet main sync loop" Jul 15 16:56:23.170886 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:21.385176 2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 15 16:56:23.170886 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:21.387508 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 16:56:23.170886 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:23.084026 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-149.ec2.internal\" not found" Jul 15 16:56:23.170886 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.085100 2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-149.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-149.ec2.internal"] Jul 15 16:56:23.170886 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.085189 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:56:23.170886 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.086996 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-149.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:56:23.170886 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.087024 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-149.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:56:23.170886 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.087034 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-149.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:56:23.170886 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.089238 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:56:23.170886 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.089354 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-149.ec2.internal" Jul 15 16:56:23.170886 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.089378 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:56:23.170886 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.090014 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-149.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:56:23.170886 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.090040 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-149.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:56:23.170886 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.090049 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-149.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:56:23.170886 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.090614 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-149.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:56:23.170886 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.090635 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-149.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:56:23.170886 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.090647 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-149.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:56:23.170886 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.090632 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/97c17808d0380dfd6329669266413115-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-149.ec2.internal\" (UID: \"97c17808d0380dfd6329669266413115\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-149.ec2.internal" Jul 15 16:56:23.170886 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.090733 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/97c17808d0380dfd6329669266413115-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-149.ec2.internal\" (UID: \"97c17808d0380dfd6329669266413115\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-149.ec2.internal" Jul 15 16:56:23.170886 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.090762 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3ab212e854df387dd5d9339a36377d05-config\") pod \"kube-apiserver-proxy-ip-10-0-140-149.ec2.internal\" (UID: \"3ab212e854df387dd5d9339a36377d05\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-149.ec2.internal" Jul 15 16:56:23.170886 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.092178 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-149.ec2.internal" Jul 15 16:56:23.170886 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.092206 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:56:23.170886 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.092921 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-149.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:56:23.170886 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.092943 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-149.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:56:23.170886 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.092958 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-149.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:56:23.172615 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:23.119214 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-149.ec2.internal\" not found" node="ip-10-0-140-149.ec2.internal" Jul 15 16:56:23.172615 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:23.123367 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-149.ec2.internal\" not found" node="ip-10-0-140-149.ec2.internal" Jul 15 16:56:23.381353 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:23.184627 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-149.ec2.internal\" not found" Jul 15 16:56:23.381353 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.191028 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/97c17808d0380dfd6329669266413115-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-149.ec2.internal\" (UID: \"97c17808d0380dfd6329669266413115\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-149.ec2.internal" Jul 15 16:56:23.381353 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.191048 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/97c17808d0380dfd6329669266413115-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-149.ec2.internal\" (UID: \"97c17808d0380dfd6329669266413115\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-149.ec2.internal" Jul 15 16:56:23.381353 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.191064 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3ab212e854df387dd5d9339a36377d05-config\") pod \"kube-apiserver-proxy-ip-10-0-140-149.ec2.internal\" (UID: \"3ab212e854df387dd5d9339a36377d05\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-149.ec2.internal" Jul 15 16:56:23.381353 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.191095 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3ab212e854df387dd5d9339a36377d05-config\") pod \"kube-apiserver-proxy-ip-10-0-140-149.ec2.internal\" (UID: \"3ab212e854df387dd5d9339a36377d05\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-149.ec2.internal" Jul 15 16:56:23.381353 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.191113 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/97c17808d0380dfd6329669266413115-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-149.ec2.internal\" (UID: \"97c17808d0380dfd6329669266413115\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-149.ec2.internal" Jul 15 16:56:23.381353 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.191125 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/97c17808d0380dfd6329669266413115-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-149.ec2.internal\" (UID: \"97c17808d0380dfd6329669266413115\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-149.ec2.internal" Jul 15 16:56:23.381353 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:23.285455 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-149.ec2.internal\" not found" Jul 15 16:56:23.576692 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:23.385940 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-149.ec2.internal\" not found" Jul 15 16:56:23.576692 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.421423 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-149.ec2.internal" Jul 15 16:56:23.576692 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.426288 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-149.ec2.internal" Jul 15 16:56:23.576692 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:23.486365 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-149.ec2.internal\" not found" Jul 15 16:56:23.756409 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:23.587854 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-149.ec2.internal\" not found" Jul 15 16:56:23.756409 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:23.688654 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-149.ec2.internal\" not found" Jul 15 16:56:23.756409 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.734616 2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 15 16:56:23.756409 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.734940 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 15 16:56:23.959208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.767710 2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 16:56:23.959208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.771885 2578 apiserver.go:52] "Watching apiserver" Jul 15 16:56:23.959208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.776759 2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 16:56:23.959208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.780413 2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 15 16:56:23.959208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.780729 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-g2kh8","openshift-ovn-kubernetes/ovnkube-node-29wtx","openshift-cluster-node-tuning-operator/tuned-zbvt7","openshift-dns/node-resolver-cvlbn","openshift-multus/multus-7c67r","openshift-multus/multus-additional-cni-plugins-tv5fg","openshift-network-diagnostics/network-check-target-r6txx","kube-system/global-pull-secret-syncer-nzk7p","kube-system/konnectivity-agent-9whs6","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kmrqt","openshift-image-registry/node-ca-whrn6","openshift-multus/network-metrics-daemon-62hbp"] Jul 15 16:56:23.959208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.783232 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.959208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.787898 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 16:56:23.959208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.790056 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 15 16:56:23.959208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.790104 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 15 16:56:23.959208 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:23.790082 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-62hbp" podUID="df1447df-1b9a-4dc3-ba2c-ecbab1514d09" Jul 15 16:56:23.959208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.790157 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-66hnv\"" Jul 15 16:56:23.959208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.790317 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 15 16:56:23.959208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.790515 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-149.ec2.internal" Jul 15 16:56:23.959208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.791119 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-g2kh8" Jul 15 16:56:23.959208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.793285 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 15 16:56:23.959208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.793477 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-727ht\"" Jul 15 16:56:23.959208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.793480 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 15 16:56:23.959208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.793506 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 15 16:56:23.959208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.793537 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:23.959208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.793929 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/12f6b5f7-a4ba-492b-b150-aedd91c64e53-etc-modprobe-d\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.959208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.793947 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/12f6b5f7-a4ba-492b-b150-aedd91c64e53-etc-sysconfig\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.959208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.793969 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/12f6b5f7-a4ba-492b-b150-aedd91c64e53-etc-systemd\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.959208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.793984 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/12f6b5f7-a4ba-492b-b150-aedd91c64e53-run\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.961578 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.794003 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ffc75c8d-b600-4be4-a811-6d57e6f81ea3-host-slash\") pod \"iptables-alerter-g2kh8\" (UID: \"ffc75c8d-b600-4be4-a811-6d57e6f81ea3\") " pod="openshift-network-operator/iptables-alerter-g2kh8" Jul 15 16:56:23.961578 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.794035 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/12f6b5f7-a4ba-492b-b150-aedd91c64e53-etc-kubernetes\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.961578 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.794049 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/12f6b5f7-a4ba-492b-b150-aedd91c64e53-etc-sysctl-d\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.961578 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.794062 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/12f6b5f7-a4ba-492b-b150-aedd91c64e53-etc-sysctl-conf\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.961578 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.794076 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12f6b5f7-a4ba-492b-b150-aedd91c64e53-tmp\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.961578 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.794095 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-metrics-certs\") pod \"network-metrics-daemon-62hbp\" (UID: \"df1447df-1b9a-4dc3-ba2c-ecbab1514d09\") " pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 16:56:23.961578 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.794112 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsv7w\" (UniqueName: \"kubernetes.io/projected/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-kube-api-access-wsv7w\") pod \"network-metrics-daemon-62hbp\" (UID: \"df1447df-1b9a-4dc3-ba2c-ecbab1514d09\") " pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 16:56:23.961578 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.794126 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/12f6b5f7-a4ba-492b-b150-aedd91c64e53-sys\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.961578 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.794150 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/12f6b5f7-a4ba-492b-b150-aedd91c64e53-var-lib-kubelet\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.961578 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.794170 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/12f6b5f7-a4ba-492b-b150-aedd91c64e53-host\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.961578 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.794190 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/12f6b5f7-a4ba-492b-b150-aedd91c64e53-etc-tuned\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.961578 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.794205 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xpbj\" (UniqueName: \"kubernetes.io/projected/12f6b5f7-a4ba-492b-b150-aedd91c64e53-kube-api-access-5xpbj\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.961578 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.794220 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ffc75c8d-b600-4be4-a811-6d57e6f81ea3-iptables-alerter-script\") pod \"iptables-alerter-g2kh8\" (UID: \"ffc75c8d-b600-4be4-a811-6d57e6f81ea3\") " pod="openshift-network-operator/iptables-alerter-g2kh8" Jul 15 16:56:23.961578 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.794235 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjqv7\" (UniqueName: \"kubernetes.io/projected/ffc75c8d-b600-4be4-a811-6d57e6f81ea3-kube-api-access-pjqv7\") pod \"iptables-alerter-g2kh8\" (UID: \"ffc75c8d-b600-4be4-a811-6d57e6f81ea3\") " pod="openshift-network-operator/iptables-alerter-g2kh8" Jul 15 16:56:23.962208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.794263 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/12f6b5f7-a4ba-492b-b150-aedd91c64e53-lib-modules\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.962208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.795723 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cvlbn" Jul 15 16:56:23.962208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.795868 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 15 16:56:23.962208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.796092 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 15 16:56:23.962208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.796105 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 15 16:56:23.962208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.796135 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 15 16:56:23.962208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.796393 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 15 16:56:23.962208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.796445 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 15 16:56:23.962208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.796457 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pdknh\"" Jul 15 16:56:23.962208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.797921 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7c67r" Jul 15 16:56:23.962208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.798029 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 15 16:56:23.962208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.798262 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-8xhf5\"" Jul 15 16:56:23.962208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.798329 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 15 16:56:23.962208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.800176 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tv5fg" Jul 15 16:56:23.962208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.802739 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 15 16:56:23.962208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.802866 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 15 16:56:23.962208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.802900 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 15 16:56:23.962208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.802921 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 15 16:56:23.962208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.803039 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 15 16:56:23.962208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.803065 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r6txx" Jul 15 16:56:23.962208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.803112 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-g9xvl\"" Jul 15 16:56:23.962208 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:23.803119 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r6txx" podUID="1542bfc7-eb45-488a-b867-257a2978997b" Jul 15 16:56:23.962208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.803263 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qkkz6\"" Jul 15 16:56:23.962208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.803357 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 15 16:56:23.962208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.804287 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:56:23.962208 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:23.804384 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzk7p" podUID="b25e2334-b15b-4c03-9754-cd9a14f177cc" Jul 15 16:56:23.962208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.805245 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 16:56:23.962208 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.806977 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9whs6" Jul 15 16:56:23.963339 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.807295 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 16:56:23.963339 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.807357 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-149.ec2.internal" Jul 15 16:56:23.963339 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.809159 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kmrqt" Jul 15 16:56:23.963339 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.809522 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 15 16:56:23.963339 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.809573 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 15 16:56:23.963339 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.809611 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-2stph\"" Jul 15 16:56:23.963339 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.811288 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-149.ec2.internal"] Jul 15 16:56:23.963339 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.811349 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-whrn6" Jul 15 16:56:23.963339 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.811527 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 15 16:56:23.963339 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.811649 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 15 16:56:23.963339 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.811842 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-9p622\"" Jul 15 16:56:23.963339 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.811863 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 15 16:56:23.963339 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.813547 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 15 16:56:23.963339 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.813882 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 15 16:56:23.963339 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.814003 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zdhrx\"" Jul 15 16:56:23.963339 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.814383 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 15 16:56:23.963339 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.816636 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 16:56:23.963339 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.816825 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-140-149.ec2.internal"] Jul 15 16:56:23.963339 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.827372 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 16:51:22 +0000 UTC" deadline="2028-01-08 20:51:27.888578119 +0000 UTC" Jul 15 16:56:23.963339 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.827398 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13011h55m4.061182153s" Jul 15 16:56:23.963339 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.833642 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-f4w5c" Jul 15 16:56:23.963339 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.839507 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-f4w5c" Jul 15 16:56:23.963339 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.890875 2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 15 16:56:23.963339 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895247 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a488304f-ae1d-4dd5-9af5-a1906ce229a9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tv5fg\" (UID: \"a488304f-ae1d-4dd5-9af5-a1906ce229a9\") " pod="openshift-multus/multus-additional-cni-plugins-tv5fg" Jul 15 16:56:23.963339 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895276 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/55622740-75bd-4532-bda0-e3fb5aa35185-konnectivity-ca\") pod \"konnectivity-agent-9whs6\" (UID: \"55622740-75bd-4532-bda0-e3fb5aa35185\") " pod="kube-system/konnectivity-agent-9whs6" Jul 15 16:56:23.963339 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895305 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsv7w\" (UniqueName: \"kubernetes.io/projected/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-kube-api-access-wsv7w\") pod \"network-metrics-daemon-62hbp\" (UID: \"df1447df-1b9a-4dc3-ba2c-ecbab1514d09\") " pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 16:56:23.964256 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895328 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a488304f-ae1d-4dd5-9af5-a1906ce229a9-cnibin\") pod \"multus-additional-cni-plugins-tv5fg\" (UID: \"a488304f-ae1d-4dd5-9af5-a1906ce229a9\") " pod="openshift-multus/multus-additional-cni-plugins-tv5fg" Jul 15 16:56:23.964256 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895350 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a488304f-ae1d-4dd5-9af5-a1906ce229a9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tv5fg\" (UID: \"a488304f-ae1d-4dd5-9af5-a1906ce229a9\") " pod="openshift-multus/multus-additional-cni-plugins-tv5fg" Jul 15 16:56:23.964256 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895371 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7d41273f-cc9a-4a08-9225-3a89453afeea-device-dir\") pod \"aws-ebs-csi-driver-node-kmrqt\" (UID: \"7d41273f-cc9a-4a08-9225-3a89453afeea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kmrqt" Jul 15 16:56:23.964256 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895399 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/88f8928c-237e-457f-a032-707f48111f36-ovnkube-config\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:23.964256 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895421 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/88f8928c-237e-457f-a032-707f48111f36-ovnkube-script-lib\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:23.964256 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895437 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-multus-conf-dir\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:23.964256 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895458 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7d41273f-cc9a-4a08-9225-3a89453afeea-registration-dir\") pod \"aws-ebs-csi-driver-node-kmrqt\" (UID: \"7d41273f-cc9a-4a08-9225-3a89453afeea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kmrqt" Jul 15 16:56:23.964256 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895474 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7d41273f-cc9a-4a08-9225-3a89453afeea-etc-selinux\") pod \"aws-ebs-csi-driver-node-kmrqt\" (UID: \"7d41273f-cc9a-4a08-9225-3a89453afeea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kmrqt" Jul 15 16:56:23.964256 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895489 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-multus-socket-dir-parent\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:23.964256 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895503 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-host-var-lib-cni-bin\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:23.964256 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895518 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-etc-kubernetes\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:23.964256 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895547 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d0867616-074a-4b55-94ce-df0e85196692-host\") pod \"node-ca-whrn6\" (UID: \"d0867616-074a-4b55-94ce-df0e85196692\") " pod="openshift-image-registry/node-ca-whrn6" Jul 15 16:56:23.964256 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895566 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/12f6b5f7-a4ba-492b-b150-aedd91c64e53-etc-systemd\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.964256 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895581 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-host-run-netns\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:23.965113 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895609 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-cnibin\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:23.965113 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895623 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b25e2334-b15b-4c03-9754-cd9a14f177cc-dbus\") pod \"global-pull-secret-syncer-nzk7p\" (UID: \"b25e2334-b15b-4c03-9754-cd9a14f177cc\") " pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:56:23.965113 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895666 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12f6b5f7-a4ba-492b-b150-aedd91c64e53-tmp\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.965113 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895673 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/12f6b5f7-a4ba-492b-b150-aedd91c64e53-etc-systemd\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.965113 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895703 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-etc-openvswitch\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:23.965113 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895718 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-host-cni-bin\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:23.965113 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895743 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-host-run-k8s-cni-cncf-io\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:23.965113 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895758 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx7gk\" (UniqueName: \"kubernetes.io/projected/1542bfc7-eb45-488a-b867-257a2978997b-kube-api-access-gx7gk\") pod \"network-check-target-r6txx\" (UID: \"1542bfc7-eb45-488a-b867-257a2978997b\") " pod="openshift-network-diagnostics/network-check-target-r6txx" Jul 15 16:56:23.965113 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895784 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7d41273f-cc9a-4a08-9225-3a89453afeea-sys-fs\") pod \"aws-ebs-csi-driver-node-kmrqt\" (UID: \"7d41273f-cc9a-4a08-9225-3a89453afeea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kmrqt" Jul 15 16:56:23.965113 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895800 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/12f6b5f7-a4ba-492b-b150-aedd91c64e53-var-lib-kubelet\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.965113 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895816 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/12f6b5f7-a4ba-492b-b150-aedd91c64e53-etc-tuned\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.965113 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895866 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xpbj\" (UniqueName: \"kubernetes.io/projected/12f6b5f7-a4ba-492b-b150-aedd91c64e53-kube-api-access-5xpbj\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.965113 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895882 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ffc75c8d-b600-4be4-a811-6d57e6f81ea3-iptables-alerter-script\") pod \"iptables-alerter-g2kh8\" (UID: \"ffc75c8d-b600-4be4-a811-6d57e6f81ea3\") " pod="openshift-network-operator/iptables-alerter-g2kh8" Jul 15 16:56:23.965113 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895901 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-host-var-lib-cni-multus\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:23.965866 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895921 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a488304f-ae1d-4dd5-9af5-a1906ce229a9-os-release\") pod \"multus-additional-cni-plugins-tv5fg\" (UID: \"a488304f-ae1d-4dd5-9af5-a1906ce229a9\") " pod="openshift-multus/multus-additional-cni-plugins-tv5fg" Jul 15 16:56:23.965866 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895925 2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 15 16:56:23.965866 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895943 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7d41273f-cc9a-4a08-9225-3a89453afeea-socket-dir\") pod \"aws-ebs-csi-driver-node-kmrqt\" (UID: \"7d41273f-cc9a-4a08-9225-3a89453afeea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kmrqt" Jul 15 16:56:23.965866 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895959 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/12f6b5f7-a4ba-492b-b150-aedd91c64e53-lib-modules\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.965866 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895973 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-systemd-units\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:23.965866 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.895987 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-host-var-lib-kubelet\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:23.965866 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896002 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b25e2334-b15b-4c03-9754-cd9a14f177cc-original-pull-secret\") pod \"global-pull-secret-syncer-nzk7p\" (UID: \"b25e2334-b15b-4c03-9754-cd9a14f177cc\") " pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:56:23.965866 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896055 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/12f6b5f7-a4ba-492b-b150-aedd91c64e53-etc-sysconfig\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.965866 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896070 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-system-cni-dir\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:23.965866 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896086 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-host-run-netns\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:23.965866 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896106 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/16c5ca09-925c-4046-8ade-ae582e290339-multus-daemon-config\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:23.965866 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896137 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a488304f-ae1d-4dd5-9af5-a1906ce229a9-cni-binary-copy\") pod \"multus-additional-cni-plugins-tv5fg\" (UID: \"a488304f-ae1d-4dd5-9af5-a1906ce229a9\") " pod="openshift-multus/multus-additional-cni-plugins-tv5fg" Jul 15 16:56:23.965866 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896161 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9wh7\" (UniqueName: \"kubernetes.io/projected/7d41273f-cc9a-4a08-9225-3a89453afeea-kube-api-access-k9wh7\") pod \"aws-ebs-csi-driver-node-kmrqt\" (UID: \"7d41273f-cc9a-4a08-9225-3a89453afeea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kmrqt" Jul 15 16:56:23.965866 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896186 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/12f6b5f7-a4ba-492b-b150-aedd91c64e53-etc-kubernetes\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.966961 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896209 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/12f6b5f7-a4ba-492b-b150-aedd91c64e53-etc-sysctl-d\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.966961 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896231 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/12f6b5f7-a4ba-492b-b150-aedd91c64e53-etc-sysctl-conf\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.966961 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896255 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-metrics-certs\") pod \"network-metrics-daemon-62hbp\" (UID: \"df1447df-1b9a-4dc3-ba2c-ecbab1514d09\") " pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 16:56:23.966961 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896281 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-run-openvswitch\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:23.966961 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896304 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-node-log\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:23.966961 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896330 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-host-run-ovn-kubernetes\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:23.966961 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896353 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/12f6b5f7-a4ba-492b-b150-aedd91c64e53-sys\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.966961 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896378 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/12f6b5f7-a4ba-492b-b150-aedd91c64e53-host\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.966961 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896404 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjqv7\" (UniqueName: \"kubernetes.io/projected/ffc75c8d-b600-4be4-a811-6d57e6f81ea3-kube-api-access-pjqv7\") pod \"iptables-alerter-g2kh8\" (UID: \"ffc75c8d-b600-4be4-a811-6d57e6f81ea3\") " pod="openshift-network-operator/iptables-alerter-g2kh8" Jul 15 16:56:23.966961 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896538 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-run-systemd\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:23.966961 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896557 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/88f8928c-237e-457f-a032-707f48111f36-env-overrides\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:23.966961 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896573 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzprq\" (UniqueName: \"kubernetes.io/projected/88f8928c-237e-457f-a032-707f48111f36-kube-api-access-rzprq\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:23.966961 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896609 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b25e2334-b15b-4c03-9754-cd9a14f177cc-kubelet-config\") pod \"global-pull-secret-syncer-nzk7p\" (UID: \"b25e2334-b15b-4c03-9754-cd9a14f177cc\") " pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:56:23.966961 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896626 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7d41273f-cc9a-4a08-9225-3a89453afeea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kmrqt\" (UID: \"7d41273f-cc9a-4a08-9225-3a89453afeea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kmrqt" Jul 15 16:56:23.967502 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896641 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-host-slash\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:23.967502 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896655 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-var-lib-openvswitch\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:23.967502 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896671 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-host-cni-netd\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:23.967502 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896694 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlspn\" (UniqueName: \"kubernetes.io/projected/16c5ca09-925c-4046-8ade-ae582e290339-kube-api-access-nlspn\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:23.967502 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896714 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d0867616-074a-4b55-94ce-df0e85196692-serviceca\") pod \"node-ca-whrn6\" (UID: \"d0867616-074a-4b55-94ce-df0e85196692\") " pod="openshift-image-registry/node-ca-whrn6" Jul 15 16:56:23.967502 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896729 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qr6sn\" (UniqueName: \"kubernetes.io/projected/d0867616-074a-4b55-94ce-df0e85196692-kube-api-access-qr6sn\") pod \"node-ca-whrn6\" (UID: \"d0867616-074a-4b55-94ce-df0e85196692\") " pod="openshift-image-registry/node-ca-whrn6" Jul 15 16:56:23.967502 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896745 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/12f6b5f7-a4ba-492b-b150-aedd91c64e53-run\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.967502 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896759 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/88f8928c-237e-457f-a032-707f48111f36-ovn-node-metrics-cert\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:23.967502 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896774 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-multus-cni-dir\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:23.967502 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896799 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-os-release\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:23.967502 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896838 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-host-run-multus-certs\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:23.967502 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896856 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a488304f-ae1d-4dd5-9af5-a1906ce229a9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tv5fg\" (UID: \"a488304f-ae1d-4dd5-9af5-a1906ce229a9\") " pod="openshift-multus/multus-additional-cni-plugins-tv5fg" Jul 15 16:56:23.967502 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896890 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9c2ba93-cd7a-4aa9-81f4-b5afeee112db-tmp-dir\") pod \"node-resolver-cvlbn\" (UID: \"c9c2ba93-cd7a-4aa9-81f4-b5afeee112db\") " pod="openshift-dns/node-resolver-cvlbn" Jul 15 16:56:23.967502 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896909 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dx8x\" (UniqueName: \"kubernetes.io/projected/a488304f-ae1d-4dd5-9af5-a1906ce229a9-kube-api-access-4dx8x\") pod \"multus-additional-cni-plugins-tv5fg\" (UID: \"a488304f-ae1d-4dd5-9af5-a1906ce229a9\") " pod="openshift-multus/multus-additional-cni-plugins-tv5fg" Jul 15 16:56:23.968148 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896929 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-run-ovn\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:23.968148 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896944 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-log-socket\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:23.968148 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896960 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c9c2ba93-cd7a-4aa9-81f4-b5afeee112db-hosts-file\") pod \"node-resolver-cvlbn\" (UID: \"c9c2ba93-cd7a-4aa9-81f4-b5afeee112db\") " pod="openshift-dns/node-resolver-cvlbn" Jul 15 16:56:23.968148 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896979 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-hostroot\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:23.968148 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.896994 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a488304f-ae1d-4dd5-9af5-a1906ce229a9-system-cni-dir\") pod \"multus-additional-cni-plugins-tv5fg\" (UID: \"a488304f-ae1d-4dd5-9af5-a1906ce229a9\") " pod="openshift-multus/multus-additional-cni-plugins-tv5fg" Jul 15 16:56:23.968148 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.897009 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gsrx\" (UniqueName: \"kubernetes.io/projected/c9c2ba93-cd7a-4aa9-81f4-b5afeee112db-kube-api-access-8gsrx\") pod \"node-resolver-cvlbn\" (UID: \"c9c2ba93-cd7a-4aa9-81f4-b5afeee112db\") " pod="openshift-dns/node-resolver-cvlbn" Jul 15 16:56:23.968148 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.897024 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/16c5ca09-925c-4046-8ade-ae582e290339-cni-binary-copy\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:23.968148 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.897039 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/55622740-75bd-4532-bda0-e3fb5aa35185-agent-certs\") pod \"konnectivity-agent-9whs6\" (UID: \"55622740-75bd-4532-bda0-e3fb5aa35185\") " pod="kube-system/konnectivity-agent-9whs6" Jul 15 16:56:23.968148 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.897057 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/12f6b5f7-a4ba-492b-b150-aedd91c64e53-etc-modprobe-d\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.968148 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.897074 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ffc75c8d-b600-4be4-a811-6d57e6f81ea3-host-slash\") pod \"iptables-alerter-g2kh8\" (UID: \"ffc75c8d-b600-4be4-a811-6d57e6f81ea3\") " pod="openshift-network-operator/iptables-alerter-g2kh8" Jul 15 16:56:23.968148 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.897094 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-host-kubelet\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:23.968148 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.897110 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:23.968148 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.897785 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ffc75c8d-b600-4be4-a811-6d57e6f81ea3-iptables-alerter-script\") pod \"iptables-alerter-g2kh8\" (UID: \"ffc75c8d-b600-4be4-a811-6d57e6f81ea3\") " pod="openshift-network-operator/iptables-alerter-g2kh8" Jul 15 16:56:23.968148 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.897882 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/12f6b5f7-a4ba-492b-b150-aedd91c64e53-lib-modules\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.968620 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.897915 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/12f6b5f7-a4ba-492b-b150-aedd91c64e53-etc-sysconfig\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.968620 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.897949 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/12f6b5f7-a4ba-492b-b150-aedd91c64e53-etc-kubernetes\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.968620 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.898002 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/12f6b5f7-a4ba-492b-b150-aedd91c64e53-etc-sysctl-d\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.968620 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.898063 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/12f6b5f7-a4ba-492b-b150-aedd91c64e53-etc-sysctl-conf\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.968620 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:23.898114 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:56:23.968620 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:23.898163 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-metrics-certs podName:df1447df-1b9a-4dc3-ba2c-ecbab1514d09 nodeName:}" failed. No retries permitted until 2026-07-15 16:56:24.398149123 +0000 UTC m=+1.951279320 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-metrics-certs") pod "network-metrics-daemon-62hbp" (UID: "df1447df-1b9a-4dc3-ba2c-ecbab1514d09") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:56:23.968620 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.898312 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/12f6b5f7-a4ba-492b-b150-aedd91c64e53-sys\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.968620 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.898338 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/12f6b5f7-a4ba-492b-b150-aedd91c64e53-host\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.968620 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.898401 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/12f6b5f7-a4ba-492b-b150-aedd91c64e53-run\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.968620 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.898468 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/12f6b5f7-a4ba-492b-b150-aedd91c64e53-etc-modprobe-d\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.968620 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.898491 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ffc75c8d-b600-4be4-a811-6d57e6f81ea3-host-slash\") pod \"iptables-alerter-g2kh8\" (UID: \"ffc75c8d-b600-4be4-a811-6d57e6f81ea3\") " pod="openshift-network-operator/iptables-alerter-g2kh8" Jul 15 16:56:23.968620 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.898548 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/12f6b5f7-a4ba-492b-b150-aedd91c64e53-var-lib-kubelet\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.968620 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.898849 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/12f6b5f7-a4ba-492b-b150-aedd91c64e53-etc-tuned\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.968620 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.898878 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12f6b5f7-a4ba-492b-b150-aedd91c64e53-tmp\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:23.968620 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.905517 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjqv7\" (UniqueName: \"kubernetes.io/projected/ffc75c8d-b600-4be4-a811-6d57e6f81ea3-kube-api-access-pjqv7\") pod \"iptables-alerter-g2kh8\" (UID: \"ffc75c8d-b600-4be4-a811-6d57e6f81ea3\") " pod="openshift-network-operator/iptables-alerter-g2kh8" Jul 15 16:56:24.171069 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.907526 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xpbj\" (UniqueName: \"kubernetes.io/projected/12f6b5f7-a4ba-492b-b150-aedd91c64e53-kube-api-access-5xpbj\") pod \"tuned-zbvt7\" (UID: \"12f6b5f7-a4ba-492b-b150-aedd91c64e53\") " pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:24.171069 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.907875 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsv7w\" (UniqueName: \"kubernetes.io/projected/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-kube-api-access-wsv7w\") pod \"network-metrics-daemon-62hbp\" (UID: \"df1447df-1b9a-4dc3-ba2c-ecbab1514d09\") " pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 16:56:24.171069 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.998099 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/88f8928c-237e-457f-a032-707f48111f36-ovn-node-metrics-cert\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.171069 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.998144 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-multus-cni-dir\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:24.171069 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.998176 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-os-release\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:24.171069 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.998206 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-host-run-multus-certs\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:24.171069 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.998233 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a488304f-ae1d-4dd5-9af5-a1906ce229a9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tv5fg\" (UID: \"a488304f-ae1d-4dd5-9af5-a1906ce229a9\") " pod="openshift-multus/multus-additional-cni-plugins-tv5fg" Jul 15 16:56:24.171069 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.998267 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9c2ba93-cd7a-4aa9-81f4-b5afeee112db-tmp-dir\") pod \"node-resolver-cvlbn\" (UID: \"c9c2ba93-cd7a-4aa9-81f4-b5afeee112db\") " pod="openshift-dns/node-resolver-cvlbn" Jul 15 16:56:24.171069 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.998296 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dx8x\" (UniqueName: \"kubernetes.io/projected/a488304f-ae1d-4dd5-9af5-a1906ce229a9-kube-api-access-4dx8x\") pod \"multus-additional-cni-plugins-tv5fg\" (UID: \"a488304f-ae1d-4dd5-9af5-a1906ce229a9\") " pod="openshift-multus/multus-additional-cni-plugins-tv5fg" Jul 15 16:56:24.171069 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.998324 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-run-ovn\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.171069 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.998347 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-log-socket\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.171069 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.998375 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c9c2ba93-cd7a-4aa9-81f4-b5afeee112db-hosts-file\") pod \"node-resolver-cvlbn\" (UID: \"c9c2ba93-cd7a-4aa9-81f4-b5afeee112db\") " pod="openshift-dns/node-resolver-cvlbn" Jul 15 16:56:24.171069 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.998403 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-hostroot\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:24.171069 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.998433 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a488304f-ae1d-4dd5-9af5-a1906ce229a9-system-cni-dir\") pod \"multus-additional-cni-plugins-tv5fg\" (UID: \"a488304f-ae1d-4dd5-9af5-a1906ce229a9\") " pod="openshift-multus/multus-additional-cni-plugins-tv5fg" Jul 15 16:56:24.172288 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.998467 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gsrx\" (UniqueName: \"kubernetes.io/projected/c9c2ba93-cd7a-4aa9-81f4-b5afeee112db-kube-api-access-8gsrx\") pod \"node-resolver-cvlbn\" (UID: \"c9c2ba93-cd7a-4aa9-81f4-b5afeee112db\") " pod="openshift-dns/node-resolver-cvlbn" Jul 15 16:56:24.172288 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.998492 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/16c5ca09-925c-4046-8ade-ae582e290339-cni-binary-copy\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:24.172288 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.998521 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/55622740-75bd-4532-bda0-e3fb5aa35185-agent-certs\") pod \"konnectivity-agent-9whs6\" (UID: \"55622740-75bd-4532-bda0-e3fb5aa35185\") " pod="kube-system/konnectivity-agent-9whs6" Jul 15 16:56:24.172288 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.998554 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-host-kubelet\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.172288 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.998584 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.172288 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.998631 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a488304f-ae1d-4dd5-9af5-a1906ce229a9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tv5fg\" (UID: \"a488304f-ae1d-4dd5-9af5-a1906ce229a9\") " pod="openshift-multus/multus-additional-cni-plugins-tv5fg" Jul 15 16:56:24.172288 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.998656 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/55622740-75bd-4532-bda0-e3fb5aa35185-konnectivity-ca\") pod \"konnectivity-agent-9whs6\" (UID: \"55622740-75bd-4532-bda0-e3fb5aa35185\") " pod="kube-system/konnectivity-agent-9whs6" Jul 15 16:56:24.172288 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.998688 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a488304f-ae1d-4dd5-9af5-a1906ce229a9-cnibin\") pod \"multus-additional-cni-plugins-tv5fg\" (UID: \"a488304f-ae1d-4dd5-9af5-a1906ce229a9\") " pod="openshift-multus/multus-additional-cni-plugins-tv5fg" Jul 15 16:56:24.172288 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.998719 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a488304f-ae1d-4dd5-9af5-a1906ce229a9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tv5fg\" (UID: \"a488304f-ae1d-4dd5-9af5-a1906ce229a9\") " pod="openshift-multus/multus-additional-cni-plugins-tv5fg" Jul 15 16:56:24.172288 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.998750 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7d41273f-cc9a-4a08-9225-3a89453afeea-device-dir\") pod \"aws-ebs-csi-driver-node-kmrqt\" (UID: \"7d41273f-cc9a-4a08-9225-3a89453afeea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kmrqt" Jul 15 16:56:24.172288 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.998775 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/88f8928c-237e-457f-a032-707f48111f36-ovnkube-config\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.172288 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.998806 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/88f8928c-237e-457f-a032-707f48111f36-ovnkube-script-lib\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.172288 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.998836 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-multus-conf-dir\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:24.172288 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.998865 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7d41273f-cc9a-4a08-9225-3a89453afeea-registration-dir\") pod \"aws-ebs-csi-driver-node-kmrqt\" (UID: \"7d41273f-cc9a-4a08-9225-3a89453afeea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kmrqt" Jul 15 16:56:24.173088 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.998898 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7d41273f-cc9a-4a08-9225-3a89453afeea-etc-selinux\") pod \"aws-ebs-csi-driver-node-kmrqt\" (UID: \"7d41273f-cc9a-4a08-9225-3a89453afeea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kmrqt" Jul 15 16:56:24.173088 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.998926 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-multus-socket-dir-parent\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:24.173088 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.998967 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-host-var-lib-cni-bin\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:24.173088 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.998997 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-etc-kubernetes\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:24.173088 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.999026 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d0867616-074a-4b55-94ce-df0e85196692-host\") pod \"node-ca-whrn6\" (UID: \"d0867616-074a-4b55-94ce-df0e85196692\") " pod="openshift-image-registry/node-ca-whrn6" Jul 15 16:56:24.173088 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.999058 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-host-run-netns\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.173088 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.999082 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-cnibin\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:24.173088 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.999112 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b25e2334-b15b-4c03-9754-cd9a14f177cc-dbus\") pod \"global-pull-secret-syncer-nzk7p\" (UID: \"b25e2334-b15b-4c03-9754-cd9a14f177cc\") " pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:56:24.173088 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.999142 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-etc-openvswitch\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.173088 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.999173 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-host-cni-bin\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.173088 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.999198 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-host-run-k8s-cni-cncf-io\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:24.173088 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.999235 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx7gk\" (UniqueName: \"kubernetes.io/projected/1542bfc7-eb45-488a-b867-257a2978997b-kube-api-access-gx7gk\") pod \"network-check-target-r6txx\" (UID: \"1542bfc7-eb45-488a-b867-257a2978997b\") " pod="openshift-network-diagnostics/network-check-target-r6txx" Jul 15 16:56:24.173088 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.999265 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7d41273f-cc9a-4a08-9225-3a89453afeea-sys-fs\") pod \"aws-ebs-csi-driver-node-kmrqt\" (UID: \"7d41273f-cc9a-4a08-9225-3a89453afeea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kmrqt" Jul 15 16:56:24.173088 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.999300 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-host-var-lib-cni-multus\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:24.173088 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.999331 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a488304f-ae1d-4dd5-9af5-a1906ce229a9-os-release\") pod \"multus-additional-cni-plugins-tv5fg\" (UID: \"a488304f-ae1d-4dd5-9af5-a1906ce229a9\") " pod="openshift-multus/multus-additional-cni-plugins-tv5fg" Jul 15 16:56:24.173643 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.999356 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7d41273f-cc9a-4a08-9225-3a89453afeea-socket-dir\") pod \"aws-ebs-csi-driver-node-kmrqt\" (UID: \"7d41273f-cc9a-4a08-9225-3a89453afeea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kmrqt" Jul 15 16:56:24.173643 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.999389 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-systemd-units\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.173643 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.999421 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-host-var-lib-kubelet\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:24.173643 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.999451 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b25e2334-b15b-4c03-9754-cd9a14f177cc-original-pull-secret\") pod \"global-pull-secret-syncer-nzk7p\" (UID: \"b25e2334-b15b-4c03-9754-cd9a14f177cc\") " pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:56:24.173643 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.999482 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-system-cni-dir\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:24.173643 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.999510 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-host-run-netns\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:24.173643 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.999538 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/16c5ca09-925c-4046-8ade-ae582e290339-multus-daemon-config\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:24.173643 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.999568 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a488304f-ae1d-4dd5-9af5-a1906ce229a9-cni-binary-copy\") pod \"multus-additional-cni-plugins-tv5fg\" (UID: \"a488304f-ae1d-4dd5-9af5-a1906ce229a9\") " pod="openshift-multus/multus-additional-cni-plugins-tv5fg" Jul 15 16:56:24.173643 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.999614 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9wh7\" (UniqueName: \"kubernetes.io/projected/7d41273f-cc9a-4a08-9225-3a89453afeea-kube-api-access-k9wh7\") pod \"aws-ebs-csi-driver-node-kmrqt\" (UID: \"7d41273f-cc9a-4a08-9225-3a89453afeea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kmrqt" Jul 15 16:56:24.173643 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.999671 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-run-openvswitch\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.173643 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.999702 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-node-log\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.173643 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.999736 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-host-run-ovn-kubernetes\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.173643 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.999765 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-run-systemd\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.173643 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.999793 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/88f8928c-237e-457f-a032-707f48111f36-env-overrides\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.174890 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.999824 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzprq\" (UniqueName: \"kubernetes.io/projected/88f8928c-237e-457f-a032-707f48111f36-kube-api-access-rzprq\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.174890 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.999855 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b25e2334-b15b-4c03-9754-cd9a14f177cc-kubelet-config\") pod \"global-pull-secret-syncer-nzk7p\" (UID: \"b25e2334-b15b-4c03-9754-cd9a14f177cc\") " pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:56:24.174890 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.999885 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7d41273f-cc9a-4a08-9225-3a89453afeea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kmrqt\" (UID: \"7d41273f-cc9a-4a08-9225-3a89453afeea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kmrqt" Jul 15 16:56:24.174890 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.999910 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-host-slash\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.174890 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.999939 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-var-lib-openvswitch\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.174890 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:23.999971 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-host-cni-netd\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.174890 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.000000 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlspn\" (UniqueName: \"kubernetes.io/projected/16c5ca09-925c-4046-8ade-ae582e290339-kube-api-access-nlspn\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:24.174890 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.000034 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d0867616-074a-4b55-94ce-df0e85196692-serviceca\") pod \"node-ca-whrn6\" (UID: \"d0867616-074a-4b55-94ce-df0e85196692\") " pod="openshift-image-registry/node-ca-whrn6" Jul 15 16:56:24.174890 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.000060 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qr6sn\" (UniqueName: \"kubernetes.io/projected/d0867616-074a-4b55-94ce-df0e85196692-kube-api-access-qr6sn\") pod \"node-ca-whrn6\" (UID: \"d0867616-074a-4b55-94ce-df0e85196692\") " pod="openshift-image-registry/node-ca-whrn6" Jul 15 16:56:24.174890 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.000157 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-run-openvswitch\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.174890 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.000230 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-host-run-ovn-kubernetes\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.174890 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.000242 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-host-cni-bin\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.174890 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.000369 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-host-var-lib-kubelet\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:24.174890 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.000811 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b25e2334-b15b-4c03-9754-cd9a14f177cc-dbus\") pod \"global-pull-secret-syncer-nzk7p\" (UID: \"b25e2334-b15b-4c03-9754-cd9a14f177cc\") " pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:56:24.174890 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.000691 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d0867616-074a-4b55-94ce-df0e85196692-host\") pod \"node-ca-whrn6\" (UID: \"d0867616-074a-4b55-94ce-df0e85196692\") " pod="openshift-image-registry/node-ca-whrn6" Jul 15 16:56:24.175760 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.000625 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7d41273f-cc9a-4a08-9225-3a89453afeea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kmrqt\" (UID: \"7d41273f-cc9a-4a08-9225-3a89453afeea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kmrqt" Jul 15 16:56:24.175760 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.000700 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-system-cni-dir\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:24.175760 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.000717 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-host-run-netns\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:24.175760 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.000724 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-os-release\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:24.175760 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.000741 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a488304f-ae1d-4dd5-9af5-a1906ce229a9-os-release\") pod \"multus-additional-cni-plugins-tv5fg\" (UID: \"a488304f-ae1d-4dd5-9af5-a1906ce229a9\") " pod="openshift-multus/multus-additional-cni-plugins-tv5fg" Jul 15 16:56:24.175760 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.000806 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7d41273f-cc9a-4a08-9225-3a89453afeea-socket-dir\") pod \"aws-ebs-csi-driver-node-kmrqt\" (UID: \"7d41273f-cc9a-4a08-9225-3a89453afeea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kmrqt" Jul 15 16:56:24.175760 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.000817 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-multus-cni-dir\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:24.175760 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.000854 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-var-lib-openvswitch\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.175760 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.000901 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-host-var-lib-cni-multus\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:24.175760 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.000913 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-host-run-k8s-cni-cncf-io\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:24.175760 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.000914 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-host-run-netns\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.175760 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.000941 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-host-slash\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.175760 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.000948 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-node-log\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.175760 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.000982 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-etc-kubernetes\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:24.175760 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.001005 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b25e2334-b15b-4c03-9754-cd9a14f177cc-kubelet-config\") pod \"global-pull-secret-syncer-nzk7p\" (UID: \"b25e2334-b15b-4c03-9754-cd9a14f177cc\") " pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:56:24.176610 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.001022 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-cnibin\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:24.176610 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.001031 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/88f8928c-237e-457f-a032-707f48111f36-env-overrides\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.176610 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.001045 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-run-systemd\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.176610 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.001064 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-systemd-units\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.176610 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.001077 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-etc-openvswitch\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.176610 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.001068 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7d41273f-cc9a-4a08-9225-3a89453afeea-sys-fs\") pod \"aws-ebs-csi-driver-node-kmrqt\" (UID: \"7d41273f-cc9a-4a08-9225-3a89453afeea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kmrqt" Jul 15 16:56:24.176610 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.001102 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-host-cni-netd\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.176610 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.001121 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-host-run-multus-certs\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:24.176610 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.001265 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a488304f-ae1d-4dd5-9af5-a1906ce229a9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tv5fg\" (UID: \"a488304f-ae1d-4dd5-9af5-a1906ce229a9\") " pod="openshift-multus/multus-additional-cni-plugins-tv5fg" Jul 15 16:56:24.176610 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.001403 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a488304f-ae1d-4dd5-9af5-a1906ce229a9-cni-binary-copy\") pod \"multus-additional-cni-plugins-tv5fg\" (UID: \"a488304f-ae1d-4dd5-9af5-a1906ce229a9\") " pod="openshift-multus/multus-additional-cni-plugins-tv5fg" Jul 15 16:56:24.176610 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.001437 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a488304f-ae1d-4dd5-9af5-a1906ce229a9-system-cni-dir\") pod \"multus-additional-cni-plugins-tv5fg\" (UID: \"a488304f-ae1d-4dd5-9af5-a1906ce229a9\") " pod="openshift-multus/multus-additional-cni-plugins-tv5fg" Jul 15 16:56:24.176610 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.001443 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d0867616-074a-4b55-94ce-df0e85196692-serviceca\") pod \"node-ca-whrn6\" (UID: \"d0867616-074a-4b55-94ce-df0e85196692\") " pod="openshift-image-registry/node-ca-whrn6" Jul 15 16:56:24.176610 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.001477 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-run-ovn\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.176610 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.001463 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-host-kubelet\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.176610 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.001519 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-log-socket\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.177853 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.001563 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a488304f-ae1d-4dd5-9af5-a1906ce229a9-cnibin\") pod \"multus-additional-cni-plugins-tv5fg\" (UID: \"a488304f-ae1d-4dd5-9af5-a1906ce229a9\") " pod="openshift-multus/multus-additional-cni-plugins-tv5fg" Jul 15 16:56:24.177853 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:24.001572 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 16:56:24.177853 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.001572 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a488304f-ae1d-4dd5-9af5-a1906ce229a9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tv5fg\" (UID: \"a488304f-ae1d-4dd5-9af5-a1906ce229a9\") " pod="openshift-multus/multus-additional-cni-plugins-tv5fg" Jul 15 16:56:24.177853 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:24.001645 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b25e2334-b15b-4c03-9754-cd9a14f177cc-original-pull-secret podName:b25e2334-b15b-4c03-9754-cd9a14f177cc nodeName:}" failed. No retries permitted until 2026-07-15 16:56:24.501628504 +0000 UTC m=+2.054758697 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b25e2334-b15b-4c03-9754-cd9a14f177cc-original-pull-secret") pod "global-pull-secret-syncer-nzk7p" (UID: "b25e2334-b15b-4c03-9754-cd9a14f177cc") : object "kube-system"/"original-pull-secret" not registered Jul 15 16:56:24.177853 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.001645 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-hostroot\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:24.177853 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.001675 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-host-var-lib-cni-bin\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:24.177853 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.001684 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7d41273f-cc9a-4a08-9225-3a89453afeea-etc-selinux\") pod \"aws-ebs-csi-driver-node-kmrqt\" (UID: \"7d41273f-cc9a-4a08-9225-3a89453afeea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kmrqt" Jul 15 16:56:24.177853 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.001718 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-multus-socket-dir-parent\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:24.177853 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.001728 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c9c2ba93-cd7a-4aa9-81f4-b5afeee112db-hosts-file\") pod \"node-resolver-cvlbn\" (UID: \"c9c2ba93-cd7a-4aa9-81f4-b5afeee112db\") " pod="openshift-dns/node-resolver-cvlbn" Jul 15 16:56:24.177853 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.001758 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/16c5ca09-925c-4046-8ade-ae582e290339-multus-conf-dir\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:24.177853 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.001782 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/16c5ca09-925c-4046-8ade-ae582e290339-cni-binary-copy\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:24.177853 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.001800 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7d41273f-cc9a-4a08-9225-3a89453afeea-device-dir\") pod \"aws-ebs-csi-driver-node-kmrqt\" (UID: \"7d41273f-cc9a-4a08-9225-3a89453afeea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kmrqt" Jul 15 16:56:24.177853 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.001836 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88f8928c-237e-457f-a032-707f48111f36-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.177853 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.001854 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9c2ba93-cd7a-4aa9-81f4-b5afeee112db-tmp-dir\") pod \"node-resolver-cvlbn\" (UID: \"c9c2ba93-cd7a-4aa9-81f4-b5afeee112db\") " pod="openshift-dns/node-resolver-cvlbn" Jul 15 16:56:24.178645 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.001886 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7d41273f-cc9a-4a08-9225-3a89453afeea-registration-dir\") pod \"aws-ebs-csi-driver-node-kmrqt\" (UID: \"7d41273f-cc9a-4a08-9225-3a89453afeea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kmrqt" Jul 15 16:56:24.178645 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.002029 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/16c5ca09-925c-4046-8ade-ae582e290339-multus-daemon-config\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:24.178645 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.002109 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/88f8928c-237e-457f-a032-707f48111f36-ovnkube-script-lib\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.178645 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.002433 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a488304f-ae1d-4dd5-9af5-a1906ce229a9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tv5fg\" (UID: \"a488304f-ae1d-4dd5-9af5-a1906ce229a9\") " pod="openshift-multus/multus-additional-cni-plugins-tv5fg" Jul 15 16:56:24.178645 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.002543 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/88f8928c-237e-457f-a032-707f48111f36-ovnkube-config\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.178645 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.003171 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/55622740-75bd-4532-bda0-e3fb5aa35185-konnectivity-ca\") pod \"konnectivity-agent-9whs6\" (UID: \"55622740-75bd-4532-bda0-e3fb5aa35185\") " pod="kube-system/konnectivity-agent-9whs6" Jul 15 16:56:24.178645 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.003554 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/88f8928c-237e-457f-a032-707f48111f36-ovn-node-metrics-cert\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.178645 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.003689 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/55622740-75bd-4532-bda0-e3fb5aa35185-agent-certs\") pod \"konnectivity-agent-9whs6\" (UID: \"55622740-75bd-4532-bda0-e3fb5aa35185\") " pod="kube-system/konnectivity-agent-9whs6" Jul 15 16:56:24.178645 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:24.012451 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 16:56:24.178645 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:24.012468 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 16:56:24.178645 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:24.012478 2578 projected.go:196] Error preparing data for projected volume kube-api-access-gx7gk for pod openshift-network-diagnostics/network-check-target-r6txx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:56:24.178645 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:24.012533 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1542bfc7-eb45-488a-b867-257a2978997b-kube-api-access-gx7gk podName:1542bfc7-eb45-488a-b867-257a2978997b nodeName:}" failed. No retries permitted until 2026-07-15 16:56:24.512521137 +0000 UTC m=+2.065651329 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-gx7gk" (UniqueName: "kubernetes.io/projected/1542bfc7-eb45-488a-b867-257a2978997b-kube-api-access-gx7gk") pod "network-check-target-r6txx" (UID: "1542bfc7-eb45-488a-b867-257a2978997b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:56:24.178645 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.014625 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9wh7\" (UniqueName: \"kubernetes.io/projected/7d41273f-cc9a-4a08-9225-3a89453afeea-kube-api-access-k9wh7\") pod \"aws-ebs-csi-driver-node-kmrqt\" (UID: \"7d41273f-cc9a-4a08-9225-3a89453afeea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kmrqt" Jul 15 16:56:24.178645 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.015035 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dx8x\" (UniqueName: \"kubernetes.io/projected/a488304f-ae1d-4dd5-9af5-a1906ce229a9-kube-api-access-4dx8x\") pod \"multus-additional-cni-plugins-tv5fg\" (UID: \"a488304f-ae1d-4dd5-9af5-a1906ce229a9\") " pod="openshift-multus/multus-additional-cni-plugins-tv5fg" Jul 15 16:56:24.179406 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.015045 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlspn\" (UniqueName: \"kubernetes.io/projected/16c5ca09-925c-4046-8ade-ae582e290339-kube-api-access-nlspn\") pod \"multus-7c67r\" (UID: \"16c5ca09-925c-4046-8ade-ae582e290339\") " pod="openshift-multus/multus-7c67r" Jul 15 16:56:24.179406 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.016983 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qr6sn\" (UniqueName: \"kubernetes.io/projected/d0867616-074a-4b55-94ce-df0e85196692-kube-api-access-qr6sn\") pod \"node-ca-whrn6\" (UID: \"d0867616-074a-4b55-94ce-df0e85196692\") " pod="openshift-image-registry/node-ca-whrn6" Jul 15 16:56:24.179406 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.017190 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzprq\" (UniqueName: \"kubernetes.io/projected/88f8928c-237e-457f-a032-707f48111f36-kube-api-access-rzprq\") pod \"ovnkube-node-29wtx\" (UID: \"88f8928c-237e-457f-a032-707f48111f36\") " pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.179406 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.020298 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gsrx\" (UniqueName: \"kubernetes.io/projected/c9c2ba93-cd7a-4aa9-81f4-b5afeee112db-kube-api-access-8gsrx\") pod \"node-resolver-cvlbn\" (UID: \"c9c2ba93-cd7a-4aa9-81f4-b5afeee112db\") " pod="openshift-dns/node-resolver-cvlbn" Jul 15 16:56:24.179406 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:24.035691 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97c17808d0380dfd6329669266413115.slice/crio-22dda80847300fe3d2071666ddc2f6036068217d82c5beae9cb768e641a2ed94 WatchSource:0}: Error finding container 22dda80847300fe3d2071666ddc2f6036068217d82c5beae9cb768e641a2ed94: Status 404 returned error can't find the container with id 22dda80847300fe3d2071666ddc2f6036068217d82c5beae9cb768e641a2ed94 Jul 15 16:56:24.179406 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.042895 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 16:56:24.179406 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.122354 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" Jul 15 16:56:24.179406 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:24.131713 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12f6b5f7_a4ba_492b_b150_aedd91c64e53.slice/crio-18ae1ba3e8f74e39228fcc6256675724cdf107172567020dcd916d682866f9f8 WatchSource:0}: Error finding container 18ae1ba3e8f74e39228fcc6256675724cdf107172567020dcd916d682866f9f8: Status 404 returned error can't find the container with id 18ae1ba3e8f74e39228fcc6256675724cdf107172567020dcd916d682866f9f8 Jul 15 16:56:24.179406 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.147107 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-g2kh8" Jul 15 16:56:24.179406 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:24.155560 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffc75c8d_b600_4be4_a811_6d57e6f81ea3.slice/crio-b6afafb89cc11846cd2fed68091693855dca9ead53e00b8f51e2a525b8f68489 WatchSource:0}: Error finding container b6afafb89cc11846cd2fed68091693855dca9ead53e00b8f51e2a525b8f68489: Status 404 returned error can't find the container with id b6afafb89cc11846cd2fed68091693855dca9ead53e00b8f51e2a525b8f68489 Jul 15 16:56:24.179406 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.159331 2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 16:56:24.179406 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.166119 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:24.179406 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:24.175796 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88f8928c_237e_457f_a032_707f48111f36.slice/crio-1fcdd55c53b5c7f4a2d5f3d15547053d090dd8f15cbb0e62bc9498505621cada WatchSource:0}: Error finding container 1fcdd55c53b5c7f4a2d5f3d15547053d090dd8f15cbb0e62bc9498505621cada: Status 404 returned error can't find the container with id 1fcdd55c53b5c7f4a2d5f3d15547053d090dd8f15cbb0e62bc9498505621cada Jul 15 16:56:24.179406 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.176846 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cvlbn" Jul 15 16:56:24.353112 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.195945 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7c67r" Jul 15 16:56:24.353112 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:24.203823 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16c5ca09_925c_4046_8ade_ae582e290339.slice/crio-2d58c14c44e7000f372ce75338e556ae6b450039fb77b838d77f0abcfb4c194e WatchSource:0}: Error finding container 2d58c14c44e7000f372ce75338e556ae6b450039fb77b838d77f0abcfb4c194e: Status 404 returned error can't find the container with id 2d58c14c44e7000f372ce75338e556ae6b450039fb77b838d77f0abcfb4c194e Jul 15 16:56:24.353112 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.220050 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tv5fg" Jul 15 16:56:24.353112 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:24.228172 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda488304f_ae1d_4dd5_9af5_a1906ce229a9.slice/crio-110b0434b6441de527a678ef3a01ee4b7a6970b13aec21192d473bbb490b2475 WatchSource:0}: Error finding container 110b0434b6441de527a678ef3a01ee4b7a6970b13aec21192d473bbb490b2475: Status 404 returned error can't find the container with id 110b0434b6441de527a678ef3a01ee4b7a6970b13aec21192d473bbb490b2475 Jul 15 16:56:24.353112 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.238141 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9whs6" Jul 15 16:56:24.353112 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:24.246045 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55622740_75bd_4532_bda0_e3fb5aa35185.slice/crio-4706fa1c62988063d316719a6e06a8d17f5fa7b5d204f4e495652da8d57fda72 WatchSource:0}: Error finding container 4706fa1c62988063d316719a6e06a8d17f5fa7b5d204f4e495652da8d57fda72: Status 404 returned error can't find the container with id 4706fa1c62988063d316719a6e06a8d17f5fa7b5d204f4e495652da8d57fda72 Jul 15 16:56:24.353112 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.254797 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kmrqt" Jul 15 16:56:24.353112 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.260462 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-whrn6" Jul 15 16:56:24.353112 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:24.262671 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d41273f_cc9a_4a08_9225_3a89453afeea.slice/crio-94156f62b250cb482e65d8ba8c415e4a6c283cf574be65cd0c8a4c81941ec636 WatchSource:0}: Error finding container 94156f62b250cb482e65d8ba8c415e4a6c283cf574be65cd0c8a4c81941ec636: Status 404 returned error can't find the container with id 94156f62b250cb482e65d8ba8c415e4a6c283cf574be65cd0c8a4c81941ec636 Jul 15 16:56:24.353112 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:24.268068 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0867616_074a_4b55_94ce_df0e85196692.slice/crio-666c9b308e722e2bdcecec0fe92961b5624b80f5f7699c8b8cbdb03d40e46844 WatchSource:0}: Error finding container 666c9b308e722e2bdcecec0fe92961b5624b80f5f7699c8b8cbdb03d40e46844: Status 404 returned error can't find the container with id 666c9b308e722e2bdcecec0fe92961b5624b80f5f7699c8b8cbdb03d40e46844 Jul 15 16:56:24.479061 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.409812 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-metrics-certs\") pod \"network-metrics-daemon-62hbp\" (UID: \"df1447df-1b9a-4dc3-ba2c-ecbab1514d09\") " pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 16:56:24.479061 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:24.409940 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:56:24.479061 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:24.409997 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-metrics-certs podName:df1447df-1b9a-4dc3-ba2c-ecbab1514d09 nodeName:}" failed. No retries permitted until 2026-07-15 16:56:25.409978467 +0000 UTC m=+2.963108660 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-metrics-certs") pod "network-metrics-daemon-62hbp" (UID: "df1447df-1b9a-4dc3-ba2c-ecbab1514d09") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:56:24.510329 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.510297 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b25e2334-b15b-4c03-9754-cd9a14f177cc-original-pull-secret\") pod \"global-pull-secret-syncer-nzk7p\" (UID: \"b25e2334-b15b-4c03-9754-cd9a14f177cc\") " pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:56:24.510472 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:24.510413 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 16:56:24.510546 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:24.510488 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b25e2334-b15b-4c03-9754-cd9a14f177cc-original-pull-secret podName:b25e2334-b15b-4c03-9754-cd9a14f177cc nodeName:}" failed. No retries permitted until 2026-07-15 16:56:25.510468724 +0000 UTC m=+3.063598928 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b25e2334-b15b-4c03-9754-cd9a14f177cc-original-pull-secret") pod "global-pull-secret-syncer-nzk7p" (UID: "b25e2334-b15b-4c03-9754-cd9a14f177cc") : object "kube-system"/"original-pull-secret" not registered Jul 15 16:56:24.611095 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.611054 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx7gk\" (UniqueName: \"kubernetes.io/projected/1542bfc7-eb45-488a-b867-257a2978997b-kube-api-access-gx7gk\") pod \"network-check-target-r6txx\" (UID: \"1542bfc7-eb45-488a-b867-257a2978997b\") " pod="openshift-network-diagnostics/network-check-target-r6txx" Jul 15 16:56:24.611278 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:24.611240 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 16:56:24.611278 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:24.611267 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 16:56:24.611278 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:24.611280 2578 projected.go:196] Error preparing data for projected volume kube-api-access-gx7gk for pod openshift-network-diagnostics/network-check-target-r6txx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:56:24.611433 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:24.611340 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1542bfc7-eb45-488a-b867-257a2978997b-kube-api-access-gx7gk podName:1542bfc7-eb45-488a-b867-257a2978997b nodeName:}" failed. No retries permitted until 2026-07-15 16:56:25.611321137 +0000 UTC m=+3.164451348 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-gx7gk" (UniqueName: "kubernetes.io/projected/1542bfc7-eb45-488a-b867-257a2978997b-kube-api-access-gx7gk") pod "network-check-target-r6txx" (UID: "1542bfc7-eb45-488a-b867-257a2978997b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:56:24.633080 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.633050 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 16:56:24.844636 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.844535 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 16:51:23 +0000 UTC" deadline="2028-04-04 19:56:17.136612934 +0000 UTC" Jul 15 16:56:24.844636 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.844578 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15098h59m52.292039236s" Jul 15 16:56:24.986253 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.986226 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r6txx" Jul 15 16:56:24.986398 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:24.986345 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r6txx" podUID="1542bfc7-eb45-488a-b867-257a2978997b" Jul 15 16:56:24.987057 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:24.987035 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 16:56:24.987169 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:24.987140 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-62hbp" podUID="df1447df-1b9a-4dc3-ba2c-ecbab1514d09" Jul 15 16:56:25.009336 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:25.009281 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-whrn6" event={"ID":"d0867616-074a-4b55-94ce-df0e85196692","Type":"ContainerStarted","Data":"666c9b308e722e2bdcecec0fe92961b5624b80f5f7699c8b8cbdb03d40e46844"} Jul 15 16:56:25.016744 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:25.016715 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9whs6" event={"ID":"55622740-75bd-4532-bda0-e3fb5aa35185","Type":"ContainerStarted","Data":"4706fa1c62988063d316719a6e06a8d17f5fa7b5d204f4e495652da8d57fda72"} Jul 15 16:56:25.023952 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:25.023572 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tv5fg" event={"ID":"a488304f-ae1d-4dd5-9af5-a1906ce229a9","Type":"ContainerStarted","Data":"110b0434b6441de527a678ef3a01ee4b7a6970b13aec21192d473bbb490b2475"} Jul 15 16:56:25.032751 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:25.032725 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7c67r" event={"ID":"16c5ca09-925c-4046-8ade-ae582e290339","Type":"ContainerStarted","Data":"2d58c14c44e7000f372ce75338e556ae6b450039fb77b838d77f0abcfb4c194e"} Jul 15 16:56:25.035143 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:25.035117 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cvlbn" event={"ID":"c9c2ba93-cd7a-4aa9-81f4-b5afeee112db","Type":"ContainerStarted","Data":"3310a5e22106281bc8864fbdafd0c3054d26eaacc3735b9f257048f2ff76277b"} Jul 15 16:56:25.037338 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:25.037312 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-g2kh8" event={"ID":"ffc75c8d-b600-4be4-a811-6d57e6f81ea3","Type":"ContainerStarted","Data":"b6afafb89cc11846cd2fed68091693855dca9ead53e00b8f51e2a525b8f68489"} Jul 15 16:56:25.040268 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:25.040242 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" event={"ID":"12f6b5f7-a4ba-492b-b150-aedd91c64e53","Type":"ContainerStarted","Data":"18ae1ba3e8f74e39228fcc6256675724cdf107172567020dcd916d682866f9f8"} Jul 15 16:56:25.042410 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:25.042364 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-149.ec2.internal" event={"ID":"97c17808d0380dfd6329669266413115","Type":"ContainerStarted","Data":"22dda80847300fe3d2071666ddc2f6036068217d82c5beae9cb768e641a2ed94"} Jul 15 16:56:25.049819 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:25.049521 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kmrqt" event={"ID":"7d41273f-cc9a-4a08-9225-3a89453afeea","Type":"ContainerStarted","Data":"94156f62b250cb482e65d8ba8c415e4a6c283cf574be65cd0c8a4c81941ec636"} Jul 15 16:56:25.056490 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:25.056405 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" event={"ID":"88f8928c-237e-457f-a032-707f48111f36","Type":"ContainerStarted","Data":"1fcdd55c53b5c7f4a2d5f3d15547053d090dd8f15cbb0e62bc9498505621cada"} Jul 15 16:56:25.418110 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:25.418073 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-metrics-certs\") pod \"network-metrics-daemon-62hbp\" (UID: \"df1447df-1b9a-4dc3-ba2c-ecbab1514d09\") " pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 16:56:25.418278 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:25.418249 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:56:25.418349 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:25.418311 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-metrics-certs podName:df1447df-1b9a-4dc3-ba2c-ecbab1514d09 nodeName:}" failed. No retries permitted until 2026-07-15 16:56:27.418291971 +0000 UTC m=+4.971422164 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-metrics-certs") pod "network-metrics-daemon-62hbp" (UID: "df1447df-1b9a-4dc3-ba2c-ecbab1514d09") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:56:25.519165 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:25.519129 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b25e2334-b15b-4c03-9754-cd9a14f177cc-original-pull-secret\") pod \"global-pull-secret-syncer-nzk7p\" (UID: \"b25e2334-b15b-4c03-9754-cd9a14f177cc\") " pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:56:25.519356 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:25.519281 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 16:56:25.519356 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:25.519342 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b25e2334-b15b-4c03-9754-cd9a14f177cc-original-pull-secret podName:b25e2334-b15b-4c03-9754-cd9a14f177cc nodeName:}" failed. No retries permitted until 2026-07-15 16:56:27.519323029 +0000 UTC m=+5.072453223 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b25e2334-b15b-4c03-9754-cd9a14f177cc-original-pull-secret") pod "global-pull-secret-syncer-nzk7p" (UID: "b25e2334-b15b-4c03-9754-cd9a14f177cc") : object "kube-system"/"original-pull-secret" not registered Jul 15 16:56:25.619552 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:25.619510 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx7gk\" (UniqueName: \"kubernetes.io/projected/1542bfc7-eb45-488a-b867-257a2978997b-kube-api-access-gx7gk\") pod \"network-check-target-r6txx\" (UID: \"1542bfc7-eb45-488a-b867-257a2978997b\") " pod="openshift-network-diagnostics/network-check-target-r6txx" Jul 15 16:56:25.619785 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:25.619732 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 16:56:25.619785 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:25.619762 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 16:56:25.619785 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:25.619776 2578 projected.go:196] Error preparing data for projected volume kube-api-access-gx7gk for pod openshift-network-diagnostics/network-check-target-r6txx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:56:25.619968 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:25.619843 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1542bfc7-eb45-488a-b867-257a2978997b-kube-api-access-gx7gk podName:1542bfc7-eb45-488a-b867-257a2978997b nodeName:}" failed. No retries permitted until 2026-07-15 16:56:27.619821384 +0000 UTC m=+5.172951575 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-gx7gk" (UniqueName: "kubernetes.io/projected/1542bfc7-eb45-488a-b867-257a2978997b-kube-api-access-gx7gk") pod "network-check-target-r6txx" (UID: "1542bfc7-eb45-488a-b867-257a2978997b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:56:25.845339 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:25.845216 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 16:51:23 +0000 UTC" deadline="2027-12-31 20:32:18.887730324 +0000 UTC" Jul 15 16:56:25.845339 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:25.845274 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12819h35m53.042460525s" Jul 15 16:56:25.985561 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:25.985088 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:56:25.985561 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:25.985216 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzk7p" podUID="b25e2334-b15b-4c03-9754-cd9a14f177cc" Jul 15 16:56:26.077226 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:26.077185 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-149.ec2.internal" event={"ID":"3ab212e854df387dd5d9339a36377d05","Type":"ContainerStarted","Data":"88a4412780c975291c0b6945160226e137d28bfd9ee76e5f3a79972582811e2c"} Jul 15 16:56:26.986247 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:26.985706 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r6txx" Jul 15 16:56:26.986247 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:26.985834 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r6txx" podUID="1542bfc7-eb45-488a-b867-257a2978997b" Jul 15 16:56:26.986978 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:26.986823 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 16:56:26.986978 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:26.986943 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-62hbp" podUID="df1447df-1b9a-4dc3-ba2c-ecbab1514d09" Jul 15 16:56:27.434684 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:27.434095 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-metrics-certs\") pod \"network-metrics-daemon-62hbp\" (UID: \"df1447df-1b9a-4dc3-ba2c-ecbab1514d09\") " pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 16:56:27.434684 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:27.434263 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:56:27.434684 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:27.434324 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-metrics-certs podName:df1447df-1b9a-4dc3-ba2c-ecbab1514d09 nodeName:}" failed. No retries permitted until 2026-07-15 16:56:31.434304643 +0000 UTC m=+8.987434838 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-metrics-certs") pod "network-metrics-daemon-62hbp" (UID: "df1447df-1b9a-4dc3-ba2c-ecbab1514d09") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:56:27.535998 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:27.535354 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b25e2334-b15b-4c03-9754-cd9a14f177cc-original-pull-secret\") pod \"global-pull-secret-syncer-nzk7p\" (UID: \"b25e2334-b15b-4c03-9754-cd9a14f177cc\") " pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:56:27.535998 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:27.535514 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 16:56:27.535998 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:27.535573 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b25e2334-b15b-4c03-9754-cd9a14f177cc-original-pull-secret podName:b25e2334-b15b-4c03-9754-cd9a14f177cc nodeName:}" failed. No retries permitted until 2026-07-15 16:56:31.535556592 +0000 UTC m=+9.088686785 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b25e2334-b15b-4c03-9754-cd9a14f177cc-original-pull-secret") pod "global-pull-secret-syncer-nzk7p" (UID: "b25e2334-b15b-4c03-9754-cd9a14f177cc") : object "kube-system"/"original-pull-secret" not registered Jul 15 16:56:27.636707 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:27.636552 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx7gk\" (UniqueName: \"kubernetes.io/projected/1542bfc7-eb45-488a-b867-257a2978997b-kube-api-access-gx7gk\") pod \"network-check-target-r6txx\" (UID: \"1542bfc7-eb45-488a-b867-257a2978997b\") " pod="openshift-network-diagnostics/network-check-target-r6txx" Jul 15 16:56:27.636910 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:27.636741 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 16:56:27.636910 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:27.636763 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 16:56:27.636910 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:27.636776 2578 projected.go:196] Error preparing data for projected volume kube-api-access-gx7gk for pod openshift-network-diagnostics/network-check-target-r6txx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:56:27.636910 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:27.636835 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1542bfc7-eb45-488a-b867-257a2978997b-kube-api-access-gx7gk podName:1542bfc7-eb45-488a-b867-257a2978997b nodeName:}" failed. No retries permitted until 2026-07-15 16:56:31.636816599 +0000 UTC m=+9.189946807 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-gx7gk" (UniqueName: "kubernetes.io/projected/1542bfc7-eb45-488a-b867-257a2978997b-kube-api-access-gx7gk") pod "network-check-target-r6txx" (UID: "1542bfc7-eb45-488a-b867-257a2978997b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:56:27.985446 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:27.984911 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:56:27.985446 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:27.985042 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzk7p" podUID="b25e2334-b15b-4c03-9754-cd9a14f177cc" Jul 15 16:56:28.985529 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:28.985031 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 16:56:28.985529 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:28.985171 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-62hbp" podUID="df1447df-1b9a-4dc3-ba2c-ecbab1514d09" Jul 15 16:56:28.986081 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:28.985641 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r6txx" Jul 15 16:56:28.986081 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:28.985760 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r6txx" podUID="1542bfc7-eb45-488a-b867-257a2978997b" Jul 15 16:56:29.984495 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:29.984346 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:56:29.984495 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:29.984487 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzk7p" podUID="b25e2334-b15b-4c03-9754-cd9a14f177cc" Jul 15 16:56:30.984566 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:30.984536 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r6txx" Jul 15 16:56:30.985013 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:30.984677 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r6txx" podUID="1542bfc7-eb45-488a-b867-257a2978997b" Jul 15 16:56:30.985013 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:30.984719 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 16:56:30.985013 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:30.984803 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-62hbp" podUID="df1447df-1b9a-4dc3-ba2c-ecbab1514d09" Jul 15 16:56:31.468926 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:31.468889 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-metrics-certs\") pod \"network-metrics-daemon-62hbp\" (UID: \"df1447df-1b9a-4dc3-ba2c-ecbab1514d09\") " pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 16:56:31.469115 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:31.469082 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:56:31.469173 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:31.469156 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-metrics-certs podName:df1447df-1b9a-4dc3-ba2c-ecbab1514d09 nodeName:}" failed. No retries permitted until 2026-07-15 16:56:39.46913466 +0000 UTC m=+17.022264872 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-metrics-certs") pod "network-metrics-daemon-62hbp" (UID: "df1447df-1b9a-4dc3-ba2c-ecbab1514d09") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:56:31.570204 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:31.570165 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b25e2334-b15b-4c03-9754-cd9a14f177cc-original-pull-secret\") pod \"global-pull-secret-syncer-nzk7p\" (UID: \"b25e2334-b15b-4c03-9754-cd9a14f177cc\") " pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:56:31.570474 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:31.570452 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 16:56:31.570637 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:31.570530 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b25e2334-b15b-4c03-9754-cd9a14f177cc-original-pull-secret podName:b25e2334-b15b-4c03-9754-cd9a14f177cc nodeName:}" failed. No retries permitted until 2026-07-15 16:56:39.570510525 +0000 UTC m=+17.123640726 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b25e2334-b15b-4c03-9754-cd9a14f177cc-original-pull-secret") pod "global-pull-secret-syncer-nzk7p" (UID: "b25e2334-b15b-4c03-9754-cd9a14f177cc") : object "kube-system"/"original-pull-secret" not registered Jul 15 16:56:31.671199 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:31.671153 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx7gk\" (UniqueName: \"kubernetes.io/projected/1542bfc7-eb45-488a-b867-257a2978997b-kube-api-access-gx7gk\") pod \"network-check-target-r6txx\" (UID: \"1542bfc7-eb45-488a-b867-257a2978997b\") " pod="openshift-network-diagnostics/network-check-target-r6txx" Jul 15 16:56:31.671392 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:31.671311 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 16:56:31.671392 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:31.671339 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 16:56:31.671392 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:31.671353 2578 projected.go:196] Error preparing data for projected volume kube-api-access-gx7gk for pod openshift-network-diagnostics/network-check-target-r6txx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:56:31.671540 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:31.671420 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1542bfc7-eb45-488a-b867-257a2978997b-kube-api-access-gx7gk podName:1542bfc7-eb45-488a-b867-257a2978997b nodeName:}" failed. No retries permitted until 2026-07-15 16:56:39.671401769 +0000 UTC m=+17.224531981 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-gx7gk" (UniqueName: "kubernetes.io/projected/1542bfc7-eb45-488a-b867-257a2978997b-kube-api-access-gx7gk") pod "network-check-target-r6txx" (UID: "1542bfc7-eb45-488a-b867-257a2978997b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:56:31.984929 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:31.984667 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:56:31.984929 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:31.984814 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzk7p" podUID="b25e2334-b15b-4c03-9754-cd9a14f177cc" Jul 15 16:56:32.985980 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:32.985924 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r6txx" Jul 15 16:56:32.986404 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:32.986046 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r6txx" podUID="1542bfc7-eb45-488a-b867-257a2978997b" Jul 15 16:56:32.986404 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:32.986056 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 16:56:32.986404 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:32.986165 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-62hbp" podUID="df1447df-1b9a-4dc3-ba2c-ecbab1514d09" Jul 15 16:56:33.985263 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:33.985136 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:56:33.985447 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:33.985284 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzk7p" podUID="b25e2334-b15b-4c03-9754-cd9a14f177cc" Jul 15 16:56:34.984820 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:34.984563 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r6txx" Jul 15 16:56:34.985274 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:34.984563 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 16:56:34.985274 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:34.984921 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r6txx" podUID="1542bfc7-eb45-488a-b867-257a2978997b" Jul 15 16:56:34.985274 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:34.984998 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-62hbp" podUID="df1447df-1b9a-4dc3-ba2c-ecbab1514d09" Jul 15 16:56:35.984708 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:35.984677 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:56:35.984907 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:35.984792 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzk7p" podUID="b25e2334-b15b-4c03-9754-cd9a14f177cc" Jul 15 16:56:36.985323 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:36.985292 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 16:56:36.985754 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:36.985420 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-62hbp" podUID="df1447df-1b9a-4dc3-ba2c-ecbab1514d09" Jul 15 16:56:36.985754 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:36.985479 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r6txx" Jul 15 16:56:36.985754 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:36.985570 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r6txx" podUID="1542bfc7-eb45-488a-b867-257a2978997b" Jul 15 16:56:37.984527 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:37.984492 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:56:37.984695 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:37.984615 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzk7p" podUID="b25e2334-b15b-4c03-9754-cd9a14f177cc" Jul 15 16:56:38.985259 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:38.985180 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r6txx" Jul 15 16:56:38.985696 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:38.985301 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r6txx" podUID="1542bfc7-eb45-488a-b867-257a2978997b" Jul 15 16:56:38.985696 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:38.985357 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 16:56:38.985696 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:38.985481 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-62hbp" podUID="df1447df-1b9a-4dc3-ba2c-ecbab1514d09" Jul 15 16:56:39.532252 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:39.532210 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-metrics-certs\") pod \"network-metrics-daemon-62hbp\" (UID: \"df1447df-1b9a-4dc3-ba2c-ecbab1514d09\") " pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 16:56:39.532451 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:39.532380 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:56:39.532497 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:39.532459 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-metrics-certs podName:df1447df-1b9a-4dc3-ba2c-ecbab1514d09 nodeName:}" failed. No retries permitted until 2026-07-15 16:56:55.532437167 +0000 UTC m=+33.085567370 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-metrics-certs") pod "network-metrics-daemon-62hbp" (UID: "df1447df-1b9a-4dc3-ba2c-ecbab1514d09") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:56:39.632749 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:39.632716 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b25e2334-b15b-4c03-9754-cd9a14f177cc-original-pull-secret\") pod \"global-pull-secret-syncer-nzk7p\" (UID: \"b25e2334-b15b-4c03-9754-cd9a14f177cc\") " pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:56:39.632892 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:39.632866 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 16:56:39.632956 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:39.632919 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b25e2334-b15b-4c03-9754-cd9a14f177cc-original-pull-secret podName:b25e2334-b15b-4c03-9754-cd9a14f177cc nodeName:}" failed. No retries permitted until 2026-07-15 16:56:55.632905934 +0000 UTC m=+33.186036125 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b25e2334-b15b-4c03-9754-cd9a14f177cc-original-pull-secret") pod "global-pull-secret-syncer-nzk7p" (UID: "b25e2334-b15b-4c03-9754-cd9a14f177cc") : object "kube-system"/"original-pull-secret" not registered Jul 15 16:56:39.733838 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:39.733801 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx7gk\" (UniqueName: \"kubernetes.io/projected/1542bfc7-eb45-488a-b867-257a2978997b-kube-api-access-gx7gk\") pod \"network-check-target-r6txx\" (UID: \"1542bfc7-eb45-488a-b867-257a2978997b\") " pod="openshift-network-diagnostics/network-check-target-r6txx" Jul 15 16:56:39.733991 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:39.733972 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 16:56:39.734073 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:39.733995 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 16:56:39.734073 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:39.734007 2578 projected.go:196] Error preparing data for projected volume kube-api-access-gx7gk for pod openshift-network-diagnostics/network-check-target-r6txx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:56:39.734166 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:39.734072 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1542bfc7-eb45-488a-b867-257a2978997b-kube-api-access-gx7gk podName:1542bfc7-eb45-488a-b867-257a2978997b nodeName:}" failed. No retries permitted until 2026-07-15 16:56:55.73405215 +0000 UTC m=+33.287182359 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-gx7gk" (UniqueName: "kubernetes.io/projected/1542bfc7-eb45-488a-b867-257a2978997b-kube-api-access-gx7gk") pod "network-check-target-r6txx" (UID: "1542bfc7-eb45-488a-b867-257a2978997b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:56:39.984554 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:39.984477 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:56:39.984718 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:39.984646 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzk7p" podUID="b25e2334-b15b-4c03-9754-cd9a14f177cc" Jul 15 16:56:40.985131 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:40.985094 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r6txx" Jul 15 16:56:40.985550 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:40.985215 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r6txx" podUID="1542bfc7-eb45-488a-b867-257a2978997b" Jul 15 16:56:40.985550 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:40.985263 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 16:56:40.985550 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:40.985397 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-62hbp" podUID="df1447df-1b9a-4dc3-ba2c-ecbab1514d09" Jul 15 16:56:41.984564 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:41.984528 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:56:41.984741 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:41.984667 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzk7p" podUID="b25e2334-b15b-4c03-9754-cd9a14f177cc" Jul 15 16:56:42.986006 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:42.985971 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r6txx" Jul 15 16:56:42.986453 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:42.986078 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r6txx" podUID="1542bfc7-eb45-488a-b867-257a2978997b" Jul 15 16:56:42.986453 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:42.986139 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 16:56:42.986453 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:42.986257 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-62hbp" podUID="df1447df-1b9a-4dc3-ba2c-ecbab1514d09" Jul 15 16:56:43.985250 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:43.985011 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:56:43.985350 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:43.985331 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzk7p" podUID="b25e2334-b15b-4c03-9754-cd9a14f177cc" Jul 15 16:56:44.111516 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:44.111484 2578 generic.go:356] "Generic (PLEG): container finished" podID="97c17808d0380dfd6329669266413115" containerID="296fa3d46ccb33d730e2a4ae83125ee6af419997e5373aa18bd8829aa30c1cb6" exitCode=0 Jul 15 16:56:44.112211 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:44.111568 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-149.ec2.internal" event={"ID":"97c17808d0380dfd6329669266413115","Type":"ContainerDied","Data":"296fa3d46ccb33d730e2a4ae83125ee6af419997e5373aa18bd8829aa30c1cb6"} Jul 15 16:56:44.113487 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:44.113462 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kmrqt" event={"ID":"7d41273f-cc9a-4a08-9225-3a89453afeea","Type":"ContainerStarted","Data":"5f52d5c716cc6d2c7674e7132f3fbcb57df6f9e493c50f2b16514997dd7b2031"} Jul 15 16:56:44.115901 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:44.115885 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29wtx_88f8928c-237e-457f-a032-707f48111f36/ovn-acl-logging/0.log" Jul 15 16:56:44.116200 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:44.116180 2578 generic.go:356] "Generic (PLEG): container finished" podID="88f8928c-237e-457f-a032-707f48111f36" containerID="cccba371d819ec88d119f891838c7fffa787da377f1508665c21ee875cc0f5f6" exitCode=1 Jul 15 16:56:44.116264 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:44.116245 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" event={"ID":"88f8928c-237e-457f-a032-707f48111f36","Type":"ContainerStarted","Data":"4d36f1b6b9f8a06cae6c8c1c1f3a687d6f10b6368782de6e77f9d2855f8f2d8c"} Jul 15 16:56:44.116365 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:44.116303 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" event={"ID":"88f8928c-237e-457f-a032-707f48111f36","Type":"ContainerStarted","Data":"361cf0eba293fc5c5942c71837d975020502e350e60baf266544bca0e351c825"} Jul 15 16:56:44.116365 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:44.116346 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" event={"ID":"88f8928c-237e-457f-a032-707f48111f36","Type":"ContainerStarted","Data":"26135cf82b011bd94a7db4238d728b0ebc4d741a853c49f490efbb1619b9af89"} Jul 15 16:56:44.116460 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:44.116378 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" event={"ID":"88f8928c-237e-457f-a032-707f48111f36","Type":"ContainerDied","Data":"cccba371d819ec88d119f891838c7fffa787da377f1508665c21ee875cc0f5f6"} Jul 15 16:56:44.116460 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:44.116410 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" event={"ID":"88f8928c-237e-457f-a032-707f48111f36","Type":"ContainerStarted","Data":"d475085b8c6c8e07ae67f67a4b15936af4fa498bc395923968725921507760f5"} Jul 15 16:56:44.117609 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:44.117577 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-149.ec2.internal" event={"ID":"3ab212e854df387dd5d9339a36377d05","Type":"ContainerStarted","Data":"eb42712be0c0c0d3c48f9d9fd4b2652a43381fb473f8ff0f451d81ffe0d7db87"} Jul 15 16:56:44.118853 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:44.118834 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-whrn6" event={"ID":"d0867616-074a-4b55-94ce-df0e85196692","Type":"ContainerStarted","Data":"871e29c0fe636b944cb26dd88c3d6b7662658c042ffceb9b49c7820e5b0a50fd"} Jul 15 16:56:44.120236 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:44.120218 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9whs6" event={"ID":"55622740-75bd-4532-bda0-e3fb5aa35185","Type":"ContainerStarted","Data":"9130b75a24596f7359211fea50a464cb03bbba9c50c408331a3909ec27c45d98"} Jul 15 16:56:44.122148 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:44.122124 2578 generic.go:356] "Generic (PLEG): container finished" podID="a488304f-ae1d-4dd5-9af5-a1906ce229a9" containerID="4f96e6ced3f51cbdc7a9db76c35bcc7204576758fef62af06867a718310c9b36" exitCode=0 Jul 15 16:56:44.122257 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:44.122221 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tv5fg" event={"ID":"a488304f-ae1d-4dd5-9af5-a1906ce229a9","Type":"ContainerDied","Data":"4f96e6ced3f51cbdc7a9db76c35bcc7204576758fef62af06867a718310c9b36"} Jul 15 16:56:44.124230 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:44.124203 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7c67r" event={"ID":"16c5ca09-925c-4046-8ade-ae582e290339","Type":"ContainerStarted","Data":"5beb32056611082607f341aa6f6e25c91b060f8dcf43b7e0a786699e90e6a5c1"} Jul 15 16:56:44.125889 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:44.125865 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cvlbn" event={"ID":"c9c2ba93-cd7a-4aa9-81f4-b5afeee112db","Type":"ContainerStarted","Data":"db2b00e8b02e0f31c5fd33f9fc8cd3ea05c6192c351e8e583075c1cefbfccec9"} Jul 15 16:56:44.127350 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:44.127330 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" event={"ID":"12f6b5f7-a4ba-492b-b150-aedd91c64e53","Type":"ContainerStarted","Data":"8bca1bf970ee4ed94e6bd2412c3c0641feefae8e09ffeb2881852979bb3dc81c"} Jul 15 16:56:44.162963 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:44.162916 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9whs6" podStartSLOduration=7.983310622 podStartE2EDuration="21.162903239s" totalImagesPullingTime="13.179592617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:56:44.16266455 +0000 UTC m=+21.715794784" watchObservedRunningTime="2026-07-15 16:56:44.162903239 +0000 UTC m=+21.716033453" Jul 15 16:56:44.196701 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:44.192956 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-149.ec2.internal" podStartSLOduration=21.192937098 podStartE2EDuration="21.192937098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:56:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:56:44.175826028 +0000 UTC m=+21.728956242" watchObservedRunningTime="2026-07-15 16:56:44.192937098 +0000 UTC m=+21.746067314" Jul 15 16:56:44.196701 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:44.193304 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7c67r" podStartSLOduration=1.853644975 podStartE2EDuration="21.193292414s" totalImagesPullingTime="19.339647439s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:56:44.192770871 +0000 UTC m=+21.745901107" watchObservedRunningTime="2026-07-15 16:56:44.193292414 +0000 UTC m=+21.746422632" Jul 15 16:56:44.209390 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:44.209349 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-whrn6" podStartSLOduration=6.760546631 podStartE2EDuration="21.209335027s" totalImagesPullingTime="14.448788396s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:56:44.209320591 +0000 UTC m=+21.762450804" watchObservedRunningTime="2026-07-15 16:56:44.209335027 +0000 UTC m=+21.762465240" Jul 15 16:56:44.226405 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:44.226365 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zbvt7" podStartSLOduration=2.813307535 podStartE2EDuration="22.226349831s" totalImagesPullingTime="19.413042296s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:56:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:56:44.226095464 +0000 UTC m=+21.779225689" watchObservedRunningTime="2026-07-15 16:56:44.226349831 +0000 UTC m=+21.779480041" Jul 15 16:56:44.243679 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:44.243633 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-cvlbn" podStartSLOduration=6.712023762 podStartE2EDuration="21.243615963s" totalImagesPullingTime="14.531592201s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:56:44.24336591 +0000 UTC m=+21.796496125" watchObservedRunningTime="2026-07-15 16:56:44.243615963 +0000 UTC m=+21.796746167" Jul 15 16:56:44.985267 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:44.985233 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r6txx" Jul 15 16:56:44.985532 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:44.985365 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r6txx" podUID="1542bfc7-eb45-488a-b867-257a2978997b" Jul 15 16:56:44.985532 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:44.985489 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 16:56:44.985674 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:44.985623 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-62hbp" podUID="df1447df-1b9a-4dc3-ba2c-ecbab1514d09" Jul 15 16:56:45.131180 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:45.131143 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-g2kh8" event={"ID":"ffc75c8d-b600-4be4-a811-6d57e6f81ea3","Type":"ContainerStarted","Data":"eb2d72bfbe8511fd030127e55feb620cabd486837b06dd9250098b9a18d55903"} Jul 15 16:56:45.133332 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:45.133305 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-149.ec2.internal" event={"ID":"97c17808d0380dfd6329669266413115","Type":"ContainerStarted","Data":"189b561afe180acd72b50709690137e0e463daac5620b4ff2db1e0cb592bbff7"} Jul 15 16:56:45.136580 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:45.136562 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29wtx_88f8928c-237e-457f-a032-707f48111f36/ovn-acl-logging/0.log" Jul 15 16:56:45.136982 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:45.136963 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" event={"ID":"88f8928c-237e-457f-a032-707f48111f36","Type":"ContainerStarted","Data":"824f98f60c8280e1d80b8afb0620d72347c5aa47c13c095aad9fec128d53bba8"} Jul 15 16:56:45.145977 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:45.145926 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-g2kh8" podStartSLOduration=2.757529789 podStartE2EDuration="22.145908348s" totalImagesPullingTime="19.388378559s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:56:45.145287686 +0000 UTC m=+22.698417926" watchObservedRunningTime="2026-07-15 16:56:45.145908348 +0000 UTC m=+22.699038564" Jul 15 16:56:45.162446 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:45.162404 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-149.ec2.internal" podStartSLOduration=22.162388704 podStartE2EDuration="22.162388704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:56:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:56:45.162209432 +0000 UTC m=+22.715339647" watchObservedRunningTime="2026-07-15 16:56:45.162388704 +0000 UTC m=+22.715518919" Jul 15 16:56:45.192076 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:45.191996 2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 15 16:56:45.864403 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:45.864270 2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T16:56:45.192021427Z","UUID":"b6677fea-217a-4478-a420-35257baeff6a","Handler":null,"Name":"","Endpoint":""} Jul 15 16:56:45.867365 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:45.867340 2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 15 16:56:45.867510 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:45.867385 2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 15 16:56:45.936145 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:45.936115 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9whs6" Jul 15 16:56:45.936805 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:45.936784 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9whs6" Jul 15 16:56:45.984737 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:45.984704 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:56:45.984911 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:45.984834 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzk7p" podUID="b25e2334-b15b-4c03-9754-cd9a14f177cc" Jul 15 16:56:46.144456 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:46.144366 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kmrqt" event={"ID":"7d41273f-cc9a-4a08-9225-3a89453afeea","Type":"ContainerStarted","Data":"158f2a1d09f464b865b5d58f0fe4f5ca372dc27f23a4040254cb6ef4f8f2e349"} Jul 15 16:56:46.144997 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:46.144745 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9whs6" Jul 15 16:56:46.145138 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:46.145120 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9whs6" Jul 15 16:56:46.985092 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:46.985063 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 16:56:46.985283 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:46.985063 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r6txx" Jul 15 16:56:46.985283 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:46.985191 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-62hbp" podUID="df1447df-1b9a-4dc3-ba2c-ecbab1514d09" Jul 15 16:56:46.985283 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:46.985258 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r6txx" podUID="1542bfc7-eb45-488a-b867-257a2978997b" Jul 15 16:56:47.148371 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:47.148334 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kmrqt" event={"ID":"7d41273f-cc9a-4a08-9225-3a89453afeea","Type":"ContainerStarted","Data":"e51502863f019578498cca991c525396d1381c7fbcc35587412f8643c9cf8d82"} Jul 15 16:56:47.151900 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:47.151881 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29wtx_88f8928c-237e-457f-a032-707f48111f36/ovn-acl-logging/0.log" Jul 15 16:56:47.152369 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:47.152349 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" event={"ID":"88f8928c-237e-457f-a032-707f48111f36","Type":"ContainerStarted","Data":"3674ae952ac72956502032b3134c9345864debdb2efedf2884627bc844fb872f"} Jul 15 16:56:47.170434 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:47.170396 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kmrqt" podStartSLOduration=7.54760201 podStartE2EDuration="24.170382288s" totalImagesPullingTime="16.622780278s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:56:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:56:47.169022894 +0000 UTC m=+24.722153109" watchObservedRunningTime="2026-07-15 16:56:47.170382288 +0000 UTC m=+24.723512501" Jul 15 16:56:47.984708 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:47.984501 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:56:47.984868 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:47.984802 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzk7p" podUID="b25e2334-b15b-4c03-9754-cd9a14f177cc" Jul 15 16:56:48.985027 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:48.984788 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r6txx" Jul 15 16:56:48.985785 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:48.984846 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 16:56:48.985785 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:48.985127 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r6txx" podUID="1542bfc7-eb45-488a-b867-257a2978997b" Jul 15 16:56:48.985785 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:48.985169 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-62hbp" podUID="df1447df-1b9a-4dc3-ba2c-ecbab1514d09" Jul 15 16:56:49.161346 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:49.161321 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29wtx_88f8928c-237e-457f-a032-707f48111f36/ovn-acl-logging/0.log" Jul 15 16:56:49.161697 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:49.161668 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" event={"ID":"88f8928c-237e-457f-a032-707f48111f36","Type":"ContainerStarted","Data":"70f3a6db974f41a21b0a8ab586c8ca5de3e8536cfbd34a107f6730336b3d927a"} Jul 15 16:56:49.162007 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:49.161987 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:49.162181 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:49.162164 2578 scope.go:117] "RemoveContainer" containerID="cccba371d819ec88d119f891838c7fffa787da377f1508665c21ee875cc0f5f6" Jul 15 16:56:49.163557 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:49.163538 2578 generic.go:356] "Generic (PLEG): container finished" podID="a488304f-ae1d-4dd5-9af5-a1906ce229a9" containerID="de967c53c6f9c1230afa2dfb764b94fe65a073fe4e1b656159e2381ea6069248" exitCode=0 Jul 15 16:56:49.163655 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:49.163570 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tv5fg" event={"ID":"a488304f-ae1d-4dd5-9af5-a1906ce229a9","Type":"ContainerDied","Data":"de967c53c6f9c1230afa2dfb764b94fe65a073fe4e1b656159e2381ea6069248"} Jul 15 16:56:49.177996 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:49.177977 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:49.984880 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:49.984856 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:56:49.985022 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:49.984956 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzk7p" podUID="b25e2334-b15b-4c03-9754-cd9a14f177cc" Jul 15 16:56:50.167910 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:50.167876 2578 generic.go:356] "Generic (PLEG): container finished" podID="a488304f-ae1d-4dd5-9af5-a1906ce229a9" containerID="8d3762eda252e14572b7e1384129bf23e1a2f7e3b92c094267630d9953999306" exitCode=0 Jul 15 16:56:50.168345 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:50.167956 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tv5fg" event={"ID":"a488304f-ae1d-4dd5-9af5-a1906ce229a9","Type":"ContainerDied","Data":"8d3762eda252e14572b7e1384129bf23e1a2f7e3b92c094267630d9953999306"} Jul 15 16:56:50.171858 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:50.171841 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29wtx_88f8928c-237e-457f-a032-707f48111f36/ovn-acl-logging/0.log" Jul 15 16:56:50.172243 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:50.172222 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" event={"ID":"88f8928c-237e-457f-a032-707f48111f36","Type":"ContainerStarted","Data":"bacad99b7a006351c86ef2aa3a85115f9fd90a2a95dce1f3b7dcc0a5d958c2d6"} Jul 15 16:56:50.172407 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:50.172394 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:50.172457 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:50.172443 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:50.187137 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:50.187118 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:56:50.240899 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:50.240855 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" podStartSLOduration=7.815187805 podStartE2EDuration="27.240843425s" totalImagesPullingTime="19.42565562s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:56:50.240664659 +0000 UTC m=+27.793794894" watchObservedRunningTime="2026-07-15 16:56:50.240843425 +0000 UTC m=+27.793973638" Jul 15 16:56:50.364818 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:50.364784 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r6txx"] Jul 15 16:56:50.364966 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:50.364914 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r6txx" Jul 15 16:56:50.365011 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:50.364995 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r6txx" podUID="1542bfc7-eb45-488a-b867-257a2978997b" Jul 15 16:56:50.368124 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:50.368097 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-62hbp"] Jul 15 16:56:50.368251 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:50.368194 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 16:56:50.368310 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:50.368291 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-62hbp" podUID="df1447df-1b9a-4dc3-ba2c-ecbab1514d09" Jul 15 16:56:50.368635 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:50.368617 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nzk7p"] Jul 15 16:56:50.368719 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:50.368705 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:56:50.368803 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:50.368786 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzk7p" podUID="b25e2334-b15b-4c03-9754-cd9a14f177cc" Jul 15 16:56:51.176673 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:51.176640 2578 generic.go:356] "Generic (PLEG): container finished" podID="a488304f-ae1d-4dd5-9af5-a1906ce229a9" containerID="ba6f6466742f083d2df6a27a49da12e8c86f49907daf177f58a0fb14ec3e34c3" exitCode=0 Jul 15 16:56:51.177085 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:51.176718 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tv5fg" event={"ID":"a488304f-ae1d-4dd5-9af5-a1906ce229a9","Type":"ContainerDied","Data":"ba6f6466742f083d2df6a27a49da12e8c86f49907daf177f58a0fb14ec3e34c3"} Jul 15 16:56:51.984493 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:51.984442 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:56:51.984493 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:51.984491 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r6txx" Jul 15 16:56:51.984733 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:51.984548 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 16:56:51.984733 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:51.984563 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzk7p" podUID="b25e2334-b15b-4c03-9754-cd9a14f177cc" Jul 15 16:56:51.984733 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:51.984640 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-62hbp" podUID="df1447df-1b9a-4dc3-ba2c-ecbab1514d09" Jul 15 16:56:51.984733 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:51.984713 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r6txx" podUID="1542bfc7-eb45-488a-b867-257a2978997b" Jul 15 16:56:53.984671 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:53.984636 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:56:53.984671 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:53.984665 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 16:56:53.985179 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:53.984665 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r6txx" Jul 15 16:56:53.985179 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:53.984769 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzk7p" podUID="b25e2334-b15b-4c03-9754-cd9a14f177cc" Jul 15 16:56:53.985179 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:53.984867 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-62hbp" podUID="df1447df-1b9a-4dc3-ba2c-ecbab1514d09" Jul 15 16:56:53.985179 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:53.984945 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r6txx" podUID="1542bfc7-eb45-488a-b867-257a2978997b" Jul 15 16:56:55.557727 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:55.557689 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-metrics-certs\") pod \"network-metrics-daemon-62hbp\" (UID: \"df1447df-1b9a-4dc3-ba2c-ecbab1514d09\") " pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 16:56:55.558243 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:55.557793 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:56:55.558243 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:55.557854 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-metrics-certs podName:df1447df-1b9a-4dc3-ba2c-ecbab1514d09 nodeName:}" failed. No retries permitted until 2026-07-15 16:57:27.557834795 +0000 UTC m=+65.110964991 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-metrics-certs") pod "network-metrics-daemon-62hbp" (UID: "df1447df-1b9a-4dc3-ba2c-ecbab1514d09") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:56:55.658564 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:55.658522 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b25e2334-b15b-4c03-9754-cd9a14f177cc-original-pull-secret\") pod \"global-pull-secret-syncer-nzk7p\" (UID: \"b25e2334-b15b-4c03-9754-cd9a14f177cc\") " pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:56:55.658763 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:55.658685 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 16:56:55.658763 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:55.658756 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b25e2334-b15b-4c03-9754-cd9a14f177cc-original-pull-secret podName:b25e2334-b15b-4c03-9754-cd9a14f177cc nodeName:}" failed. No retries permitted until 2026-07-15 16:57:27.658736874 +0000 UTC m=+65.211867074 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b25e2334-b15b-4c03-9754-cd9a14f177cc-original-pull-secret") pod "global-pull-secret-syncer-nzk7p" (UID: "b25e2334-b15b-4c03-9754-cd9a14f177cc") : object "kube-system"/"original-pull-secret" not registered Jul 15 16:56:55.759126 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:55.759091 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx7gk\" (UniqueName: \"kubernetes.io/projected/1542bfc7-eb45-488a-b867-257a2978997b-kube-api-access-gx7gk\") pod \"network-check-target-r6txx\" (UID: \"1542bfc7-eb45-488a-b867-257a2978997b\") " pod="openshift-network-diagnostics/network-check-target-r6txx" Jul 15 16:56:55.759318 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:55.759295 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 16:56:55.759390 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:55.759324 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 16:56:55.759390 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:55.759337 2578 projected.go:196] Error preparing data for projected volume kube-api-access-gx7gk for pod openshift-network-diagnostics/network-check-target-r6txx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:56:55.759479 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:55.759400 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1542bfc7-eb45-488a-b867-257a2978997b-kube-api-access-gx7gk podName:1542bfc7-eb45-488a-b867-257a2978997b nodeName:}" failed. No retries permitted until 2026-07-15 16:57:27.759380284 +0000 UTC m=+65.312510481 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-gx7gk" (UniqueName: "kubernetes.io/projected/1542bfc7-eb45-488a-b867-257a2978997b-kube-api-access-gx7gk") pod "network-check-target-r6txx" (UID: "1542bfc7-eb45-488a-b867-257a2978997b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:56:55.985842 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:55.985812 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:56:55.986058 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:55.985853 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 16:56:55.986058 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:55.985823 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r6txx" Jul 15 16:56:55.986058 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:55.985939 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzk7p" podUID="b25e2334-b15b-4c03-9754-cd9a14f177cc" Jul 15 16:56:55.986058 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:55.986018 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r6txx" podUID="1542bfc7-eb45-488a-b867-257a2978997b" Jul 15 16:56:55.986222 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:55.986098 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-62hbp" podUID="df1447df-1b9a-4dc3-ba2c-ecbab1514d09" Jul 15 16:56:57.191770 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.191736 2578 generic.go:356] "Generic (PLEG): container finished" podID="a488304f-ae1d-4dd5-9af5-a1906ce229a9" containerID="fef517a871c212261c38e7d512a0090051afa809112e1d28d07eea73b06274ba" exitCode=0 Jul 15 16:56:57.192210 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.191809 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tv5fg" event={"ID":"a488304f-ae1d-4dd5-9af5-a1906ce229a9","Type":"ContainerDied","Data":"fef517a871c212261c38e7d512a0090051afa809112e1d28d07eea73b06274ba"} Jul 15 16:56:57.283493 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.283476 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-149.ec2.internal" event="NodeReady" Jul 15 16:56:57.283615 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.283590 2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 15 16:56:57.320813 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.320790 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-795b87c6c7-xztkp"] Jul 15 16:56:57.335180 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.335158 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 16:56:57.337124 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.337092 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jvr75"] Jul 15 16:56:57.337825 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.337806 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 15 16:56:57.337935 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.337887 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hhz8f\"" Jul 15 16:56:57.337935 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.337806 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 15 16:56:57.338050 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.337944 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 15 16:56:57.343073 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.343054 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 15 16:56:57.347656 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.347638 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-795b87c6c7-xztkp"] Jul 15 16:56:57.347733 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.347674 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qsxzw"] Jul 15 16:56:57.347773 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.347742 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jvr75" Jul 15 16:56:57.350081 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.350067 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lllhc\"" Jul 15 16:56:57.350270 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.350257 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 15 16:56:57.350319 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.350287 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 15 16:56:57.359461 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.359436 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jvr75"] Jul 15 16:56:57.359564 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.359491 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qsxzw"] Jul 15 16:56:57.359653 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.359564 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qsxzw" Jul 15 16:56:57.362394 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.362375 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rf5xj\"" Jul 15 16:56:57.362486 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.362400 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 15 16:56:57.362486 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.362382 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 15 16:56:57.362606 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.362582 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 15 16:56:57.472533 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.472504 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f0d3cec0-475c-481e-beb3-012b1c8c215b-trusted-ca\") pod \"image-registry-795b87c6c7-xztkp\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 16:56:57.472684 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.472539 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-bound-sa-token\") pod \"image-registry-795b87c6c7-xztkp\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 16:56:57.472684 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.472556 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8qcs\" (UniqueName: \"kubernetes.io/projected/f7f08403-7f03-4072-993c-9b02881fd48f-kube-api-access-x8qcs\") pod \"ingress-canary-qsxzw\" (UID: \"f7f08403-7f03-4072-993c-9b02881fd48f\") " pod="openshift-ingress-canary/ingress-canary-qsxzw" Jul 15 16:56:57.472684 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.472669 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-certificates\") pod \"image-registry-795b87c6c7-xztkp\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 16:56:57.472805 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.472701 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/191e57d2-b61f-443a-a7ca-a8a7ad699155-config-volume\") pod \"dns-default-jvr75\" (UID: \"191e57d2-b61f-443a-a7ca-a8a7ad699155\") " pod="openshift-dns/dns-default-jvr75" Jul 15 16:56:57.472805 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.472723 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/191e57d2-b61f-443a-a7ca-a8a7ad699155-tmp-dir\") pod \"dns-default-jvr75\" (UID: \"191e57d2-b61f-443a-a7ca-a8a7ad699155\") " pod="openshift-dns/dns-default-jvr75" Jul 15 16:56:57.472805 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.472742 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f0d3cec0-475c-481e-beb3-012b1c8c215b-ca-trust-extracted\") pod \"image-registry-795b87c6c7-xztkp\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 16:56:57.472805 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.472758 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f0d3cec0-475c-481e-beb3-012b1c8c215b-installation-pull-secrets\") pod \"image-registry-795b87c6c7-xztkp\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 16:56:57.472805 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.472778 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f0d3cec0-475c-481e-beb3-012b1c8c215b-image-registry-private-configuration\") pod \"image-registry-795b87c6c7-xztkp\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 16:56:57.472805 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.472795 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f7f08403-7f03-4072-993c-9b02881fd48f-cert\") pod \"ingress-canary-qsxzw\" (UID: \"f7f08403-7f03-4072-993c-9b02881fd48f\") " pod="openshift-ingress-canary/ingress-canary-qsxzw" Jul 15 16:56:57.472993 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.472829 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzd7h\" (UniqueName: \"kubernetes.io/projected/191e57d2-b61f-443a-a7ca-a8a7ad699155-kube-api-access-tzd7h\") pod \"dns-default-jvr75\" (UID: \"191e57d2-b61f-443a-a7ca-a8a7ad699155\") " pod="openshift-dns/dns-default-jvr75" Jul 15 16:56:57.472993 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.472872 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-tls\") pod \"image-registry-795b87c6c7-xztkp\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 16:56:57.472993 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.472890 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wk9j7\" (UniqueName: \"kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-kube-api-access-wk9j7\") pod \"image-registry-795b87c6c7-xztkp\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 16:56:57.472993 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.472913 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/191e57d2-b61f-443a-a7ca-a8a7ad699155-metrics-tls\") pod \"dns-default-jvr75\" (UID: \"191e57d2-b61f-443a-a7ca-a8a7ad699155\") " pod="openshift-dns/dns-default-jvr75" Jul 15 16:56:57.574051 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.573975 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/191e57d2-b61f-443a-a7ca-a8a7ad699155-config-volume\") pod \"dns-default-jvr75\" (UID: \"191e57d2-b61f-443a-a7ca-a8a7ad699155\") " pod="openshift-dns/dns-default-jvr75" Jul 15 16:56:57.574051 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.574008 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/191e57d2-b61f-443a-a7ca-a8a7ad699155-tmp-dir\") pod \"dns-default-jvr75\" (UID: \"191e57d2-b61f-443a-a7ca-a8a7ad699155\") " pod="openshift-dns/dns-default-jvr75" Jul 15 16:56:57.574051 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.574026 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f0d3cec0-475c-481e-beb3-012b1c8c215b-ca-trust-extracted\") pod \"image-registry-795b87c6c7-xztkp\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 16:56:57.574051 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.574044 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f0d3cec0-475c-481e-beb3-012b1c8c215b-installation-pull-secrets\") pod \"image-registry-795b87c6c7-xztkp\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 16:56:57.574331 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.574064 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f0d3cec0-475c-481e-beb3-012b1c8c215b-image-registry-private-configuration\") pod \"image-registry-795b87c6c7-xztkp\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 16:56:57.574331 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.574079 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f7f08403-7f03-4072-993c-9b02881fd48f-cert\") pod \"ingress-canary-qsxzw\" (UID: \"f7f08403-7f03-4072-993c-9b02881fd48f\") " pod="openshift-ingress-canary/ingress-canary-qsxzw" Jul 15 16:56:57.574331 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.574137 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzd7h\" (UniqueName: \"kubernetes.io/projected/191e57d2-b61f-443a-a7ca-a8a7ad699155-kube-api-access-tzd7h\") pod \"dns-default-jvr75\" (UID: \"191e57d2-b61f-443a-a7ca-a8a7ad699155\") " pod="openshift-dns/dns-default-jvr75" Jul 15 16:56:57.574331 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.574180 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-tls\") pod \"image-registry-795b87c6c7-xztkp\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 16:56:57.574331 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.574205 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wk9j7\" (UniqueName: \"kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-kube-api-access-wk9j7\") pod \"image-registry-795b87c6c7-xztkp\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 16:56:57.574331 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.574237 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/191e57d2-b61f-443a-a7ca-a8a7ad699155-metrics-tls\") pod \"dns-default-jvr75\" (UID: \"191e57d2-b61f-443a-a7ca-a8a7ad699155\") " pod="openshift-dns/dns-default-jvr75" Jul 15 16:56:57.574331 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.574268 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f0d3cec0-475c-481e-beb3-012b1c8c215b-trusted-ca\") pod \"image-registry-795b87c6c7-xztkp\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 16:56:57.574331 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.574301 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-bound-sa-token\") pod \"image-registry-795b87c6c7-xztkp\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 16:56:57.574331 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.574324 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8qcs\" (UniqueName: \"kubernetes.io/projected/f7f08403-7f03-4072-993c-9b02881fd48f-kube-api-access-x8qcs\") pod \"ingress-canary-qsxzw\" (UID: \"f7f08403-7f03-4072-993c-9b02881fd48f\") " pod="openshift-ingress-canary/ingress-canary-qsxzw" Jul 15 16:56:57.574752 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.574405 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/191e57d2-b61f-443a-a7ca-a8a7ad699155-tmp-dir\") pod \"dns-default-jvr75\" (UID: \"191e57d2-b61f-443a-a7ca-a8a7ad699155\") " pod="openshift-dns/dns-default-jvr75" Jul 15 16:56:57.574752 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.574449 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-certificates\") pod \"image-registry-795b87c6c7-xztkp\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 16:56:57.574752 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.574462 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f0d3cec0-475c-481e-beb3-012b1c8c215b-ca-trust-extracted\") pod \"image-registry-795b87c6c7-xztkp\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 16:56:57.574752 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:57.574545 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 16:56:57.574752 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:57.574634 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/191e57d2-b61f-443a-a7ca-a8a7ad699155-metrics-tls podName:191e57d2-b61f-443a-a7ca-a8a7ad699155 nodeName:}" failed. No retries permitted until 2026-07-15 16:56:58.074581731 +0000 UTC m=+35.627711927 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/191e57d2-b61f-443a-a7ca-a8a7ad699155-metrics-tls") pod "dns-default-jvr75" (UID: "191e57d2-b61f-443a-a7ca-a8a7ad699155") : secret "dns-default-metrics-tls" not found Jul 15 16:56:57.574752 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:57.574664 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 16:56:57.574752 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:57.574677 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-795b87c6c7-xztkp: secret "image-registry-tls" not found Jul 15 16:56:57.574752 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:57.574713 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-tls podName:f0d3cec0-475c-481e-beb3-012b1c8c215b nodeName:}" failed. No retries permitted until 2026-07-15 16:56:58.074699234 +0000 UTC m=+35.627829427 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-tls") pod "image-registry-795b87c6c7-xztkp" (UID: "f0d3cec0-475c-481e-beb3-012b1c8c215b") : secret "image-registry-tls" not found Jul 15 16:56:57.575243 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.575049 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-certificates\") pod \"image-registry-795b87c6c7-xztkp\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 16:56:57.575501 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.575476 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f0d3cec0-475c-481e-beb3-012b1c8c215b-trusted-ca\") pod \"image-registry-795b87c6c7-xztkp\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 16:56:57.578082 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.578060 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f0d3cec0-475c-481e-beb3-012b1c8c215b-image-registry-private-configuration\") pod \"image-registry-795b87c6c7-xztkp\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 16:56:57.578082 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.578075 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f7f08403-7f03-4072-993c-9b02881fd48f-cert\") pod \"ingress-canary-qsxzw\" (UID: \"f7f08403-7f03-4072-993c-9b02881fd48f\") " pod="openshift-ingress-canary/ingress-canary-qsxzw" Jul 15 16:56:57.578205 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.578073 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f0d3cec0-475c-481e-beb3-012b1c8c215b-installation-pull-secrets\") pod \"image-registry-795b87c6c7-xztkp\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 16:56:57.582943 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.582919 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzd7h\" (UniqueName: \"kubernetes.io/projected/191e57d2-b61f-443a-a7ca-a8a7ad699155-kube-api-access-tzd7h\") pod \"dns-default-jvr75\" (UID: \"191e57d2-b61f-443a-a7ca-a8a7ad699155\") " pod="openshift-dns/dns-default-jvr75" Jul 15 16:56:57.583086 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.583069 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-bound-sa-token\") pod \"image-registry-795b87c6c7-xztkp\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 16:56:57.583780 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.583756 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wk9j7\" (UniqueName: \"kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-kube-api-access-wk9j7\") pod \"image-registry-795b87c6c7-xztkp\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 16:56:57.583867 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.583851 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8qcs\" (UniqueName: \"kubernetes.io/projected/f7f08403-7f03-4072-993c-9b02881fd48f-kube-api-access-x8qcs\") pod \"ingress-canary-qsxzw\" (UID: \"f7f08403-7f03-4072-993c-9b02881fd48f\") " pod="openshift-ingress-canary/ingress-canary-qsxzw" Jul 15 16:56:57.585054 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.585037 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/191e57d2-b61f-443a-a7ca-a8a7ad699155-config-volume\") pod \"dns-default-jvr75\" (UID: \"191e57d2-b61f-443a-a7ca-a8a7ad699155\") " pod="openshift-dns/dns-default-jvr75" Jul 15 16:56:57.698677 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.698641 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qsxzw" Jul 15 16:56:57.860727 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.860509 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qsxzw"] Jul 15 16:56:57.864230 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:56:57.864206 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7f08403_7f03_4072_993c_9b02881fd48f.slice/crio-b7214485de0c82da703f8458fbdac99317122e0471ee0ac945b4874229016c4d WatchSource:0}: Error finding container b7214485de0c82da703f8458fbdac99317122e0471ee0ac945b4874229016c4d: Status 404 returned error can't find the container with id b7214485de0c82da703f8458fbdac99317122e0471ee0ac945b4874229016c4d Jul 15 16:56:57.984329 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.984296 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 16:56:57.984510 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.984300 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:56:57.984510 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.984467 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r6txx" Jul 15 16:56:57.987264 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.987246 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 16:56:57.987352 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.987247 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 16:56:57.987539 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.987522 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pqk7d\"" Jul 15 16:56:57.987653 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.987637 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 16:56:57.987710 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.987653 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 16:56:57.987710 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:57.987676 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7h7tx\"" Jul 15 16:56:58.080856 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:58.080826 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-tls\") pod \"image-registry-795b87c6c7-xztkp\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 16:56:58.080995 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:58.080863 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/191e57d2-b61f-443a-a7ca-a8a7ad699155-metrics-tls\") pod \"dns-default-jvr75\" (UID: \"191e57d2-b61f-443a-a7ca-a8a7ad699155\") " pod="openshift-dns/dns-default-jvr75" Jul 15 16:56:58.080995 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:58.080976 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 16:56:58.080995 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:58.080993 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-795b87c6c7-xztkp: secret "image-registry-tls" not found Jul 15 16:56:58.081133 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:58.081027 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 16:56:58.081133 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:58.081046 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-tls podName:f0d3cec0-475c-481e-beb3-012b1c8c215b nodeName:}" failed. No retries permitted until 2026-07-15 16:56:59.081027702 +0000 UTC m=+36.634157894 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-tls") pod "image-registry-795b87c6c7-xztkp" (UID: "f0d3cec0-475c-481e-beb3-012b1c8c215b") : secret "image-registry-tls" not found Jul 15 16:56:58.081133 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:58.081077 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/191e57d2-b61f-443a-a7ca-a8a7ad699155-metrics-tls podName:191e57d2-b61f-443a-a7ca-a8a7ad699155 nodeName:}" failed. No retries permitted until 2026-07-15 16:56:59.081060081 +0000 UTC m=+36.634190289 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/191e57d2-b61f-443a-a7ca-a8a7ad699155-metrics-tls") pod "dns-default-jvr75" (UID: "191e57d2-b61f-443a-a7ca-a8a7ad699155") : secret "dns-default-metrics-tls" not found Jul 15 16:56:58.195271 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:58.195232 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qsxzw" event={"ID":"f7f08403-7f03-4072-993c-9b02881fd48f","Type":"ContainerStarted","Data":"b7214485de0c82da703f8458fbdac99317122e0471ee0ac945b4874229016c4d"} Jul 15 16:56:58.198097 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:58.198075 2578 generic.go:356] "Generic (PLEG): container finished" podID="a488304f-ae1d-4dd5-9af5-a1906ce229a9" containerID="533a62307ad5bea816973b38a9dcc9e1afc904f4154a44e5eed9aa32356cf080" exitCode=0 Jul 15 16:56:58.198213 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:58.198113 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tv5fg" event={"ID":"a488304f-ae1d-4dd5-9af5-a1906ce229a9","Type":"ContainerDied","Data":"533a62307ad5bea816973b38a9dcc9e1afc904f4154a44e5eed9aa32356cf080"} Jul 15 16:56:59.088449 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:59.088409 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/191e57d2-b61f-443a-a7ca-a8a7ad699155-metrics-tls\") pod \"dns-default-jvr75\" (UID: \"191e57d2-b61f-443a-a7ca-a8a7ad699155\") " pod="openshift-dns/dns-default-jvr75" Jul 15 16:56:59.088657 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:59.088532 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-tls\") pod \"image-registry-795b87c6c7-xztkp\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 16:56:59.088657 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:59.088576 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 16:56:59.088657 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:59.088653 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 16:56:59.088805 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:59.088669 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-795b87c6c7-xztkp: secret "image-registry-tls" not found Jul 15 16:56:59.088805 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:59.088659 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/191e57d2-b61f-443a-a7ca-a8a7ad699155-metrics-tls podName:191e57d2-b61f-443a-a7ca-a8a7ad699155 nodeName:}" failed. No retries permitted until 2026-07-15 16:57:01.088642333 +0000 UTC m=+38.641772527 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/191e57d2-b61f-443a-a7ca-a8a7ad699155-metrics-tls") pod "dns-default-jvr75" (UID: "191e57d2-b61f-443a-a7ca-a8a7ad699155") : secret "dns-default-metrics-tls" not found Jul 15 16:56:59.088805 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:56:59.088725 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-tls podName:f0d3cec0-475c-481e-beb3-012b1c8c215b nodeName:}" failed. No retries permitted until 2026-07-15 16:57:01.088709529 +0000 UTC m=+38.641839721 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-tls") pod "image-registry-795b87c6c7-xztkp" (UID: "f0d3cec0-475c-481e-beb3-012b1c8c215b") : secret "image-registry-tls" not found Jul 15 16:56:59.203246 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:59.203208 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tv5fg" event={"ID":"a488304f-ae1d-4dd5-9af5-a1906ce229a9","Type":"ContainerStarted","Data":"abee7160c5c0237356eab8434e5a7dcc1a02c0ef68690c1d0ef477dff0dbe083"} Jul 15 16:56:59.229821 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:59.229763 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-tv5fg" podStartSLOduration=10.750838042 podStartE2EDuration="36.229742959s" totalImagesPullingTime="25.478904917s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:56:23 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:56:59.226097248 +0000 UTC m=+36.779227462" watchObservedRunningTime="2026-07-15 16:56:59.229742959 +0000 UTC m=+36.782873175" Jul 15 16:56:59.409776 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:59.409704 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 16:56:59.533865 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:56:59.533833 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 16:57:01.103837 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:01.103758 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-tls\") pod \"image-registry-795b87c6c7-xztkp\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 16:57:01.103837 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:01.103795 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/191e57d2-b61f-443a-a7ca-a8a7ad699155-metrics-tls\") pod \"dns-default-jvr75\" (UID: \"191e57d2-b61f-443a-a7ca-a8a7ad699155\") " pod="openshift-dns/dns-default-jvr75" Jul 15 16:57:01.104243 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:57:01.103890 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 16:57:01.104243 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:57:01.103910 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 16:57:01.104243 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:57:01.103932 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-795b87c6c7-xztkp: secret "image-registry-tls" not found Jul 15 16:57:01.104243 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:57:01.103939 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/191e57d2-b61f-443a-a7ca-a8a7ad699155-metrics-tls podName:191e57d2-b61f-443a-a7ca-a8a7ad699155 nodeName:}" failed. No retries permitted until 2026-07-15 16:57:05.103926945 +0000 UTC m=+42.657057137 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/191e57d2-b61f-443a-a7ca-a8a7ad699155-metrics-tls") pod "dns-default-jvr75" (UID: "191e57d2-b61f-443a-a7ca-a8a7ad699155") : secret "dns-default-metrics-tls" not found Jul 15 16:57:01.104243 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:57:01.104019 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-tls podName:f0d3cec0-475c-481e-beb3-012b1c8c215b nodeName:}" failed. No retries permitted until 2026-07-15 16:57:05.104000922 +0000 UTC m=+42.657131134 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-tls") pod "image-registry-795b87c6c7-xztkp" (UID: "f0d3cec0-475c-481e-beb3-012b1c8c215b") : secret "image-registry-tls" not found Jul 15 16:57:01.209158 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:01.209124 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qsxzw" event={"ID":"f7f08403-7f03-4072-993c-9b02881fd48f","Type":"ContainerStarted","Data":"ad3ca10d0ce688b9cd8d1a3a436e6a9c1a4fee940524f5bb5fe0b820376befbd"} Jul 15 16:57:01.228074 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:01.228030 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qsxzw" podStartSLOduration=1.3516790730000001 podStartE2EDuration="4.228016099s" totalImagesPullingTime="2.876337026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:56:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:57:01.227005914 +0000 UTC m=+38.780136128" watchObservedRunningTime="2026-07-15 16:57:01.228016099 +0000 UTC m=+38.781146310" Jul 15 16:57:01.929043 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:01.928866 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 16:57:01.992209 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:01.992182 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 16:57:02.303633 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:02.303543 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 16:57:02.369992 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:02.369962 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 16:57:03.742466 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:03.742432 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 16:57:03.930575 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:03.930543 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 16:57:04.517164 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:04.517129 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 16:57:04.619313 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:04.619284 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 16:57:04.958740 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:04.958703 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 16:57:05.114852 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:05.114824 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 16:57:05.130860 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:05.130837 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-tls\") pod \"image-registry-795b87c6c7-xztkp\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 16:57:05.130936 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:05.130868 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/191e57d2-b61f-443a-a7ca-a8a7ad699155-metrics-tls\") pod \"dns-default-jvr75\" (UID: \"191e57d2-b61f-443a-a7ca-a8a7ad699155\") " pod="openshift-dns/dns-default-jvr75" Jul 15 16:57:05.130973 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:57:05.130966 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 16:57:05.131010 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:57:05.130969 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 16:57:05.131010 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:57:05.130985 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-795b87c6c7-xztkp: secret "image-registry-tls" not found Jul 15 16:57:05.131068 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:57:05.131024 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/191e57d2-b61f-443a-a7ca-a8a7ad699155-metrics-tls podName:191e57d2-b61f-443a-a7ca-a8a7ad699155 nodeName:}" failed. No retries permitted until 2026-07-15 16:57:13.131010218 +0000 UTC m=+50.684140411 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/191e57d2-b61f-443a-a7ca-a8a7ad699155-metrics-tls") pod "dns-default-jvr75" (UID: "191e57d2-b61f-443a-a7ca-a8a7ad699155") : secret "dns-default-metrics-tls" not found Jul 15 16:57:05.131068 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:57:05.131038 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-tls podName:f0d3cec0-475c-481e-beb3-012b1c8c215b nodeName:}" failed. No retries permitted until 2026-07-15 16:57:13.131032475 +0000 UTC m=+50.684162667 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-tls") pod "image-registry-795b87c6c7-xztkp" (UID: "f0d3cec0-475c-481e-beb3-012b1c8c215b") : secret "image-registry-tls" not found Jul 15 16:57:09.549068 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:09.549037 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 16:57:09.611540 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:09.611512 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 16:57:13.181784 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:13.181741 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-tls\") pod \"image-registry-795b87c6c7-xztkp\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 16:57:13.181784 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:13.181785 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/191e57d2-b61f-443a-a7ca-a8a7ad699155-metrics-tls\") pod \"dns-default-jvr75\" (UID: \"191e57d2-b61f-443a-a7ca-a8a7ad699155\") " pod="openshift-dns/dns-default-jvr75" Jul 15 16:57:13.182221 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:57:13.181888 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 16:57:13.182221 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:57:13.181940 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/191e57d2-b61f-443a-a7ca-a8a7ad699155-metrics-tls podName:191e57d2-b61f-443a-a7ca-a8a7ad699155 nodeName:}" failed. No retries permitted until 2026-07-15 16:57:29.181926497 +0000 UTC m=+66.735056689 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/191e57d2-b61f-443a-a7ca-a8a7ad699155-metrics-tls") pod "dns-default-jvr75" (UID: "191e57d2-b61f-443a-a7ca-a8a7ad699155") : secret "dns-default-metrics-tls" not found Jul 15 16:57:13.182221 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:57:13.181888 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 16:57:13.182221 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:57:13.181953 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-795b87c6c7-xztkp: secret "image-registry-tls" not found Jul 15 16:57:13.182221 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:57:13.181981 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-tls podName:f0d3cec0-475c-481e-beb3-012b1c8c215b nodeName:}" failed. No retries permitted until 2026-07-15 16:57:29.181971808 +0000 UTC m=+66.735102001 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-tls") pod "image-registry-795b87c6c7-xztkp" (UID: "f0d3cec0-475c-481e-beb3-012b1c8c215b") : secret "image-registry-tls" not found Jul 15 16:57:22.189012 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:22.188984 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-29wtx" Jul 15 16:57:22.348106 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:22.348070 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 16:57:22.420312 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:22.420284 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 16:57:24.918313 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:24.918273 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 16:57:25.019063 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:25.019032 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 16:57:25.922584 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:25.922556 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 16:57:25.999950 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:25.999922 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 16:57:27.597814 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:27.597776 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-metrics-certs\") pod \"network-metrics-daemon-62hbp\" (UID: \"df1447df-1b9a-4dc3-ba2c-ecbab1514d09\") " pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 16:57:27.600671 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:27.600650 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 16:57:27.608867 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:57:27.608849 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 15 16:57:27.608909 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:57:27.608902 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-metrics-certs podName:df1447df-1b9a-4dc3-ba2c-ecbab1514d09 nodeName:}" failed. No retries permitted until 2026-07-15 16:58:31.608888207 +0000 UTC m=+129.162018398 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-metrics-certs") pod "network-metrics-daemon-62hbp" (UID: "df1447df-1b9a-4dc3-ba2c-ecbab1514d09") : secret "metrics-daemon-secret" not found Jul 15 16:57:27.698225 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:27.698197 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b25e2334-b15b-4c03-9754-cd9a14f177cc-original-pull-secret\") pod \"global-pull-secret-syncer-nzk7p\" (UID: \"b25e2334-b15b-4c03-9754-cd9a14f177cc\") " pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:57:27.701083 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:27.701064 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 16:57:27.712107 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:27.712089 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b25e2334-b15b-4c03-9754-cd9a14f177cc-original-pull-secret\") pod \"global-pull-secret-syncer-nzk7p\" (UID: \"b25e2334-b15b-4c03-9754-cd9a14f177cc\") " pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:57:27.798826 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:27.798799 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx7gk\" (UniqueName: \"kubernetes.io/projected/1542bfc7-eb45-488a-b867-257a2978997b-kube-api-access-gx7gk\") pod \"network-check-target-r6txx\" (UID: \"1542bfc7-eb45-488a-b867-257a2978997b\") " pod="openshift-network-diagnostics/network-check-target-r6txx" Jul 15 16:57:27.801733 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:27.801714 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 16:57:27.812115 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:27.812097 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 16:57:27.822300 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:27.822277 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx7gk\" (UniqueName: \"kubernetes.io/projected/1542bfc7-eb45-488a-b867-257a2978997b-kube-api-access-gx7gk\") pod \"network-check-target-r6txx\" (UID: \"1542bfc7-eb45-488a-b867-257a2978997b\") " pod="openshift-network-diagnostics/network-check-target-r6txx" Jul 15 16:57:28.000558 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:28.000527 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzk7p" Jul 15 16:57:28.008342 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:28.008323 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7h7tx\"" Jul 15 16:57:28.016451 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:28.016434 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r6txx" Jul 15 16:57:28.175857 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:28.175826 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nzk7p"] Jul 15 16:57:28.178766 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:57:28.178742 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb25e2334_b15b_4c03_9754_cd9a14f177cc.slice/crio-89c1c4dc843dafb964458794afb69fb91d0732285a950ed8649d44df919bf709 WatchSource:0}: Error finding container 89c1c4dc843dafb964458794afb69fb91d0732285a950ed8649d44df919bf709: Status 404 returned error can't find the container with id 89c1c4dc843dafb964458794afb69fb91d0732285a950ed8649d44df919bf709 Jul 15 16:57:28.192514 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:28.192489 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r6txx"] Jul 15 16:57:28.195123 ip-10-0-140-149 kubenswrapper[2578]: W0715 16:57:28.195101 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1542bfc7_eb45_488a_b867_257a2978997b.slice/crio-f0f07210dc0234c5d668c0c32351f887e9ed7795756e81c5289c53ab257e3b36 WatchSource:0}: Error finding container f0f07210dc0234c5d668c0c32351f887e9ed7795756e81c5289c53ab257e3b36: Status 404 returned error can't find the container with id f0f07210dc0234c5d668c0c32351f887e9ed7795756e81c5289c53ab257e3b36 Jul 15 16:57:28.269160 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:28.269104 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r6txx" event={"ID":"1542bfc7-eb45-488a-b867-257a2978997b","Type":"ContainerStarted","Data":"f0f07210dc0234c5d668c0c32351f887e9ed7795756e81c5289c53ab257e3b36"} Jul 15 16:57:28.270138 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:28.270120 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nzk7p" event={"ID":"b25e2334-b15b-4c03-9754-cd9a14f177cc","Type":"ContainerStarted","Data":"89c1c4dc843dafb964458794afb69fb91d0732285a950ed8649d44df919bf709"} Jul 15 16:57:29.210029 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:29.209996 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-tls\") pod \"image-registry-795b87c6c7-xztkp\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 16:57:29.210458 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:29.210050 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/191e57d2-b61f-443a-a7ca-a8a7ad699155-metrics-tls\") pod \"dns-default-jvr75\" (UID: \"191e57d2-b61f-443a-a7ca-a8a7ad699155\") " pod="openshift-dns/dns-default-jvr75" Jul 15 16:57:29.210458 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:57:29.210168 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 16:57:29.210458 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:57:29.210192 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-795b87c6c7-xztkp: secret "image-registry-tls" not found Jul 15 16:57:29.210458 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:57:29.210204 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 16:57:29.210458 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:57:29.210256 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/191e57d2-b61f-443a-a7ca-a8a7ad699155-metrics-tls podName:191e57d2-b61f-443a-a7ca-a8a7ad699155 nodeName:}" failed. No retries permitted until 2026-07-15 16:58:01.210237217 +0000 UTC m=+98.763367423 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/191e57d2-b61f-443a-a7ca-a8a7ad699155-metrics-tls") pod "dns-default-jvr75" (UID: "191e57d2-b61f-443a-a7ca-a8a7ad699155") : secret "dns-default-metrics-tls" not found Jul 15 16:57:29.210458 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:57:29.210273 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-tls podName:f0d3cec0-475c-481e-beb3-012b1c8c215b nodeName:}" failed. No retries permitted until 2026-07-15 16:58:01.210263934 +0000 UTC m=+98.763394132 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-tls") pod "image-registry-795b87c6c7-xztkp" (UID: "f0d3cec0-475c-481e-beb3-012b1c8c215b") : secret "image-registry-tls" not found Jul 15 16:57:31.279822 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:31.279792 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r6txx" event={"ID":"1542bfc7-eb45-488a-b867-257a2978997b","Type":"ContainerStarted","Data":"5cc4446f0eea37c969ef290679fecc25989a0d0675ff3b43c8229b261e7d2b7c"} Jul 15 16:57:31.280166 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:31.279995 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-r6txx" Jul 15 16:57:31.297574 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:31.297492 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-r6txx" podStartSLOduration=65.450701467 podStartE2EDuration="1m8.297480143s" totalImagesPullingTime="2.846778676s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:57:31.29681907 +0000 UTC m=+68.849949283" watchObservedRunningTime="2026-07-15 16:57:31.297480143 +0000 UTC m=+68.850610357" Jul 15 16:57:34.288403 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:34.288364 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nzk7p" event={"ID":"b25e2334-b15b-4c03-9754-cd9a14f177cc","Type":"ContainerStarted","Data":"bfd772981dea01dfb3f80caf55417cf8de3610c1a1d0471cbc8b0f5b1873565c"} Jul 15 16:57:34.304176 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:34.304134 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-nzk7p" podStartSLOduration=66.054539066 podStartE2EDuration="1m11.30411695s" totalImagesPullingTime="5.249577884s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:57:34.303972386 +0000 UTC m=+71.857102619" watchObservedRunningTime="2026-07-15 16:57:34.30411695 +0000 UTC m=+71.857247191" Jul 15 16:57:37.533484 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:37.533410 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 16:57:37.599186 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:37.599158 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 16:57:57.336230 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:57:57.336108 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 16:58:01.224970 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:58:01.224930 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-tls\") pod \"image-registry-795b87c6c7-xztkp\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 16:58:01.224970 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:58:01.224972 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/191e57d2-b61f-443a-a7ca-a8a7ad699155-metrics-tls\") pod \"dns-default-jvr75\" (UID: \"191e57d2-b61f-443a-a7ca-a8a7ad699155\") " pod="openshift-dns/dns-default-jvr75" Jul 15 16:58:01.225357 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:58:01.225064 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 16:58:01.225357 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:58:01.225070 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 16:58:01.225357 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:58:01.225087 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-795b87c6c7-xztkp: secret "image-registry-tls" not found Jul 15 16:58:01.225357 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:58:01.225127 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/191e57d2-b61f-443a-a7ca-a8a7ad699155-metrics-tls podName:191e57d2-b61f-443a-a7ca-a8a7ad699155 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:05.225112271 +0000 UTC m=+162.778242462 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/191e57d2-b61f-443a-a7ca-a8a7ad699155-metrics-tls") pod "dns-default-jvr75" (UID: "191e57d2-b61f-443a-a7ca-a8a7ad699155") : secret "dns-default-metrics-tls" not found Jul 15 16:58:01.225357 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:58:01.225140 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-tls podName:f0d3cec0-475c-481e-beb3-012b1c8c215b nodeName:}" failed. No retries permitted until 2026-07-15 16:59:05.225134634 +0000 UTC m=+162.778264827 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-tls") pod "image-registry-795b87c6c7-xztkp" (UID: "f0d3cec0-475c-481e-beb3-012b1c8c215b") : secret "image-registry-tls" not found Jul 15 16:58:02.284531 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:58:02.284504 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-r6txx" Jul 15 16:58:09.053010 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:58:09.052978 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 16:58:31.627063 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:58:31.627011 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-metrics-certs\") pod \"network-metrics-daemon-62hbp\" (UID: \"df1447df-1b9a-4dc3-ba2c-ecbab1514d09\") " pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 16:58:31.627548 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:58:31.627159 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 15 16:58:31.627548 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:58:31.627221 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-metrics-certs podName:df1447df-1b9a-4dc3-ba2c-ecbab1514d09 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:33.627203487 +0000 UTC m=+251.180333698 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-metrics-certs") pod "network-metrics-daemon-62hbp" (UID: "df1447df-1b9a-4dc3-ba2c-ecbab1514d09") : secret "metrics-daemon-secret" not found Jul 15 16:59:00.388757 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:59:00.388718 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" podUID="f0d3cec0-475c-481e-beb3-012b1c8c215b" Jul 15 16:59:00.394819 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:59:00.394795 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-jvr75" podUID="191e57d2-b61f-443a-a7ca-a8a7ad699155" Jul 15 16:59:00.488893 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:59:00.488868 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 16:59:00.488893 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:59:00.488881 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jvr75" Jul 15 16:59:00.994982 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:59:00.994944 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-62hbp" podUID="df1447df-1b9a-4dc3-ba2c-ecbab1514d09" Jul 15 16:59:05.261310 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:59:05.261270 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-tls\") pod \"image-registry-795b87c6c7-xztkp\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 16:59:05.261310 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:59:05.261315 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/191e57d2-b61f-443a-a7ca-a8a7ad699155-metrics-tls\") pod \"dns-default-jvr75\" (UID: \"191e57d2-b61f-443a-a7ca-a8a7ad699155\") " pod="openshift-dns/dns-default-jvr75" Jul 15 16:59:05.261884 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:59:05.261429 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 16:59:05.261884 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:59:05.261445 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 16:59:05.261884 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:59:05.261453 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-795b87c6c7-xztkp: secret "image-registry-tls" not found Jul 15 16:59:05.261884 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:59:05.261511 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-tls podName:f0d3cec0-475c-481e-beb3-012b1c8c215b nodeName:}" failed. No retries permitted until 2026-07-15 17:01:07.26149206 +0000 UTC m=+284.814622266 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-tls") pod "image-registry-795b87c6c7-xztkp" (UID: "f0d3cec0-475c-481e-beb3-012b1c8c215b") : secret "image-registry-tls" not found Jul 15 16:59:05.261884 ip-10-0-140-149 kubenswrapper[2578]: E0715 16:59:05.261530 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/191e57d2-b61f-443a-a7ca-a8a7ad699155-metrics-tls podName:191e57d2-b61f-443a-a7ca-a8a7ad699155 nodeName:}" failed. No retries permitted until 2026-07-15 17:01:07.261520418 +0000 UTC m=+284.814650613 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/191e57d2-b61f-443a-a7ca-a8a7ad699155-metrics-tls") pod "dns-default-jvr75" (UID: "191e57d2-b61f-443a-a7ca-a8a7ad699155") : secret "dns-default-metrics-tls" not found Jul 15 16:59:13.984733 ip-10-0-140-149 kubenswrapper[2578]: I0715 16:59:13.984684 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 17:00:04.431019 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:04.430981 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:33.656561 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:33.656518 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-metrics-certs\") pod \"network-metrics-daemon-62hbp\" (UID: \"df1447df-1b9a-4dc3-ba2c-ecbab1514d09\") " pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 17:00:33.656960 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:33.656695 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 15 17:00:33.656960 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:33.656760 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-metrics-certs podName:df1447df-1b9a-4dc3-ba2c-ecbab1514d09 nodeName:}" failed. No retries permitted until 2026-07-15 17:02:35.656744095 +0000 UTC m=+373.209874287 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-metrics-certs") pod "network-metrics-daemon-62hbp" (UID: "df1447df-1b9a-4dc3-ba2c-ecbab1514d09") : secret "metrics-daemon-secret" not found Jul 15 17:00:37.843216 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.843184 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qtxm9"] Jul 15 17:00:37.845914 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.845892 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-5d55j"] Jul 15 17:00:37.846049 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.846030 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qtxm9" Jul 15 17:00:37.848618 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.848526 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-fzc59"] Jul 15 17:00:37.848786 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.848618 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 15 17:00:37.848786 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.848722 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-5d55j" Jul 15 17:00:37.848921 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.848618 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-qqr56\"" Jul 15 17:00:37.851189 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.851171 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 15 17:00:37.851363 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.851337 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 15 17:00:37.851363 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.851351 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-fzc59" Jul 15 17:00:37.851616 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.851429 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 15 17:00:37.852168 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.852152 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-hthbq"] Jul 15 17:00:37.852788 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.852765 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 15 17:00:37.853218 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.853198 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 15 17:00:37.855001 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.854977 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-b6qpq\"" Jul 15 17:00:37.856540 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.856421 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94wg5"] Jul 15 17:00:37.856757 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.856743 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-hthbq" Jul 15 17:00:37.856965 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.856945 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 15 17:00:37.857238 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.857214 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 15 17:00:37.857697 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.857678 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-jlp4n\"" Jul 15 17:00:37.859736 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.859583 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 15 17:00:37.859736 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.859621 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-7pbr8\"" Jul 15 17:00:37.860067 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.860045 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qtxm9"] Jul 15 17:00:37.860154 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.860106 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 15 17:00:37.860154 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.860131 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94wg5" Jul 15 17:00:37.860629 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.860301 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 15 17:00:37.860765 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.860642 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 15 17:00:37.862741 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.862392 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-5d55j"] Jul 15 17:00:37.863916 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.863713 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 15 17:00:37.863916 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.863729 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 15 17:00:37.864517 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.864497 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 15 17:00:37.864741 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.864724 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-wz4cv\"" Jul 15 17:00:37.864817 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.864806 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-hthbq"] Jul 15 17:00:37.865997 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.865976 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 15 17:00:37.873918 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.873895 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-fzc59"] Jul 15 17:00:37.878181 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.878162 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94wg5"] Jul 15 17:00:37.884984 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.884967 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5-snapshots\") pod \"insights-operator-68d9dd94cf-hthbq\" (UID: \"f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hthbq" Jul 15 17:00:37.885065 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.884997 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gssc7\" (UniqueName: \"kubernetes.io/projected/68d72523-54cd-412a-b697-0f04e891d984-kube-api-access-gssc7\") pod \"cluster-monitoring-operator-5f7f494dd4-qtxm9\" (UID: \"68d72523-54cd-412a-b697-0f04e891d984\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qtxm9" Jul 15 17:00:37.885065 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.885026 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dqjp\" (UniqueName: \"kubernetes.io/projected/f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5-kube-api-access-5dqjp\") pod \"insights-operator-68d9dd94cf-hthbq\" (UID: \"f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hthbq" Jul 15 17:00:37.885065 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.885045 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bbfcb238-e568-4881-9f87-6cbad5fe3674-nginx-conf\") pod \"networking-console-plugin-6cf856758c-fzc59\" (UID: \"bbfcb238-e568-4881-9f87-6cbad5fe3674\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-fzc59" Jul 15 17:00:37.885164 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.885090 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-hthbq\" (UID: \"f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hthbq" Jul 15 17:00:37.885164 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.885118 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfrn8\" (UniqueName: \"kubernetes.io/projected/cfb96618-c779-4569-8fbb-01cae188529c-kube-api-access-sfrn8\") pod \"cluster-samples-operator-65db858894-94wg5\" (UID: \"cfb96618-c779-4569-8fbb-01cae188529c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94wg5" Jul 15 17:00:37.885164 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.885157 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5-tmp\") pod \"insights-operator-68d9dd94cf-hthbq\" (UID: \"f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hthbq" Jul 15 17:00:37.885271 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.885177 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-hthbq\" (UID: \"f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hthbq" Jul 15 17:00:37.885271 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.885198 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvc9r\" (UniqueName: \"kubernetes.io/projected/7a492e65-62e9-48e5-ae6d-f04e48964b37-kube-api-access-hvc9r\") pod \"volume-data-source-validator-78495597f6-5d55j\" (UID: \"7a492e65-62e9-48e5-ae6d-f04e48964b37\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-5d55j" Jul 15 17:00:37.885271 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.885235 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cfb96618-c779-4569-8fbb-01cae188529c-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-94wg5\" (UID: \"cfb96618-c779-4569-8fbb-01cae188529c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94wg5" Jul 15 17:00:37.885271 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.885265 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/68d72523-54cd-412a-b697-0f04e891d984-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-qtxm9\" (UID: \"68d72523-54cd-412a-b697-0f04e891d984\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qtxm9" Jul 15 17:00:37.885411 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.885283 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bbfcb238-e568-4881-9f87-6cbad5fe3674-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-fzc59\" (UID: \"bbfcb238-e568-4881-9f87-6cbad5fe3674\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-fzc59" Jul 15 17:00:37.885411 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.885302 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5-serving-cert\") pod \"insights-operator-68d9dd94cf-hthbq\" (UID: \"f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hthbq" Jul 15 17:00:37.885411 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.885374 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/68d72523-54cd-412a-b697-0f04e891d984-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-qtxm9\" (UID: \"68d72523-54cd-412a-b697-0f04e891d984\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qtxm9" Jul 15 17:00:37.946213 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.946182 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-mmlx6"] Jul 15 17:00:37.949151 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.949137 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-mmlx6" Jul 15 17:00:37.951756 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.951738 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-cv2hn\"" Jul 15 17:00:37.964158 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.964131 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-mmlx6"] Jul 15 17:00:37.986468 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.986444 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5-snapshots\") pod \"insights-operator-68d9dd94cf-hthbq\" (UID: \"f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hthbq" Jul 15 17:00:37.986568 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.986474 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gssc7\" (UniqueName: \"kubernetes.io/projected/68d72523-54cd-412a-b697-0f04e891d984-kube-api-access-gssc7\") pod \"cluster-monitoring-operator-5f7f494dd4-qtxm9\" (UID: \"68d72523-54cd-412a-b697-0f04e891d984\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qtxm9" Jul 15 17:00:37.986568 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.986495 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dqjp\" (UniqueName: \"kubernetes.io/projected/f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5-kube-api-access-5dqjp\") pod \"insights-operator-68d9dd94cf-hthbq\" (UID: \"f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hthbq" Jul 15 17:00:37.986568 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.986513 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bbfcb238-e568-4881-9f87-6cbad5fe3674-nginx-conf\") pod \"networking-console-plugin-6cf856758c-fzc59\" (UID: \"bbfcb238-e568-4881-9f87-6cbad5fe3674\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-fzc59" Jul 15 17:00:37.986734 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.986572 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-hthbq\" (UID: \"f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hthbq" Jul 15 17:00:37.986734 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.986621 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfrn8\" (UniqueName: \"kubernetes.io/projected/cfb96618-c779-4569-8fbb-01cae188529c-kube-api-access-sfrn8\") pod \"cluster-samples-operator-65db858894-94wg5\" (UID: \"cfb96618-c779-4569-8fbb-01cae188529c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94wg5" Jul 15 17:00:37.986833 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.986731 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5-tmp\") pod \"insights-operator-68d9dd94cf-hthbq\" (UID: \"f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hthbq" Jul 15 17:00:37.986833 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.986764 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-hthbq\" (UID: \"f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hthbq" Jul 15 17:00:37.986833 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.986799 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvc9r\" (UniqueName: \"kubernetes.io/projected/7a492e65-62e9-48e5-ae6d-f04e48964b37-kube-api-access-hvc9r\") pod \"volume-data-source-validator-78495597f6-5d55j\" (UID: \"7a492e65-62e9-48e5-ae6d-f04e48964b37\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-5d55j" Jul 15 17:00:37.986833 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.986829 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4s2rg\" (UniqueName: \"kubernetes.io/projected/165963cd-378d-47e0-9b04-e6b01478473e-kube-api-access-4s2rg\") pod \"network-check-source-869dcfcdcd-mmlx6\" (UID: \"165963cd-378d-47e0-9b04-e6b01478473e\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-mmlx6" Jul 15 17:00:37.987024 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.986964 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cfb96618-c779-4569-8fbb-01cae188529c-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-94wg5\" (UID: \"cfb96618-c779-4569-8fbb-01cae188529c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94wg5" Jul 15 17:00:37.987024 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.987001 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/68d72523-54cd-412a-b697-0f04e891d984-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-qtxm9\" (UID: \"68d72523-54cd-412a-b697-0f04e891d984\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qtxm9" Jul 15 17:00:37.987126 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.987028 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bbfcb238-e568-4881-9f87-6cbad5fe3674-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-fzc59\" (UID: \"bbfcb238-e568-4881-9f87-6cbad5fe3674\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-fzc59" Jul 15 17:00:37.987126 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.987036 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5-tmp\") pod \"insights-operator-68d9dd94cf-hthbq\" (UID: \"f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hthbq" Jul 15 17:00:37.987126 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.987053 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5-serving-cert\") pod \"insights-operator-68d9dd94cf-hthbq\" (UID: \"f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hthbq" Jul 15 17:00:37.987126 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:37.987079 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 17:00:37.987126 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.987089 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/68d72523-54cd-412a-b697-0f04e891d984-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-qtxm9\" (UID: \"68d72523-54cd-412a-b697-0f04e891d984\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qtxm9" Jul 15 17:00:37.987126 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.987105 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5-snapshots\") pod \"insights-operator-68d9dd94cf-hthbq\" (UID: \"f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hthbq" Jul 15 17:00:37.987126 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:37.987125 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 17:00:37.987485 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:37.987141 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfb96618-c779-4569-8fbb-01cae188529c-samples-operator-tls podName:cfb96618-c779-4569-8fbb-01cae188529c nodeName:}" failed. No retries permitted until 2026-07-15 17:00:38.487123326 +0000 UTC m=+256.040253524 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cfb96618-c779-4569-8fbb-01cae188529c-samples-operator-tls") pod "cluster-samples-operator-65db858894-94wg5" (UID: "cfb96618-c779-4569-8fbb-01cae188529c") : secret "samples-operator-tls" not found Jul 15 17:00:37.987485 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:37.987198 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbfcb238-e568-4881-9f87-6cbad5fe3674-networking-console-plugin-cert podName:bbfcb238-e568-4881-9f87-6cbad5fe3674 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:38.487181445 +0000 UTC m=+256.040311639 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/bbfcb238-e568-4881-9f87-6cbad5fe3674-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-fzc59" (UID: "bbfcb238-e568-4881-9f87-6cbad5fe3674") : secret "networking-console-plugin-cert" not found Jul 15 17:00:37.987485 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:37.987209 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 17:00:37.987485 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:37.987251 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68d72523-54cd-412a-b697-0f04e891d984-cluster-monitoring-operator-tls podName:68d72523-54cd-412a-b697-0f04e891d984 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:38.48723974 +0000 UTC m=+256.040369933 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/68d72523-54cd-412a-b697-0f04e891d984-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-qtxm9" (UID: "68d72523-54cd-412a-b697-0f04e891d984") : secret "cluster-monitoring-operator-tls" not found Jul 15 17:00:37.987485 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.987279 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bbfcb238-e568-4881-9f87-6cbad5fe3674-nginx-conf\") pod \"networking-console-plugin-6cf856758c-fzc59\" (UID: \"bbfcb238-e568-4881-9f87-6cbad5fe3674\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-fzc59" Jul 15 17:00:37.987792 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.987529 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-hthbq\" (UID: \"f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hthbq" Jul 15 17:00:37.987792 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.987734 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-hthbq\" (UID: \"f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hthbq" Jul 15 17:00:37.987883 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.987816 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/68d72523-54cd-412a-b697-0f04e891d984-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-qtxm9\" (UID: \"68d72523-54cd-412a-b697-0f04e891d984\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qtxm9" Jul 15 17:00:37.989236 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.989218 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5-serving-cert\") pod \"insights-operator-68d9dd94cf-hthbq\" (UID: \"f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hthbq" Jul 15 17:00:37.999542 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:37.999520 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfrn8\" (UniqueName: \"kubernetes.io/projected/cfb96618-c779-4569-8fbb-01cae188529c-kube-api-access-sfrn8\") pod \"cluster-samples-operator-65db858894-94wg5\" (UID: \"cfb96618-c779-4569-8fbb-01cae188529c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94wg5" Jul 15 17:00:38.001436 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.001414 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dqjp\" (UniqueName: \"kubernetes.io/projected/f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5-kube-api-access-5dqjp\") pod \"insights-operator-68d9dd94cf-hthbq\" (UID: \"f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hthbq" Jul 15 17:00:38.001909 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.001892 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gssc7\" (UniqueName: \"kubernetes.io/projected/68d72523-54cd-412a-b697-0f04e891d984-kube-api-access-gssc7\") pod \"cluster-monitoring-operator-5f7f494dd4-qtxm9\" (UID: \"68d72523-54cd-412a-b697-0f04e891d984\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qtxm9" Jul 15 17:00:38.002082 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.002067 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvc9r\" (UniqueName: \"kubernetes.io/projected/7a492e65-62e9-48e5-ae6d-f04e48964b37-kube-api-access-hvc9r\") pod \"volume-data-source-validator-78495597f6-5d55j\" (UID: \"7a492e65-62e9-48e5-ae6d-f04e48964b37\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-5d55j" Jul 15 17:00:38.037043 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.037011 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p9wk7"] Jul 15 17:00:38.041193 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.041177 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p9wk7" Jul 15 17:00:38.043521 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.043502 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 15 17:00:38.043893 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.043875 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-jnqjf\"" Jul 15 17:00:38.043977 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.043917 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 15 17:00:38.043977 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.043943 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 15 17:00:38.044084 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.044001 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 15 17:00:38.048995 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.048975 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p9wk7"] Jul 15 17:00:38.088063 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.088034 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4s2rg\" (UniqueName: \"kubernetes.io/projected/165963cd-378d-47e0-9b04-e6b01478473e-kube-api-access-4s2rg\") pod \"network-check-source-869dcfcdcd-mmlx6\" (UID: \"165963cd-378d-47e0-9b04-e6b01478473e\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-mmlx6" Jul 15 17:00:38.088188 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.088095 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v2zz\" (UniqueName: \"kubernetes.io/projected/0055caa8-1ee7-4b69-8dba-d1201107d12a-kube-api-access-8v2zz\") pod \"service-ca-operator-7695fc6c8c-p9wk7\" (UID: \"0055caa8-1ee7-4b69-8dba-d1201107d12a\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p9wk7" Jul 15 17:00:38.088188 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.088153 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0055caa8-1ee7-4b69-8dba-d1201107d12a-serving-cert\") pod \"service-ca-operator-7695fc6c8c-p9wk7\" (UID: \"0055caa8-1ee7-4b69-8dba-d1201107d12a\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p9wk7" Jul 15 17:00:38.088299 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.088201 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0055caa8-1ee7-4b69-8dba-d1201107d12a-config\") pod \"service-ca-operator-7695fc6c8c-p9wk7\" (UID: \"0055caa8-1ee7-4b69-8dba-d1201107d12a\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p9wk7" Jul 15 17:00:38.095835 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.095787 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4s2rg\" (UniqueName: \"kubernetes.io/projected/165963cd-378d-47e0-9b04-e6b01478473e-kube-api-access-4s2rg\") pod \"network-check-source-869dcfcdcd-mmlx6\" (UID: \"165963cd-378d-47e0-9b04-e6b01478473e\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-mmlx6" Jul 15 17:00:38.111657 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.111637 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:38.166579 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.166555 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-5d55j" Jul 15 17:00:38.180716 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.180692 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-hthbq" Jul 15 17:00:38.189145 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.189119 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v2zz\" (UniqueName: \"kubernetes.io/projected/0055caa8-1ee7-4b69-8dba-d1201107d12a-kube-api-access-8v2zz\") pod \"service-ca-operator-7695fc6c8c-p9wk7\" (UID: \"0055caa8-1ee7-4b69-8dba-d1201107d12a\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p9wk7" Jul 15 17:00:38.189245 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.189186 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0055caa8-1ee7-4b69-8dba-d1201107d12a-serving-cert\") pod \"service-ca-operator-7695fc6c8c-p9wk7\" (UID: \"0055caa8-1ee7-4b69-8dba-d1201107d12a\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p9wk7" Jul 15 17:00:38.189245 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.189229 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0055caa8-1ee7-4b69-8dba-d1201107d12a-config\") pod \"service-ca-operator-7695fc6c8c-p9wk7\" (UID: \"0055caa8-1ee7-4b69-8dba-d1201107d12a\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p9wk7" Jul 15 17:00:38.189900 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.189876 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0055caa8-1ee7-4b69-8dba-d1201107d12a-config\") pod \"service-ca-operator-7695fc6c8c-p9wk7\" (UID: \"0055caa8-1ee7-4b69-8dba-d1201107d12a\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p9wk7" Jul 15 17:00:38.191365 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.191347 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0055caa8-1ee7-4b69-8dba-d1201107d12a-serving-cert\") pod \"service-ca-operator-7695fc6c8c-p9wk7\" (UID: \"0055caa8-1ee7-4b69-8dba-d1201107d12a\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p9wk7" Jul 15 17:00:38.197042 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.197021 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v2zz\" (UniqueName: \"kubernetes.io/projected/0055caa8-1ee7-4b69-8dba-d1201107d12a-kube-api-access-8v2zz\") pod \"service-ca-operator-7695fc6c8c-p9wk7\" (UID: \"0055caa8-1ee7-4b69-8dba-d1201107d12a\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p9wk7" Jul 15 17:00:38.213196 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.213136 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:38.257089 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.257060 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-mmlx6" Jul 15 17:00:38.294098 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.294027 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-5d55j"] Jul 15 17:00:38.297717 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:00:38.297692 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a492e65_62e9_48e5_ae6d_f04e48964b37.slice/crio-a3ec3fcbdacb6eb2590a10ac188947a2c9705738523918e593ab94ee74ee8957 WatchSource:0}: Error finding container a3ec3fcbdacb6eb2590a10ac188947a2c9705738523918e593ab94ee74ee8957: Status 404 returned error can't find the container with id a3ec3fcbdacb6eb2590a10ac188947a2c9705738523918e593ab94ee74ee8957 Jul 15 17:00:38.311904 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.311855 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-hthbq"] Jul 15 17:00:38.316896 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:00:38.316854 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4bf7f61_aa9c_4a55_a300_a5ba3ffc48e5.slice/crio-f97c1972d2831dbcc6f653d0c1bdfef97ddc814051961fe1c36016f224245376 WatchSource:0}: Error finding container f97c1972d2831dbcc6f653d0c1bdfef97ddc814051961fe1c36016f224245376: Status 404 returned error can't find the container with id f97c1972d2831dbcc6f653d0c1bdfef97ddc814051961fe1c36016f224245376 Jul 15 17:00:38.350078 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.350014 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p9wk7" Jul 15 17:00:38.392215 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.392072 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-mmlx6"] Jul 15 17:00:38.394691 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:00:38.394666 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod165963cd_378d_47e0_9b04_e6b01478473e.slice/crio-d641cc4ebf239b393997faefddf406fd58255adf28959d19a5e27f0d21ae2250 WatchSource:0}: Error finding container d641cc4ebf239b393997faefddf406fd58255adf28959d19a5e27f0d21ae2250: Status 404 returned error can't find the container with id d641cc4ebf239b393997faefddf406fd58255adf28959d19a5e27f0d21ae2250 Jul 15 17:00:38.474803 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.474778 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p9wk7"] Jul 15 17:00:38.477723 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:00:38.477699 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0055caa8_1ee7_4b69_8dba_d1201107d12a.slice/crio-2acc51219fc18d940d2d222e89e40f2ae835241618d64048b5ca136ea4cf33d7 WatchSource:0}: Error finding container 2acc51219fc18d940d2d222e89e40f2ae835241618d64048b5ca136ea4cf33d7: Status 404 returned error can't find the container with id 2acc51219fc18d940d2d222e89e40f2ae835241618d64048b5ca136ea4cf33d7 Jul 15 17:00:38.491280 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.491255 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/68d72523-54cd-412a-b697-0f04e891d984-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-qtxm9\" (UID: \"68d72523-54cd-412a-b697-0f04e891d984\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qtxm9" Jul 15 17:00:38.491396 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.491350 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cfb96618-c779-4569-8fbb-01cae188529c-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-94wg5\" (UID: \"cfb96618-c779-4569-8fbb-01cae188529c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94wg5" Jul 15 17:00:38.491396 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.491369 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bbfcb238-e568-4881-9f87-6cbad5fe3674-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-fzc59\" (UID: \"bbfcb238-e568-4881-9f87-6cbad5fe3674\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-fzc59" Jul 15 17:00:38.491505 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:38.491411 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 17:00:38.491505 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:38.491450 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 17:00:38.491505 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:38.491474 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 17:00:38.491505 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:38.491483 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68d72523-54cd-412a-b697-0f04e891d984-cluster-monitoring-operator-tls podName:68d72523-54cd-412a-b697-0f04e891d984 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:39.491459503 +0000 UTC m=+257.044589711 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/68d72523-54cd-412a-b697-0f04e891d984-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-qtxm9" (UID: "68d72523-54cd-412a-b697-0f04e891d984") : secret "cluster-monitoring-operator-tls" not found Jul 15 17:00:38.491505 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:38.491503 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbfcb238-e568-4881-9f87-6cbad5fe3674-networking-console-plugin-cert podName:bbfcb238-e568-4881-9f87-6cbad5fe3674 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:39.491492272 +0000 UTC m=+257.044622470 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/bbfcb238-e568-4881-9f87-6cbad5fe3674-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-fzc59" (UID: "bbfcb238-e568-4881-9f87-6cbad5fe3674") : secret "networking-console-plugin-cert" not found Jul 15 17:00:38.491708 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:38.491518 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfb96618-c779-4569-8fbb-01cae188529c-samples-operator-tls podName:cfb96618-c779-4569-8fbb-01cae188529c nodeName:}" failed. No retries permitted until 2026-07-15 17:00:39.491510218 +0000 UTC m=+257.044640412 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cfb96618-c779-4569-8fbb-01cae188529c-samples-operator-tls") pod "cluster-samples-operator-65db858894-94wg5" (UID: "cfb96618-c779-4569-8fbb-01cae188529c") : secret "samples-operator-tls" not found Jul 15 17:00:38.716473 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.716432 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-5d55j" event={"ID":"7a492e65-62e9-48e5-ae6d-f04e48964b37","Type":"ContainerStarted","Data":"a3ec3fcbdacb6eb2590a10ac188947a2c9705738523918e593ab94ee74ee8957"} Jul 15 17:00:38.717525 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.717499 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-hthbq" event={"ID":"f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5","Type":"ContainerStarted","Data":"f97c1972d2831dbcc6f653d0c1bdfef97ddc814051961fe1c36016f224245376"} Jul 15 17:00:38.718929 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.718902 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-mmlx6" event={"ID":"165963cd-378d-47e0-9b04-e6b01478473e","Type":"ContainerStarted","Data":"3f52840bfdf9bfa9b6c56cf2c89cb1c40303dc2e70b59ff26eb33c1318d123d9"} Jul 15 17:00:38.719045 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.718952 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-mmlx6" event={"ID":"165963cd-378d-47e0-9b04-e6b01478473e","Type":"ContainerStarted","Data":"d641cc4ebf239b393997faefddf406fd58255adf28959d19a5e27f0d21ae2250"} Jul 15 17:00:38.719903 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.719881 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p9wk7" event={"ID":"0055caa8-1ee7-4b69-8dba-d1201107d12a","Type":"ContainerStarted","Data":"2acc51219fc18d940d2d222e89e40f2ae835241618d64048b5ca136ea4cf33d7"} Jul 15 17:00:38.734732 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:38.734689 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-mmlx6" podStartSLOduration=1.734676122 podStartE2EDuration="1.734676122s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:00:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:00:38.733559903 +0000 UTC m=+256.286690114" watchObservedRunningTime="2026-07-15 17:00:38.734676122 +0000 UTC m=+256.287806336" Jul 15 17:00:39.500747 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:39.500707 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cfb96618-c779-4569-8fbb-01cae188529c-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-94wg5\" (UID: \"cfb96618-c779-4569-8fbb-01cae188529c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94wg5" Jul 15 17:00:39.501189 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:39.500760 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bbfcb238-e568-4881-9f87-6cbad5fe3674-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-fzc59\" (UID: \"bbfcb238-e568-4881-9f87-6cbad5fe3674\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-fzc59" Jul 15 17:00:39.501189 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:39.500797 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/68d72523-54cd-412a-b697-0f04e891d984-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-qtxm9\" (UID: \"68d72523-54cd-412a-b697-0f04e891d984\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qtxm9" Jul 15 17:00:39.501189 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:39.500897 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 17:00:39.501189 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:39.500915 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 17:00:39.501189 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:39.500993 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68d72523-54cd-412a-b697-0f04e891d984-cluster-monitoring-operator-tls podName:68d72523-54cd-412a-b697-0f04e891d984 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:41.500971208 +0000 UTC m=+259.054101403 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/68d72523-54cd-412a-b697-0f04e891d984-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-qtxm9" (UID: "68d72523-54cd-412a-b697-0f04e891d984") : secret "cluster-monitoring-operator-tls" not found Jul 15 17:00:39.501189 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:39.501012 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 17:00:39.501189 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:39.501024 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfb96618-c779-4569-8fbb-01cae188529c-samples-operator-tls podName:cfb96618-c779-4569-8fbb-01cae188529c nodeName:}" failed. No retries permitted until 2026-07-15 17:00:41.501003257 +0000 UTC m=+259.054133462 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cfb96618-c779-4569-8fbb-01cae188529c-samples-operator-tls") pod "cluster-samples-operator-65db858894-94wg5" (UID: "cfb96618-c779-4569-8fbb-01cae188529c") : secret "samples-operator-tls" not found Jul 15 17:00:39.501189 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:39.501080 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbfcb238-e568-4881-9f87-6cbad5fe3674-networking-console-plugin-cert podName:bbfcb238-e568-4881-9f87-6cbad5fe3674 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:41.501064738 +0000 UTC m=+259.054194931 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/bbfcb238-e568-4881-9f87-6cbad5fe3674-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-fzc59" (UID: "bbfcb238-e568-4881-9f87-6cbad5fe3674") : secret "networking-console-plugin-cert" not found Jul 15 17:00:41.518092 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:41.518055 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cfb96618-c779-4569-8fbb-01cae188529c-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-94wg5\" (UID: \"cfb96618-c779-4569-8fbb-01cae188529c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94wg5" Jul 15 17:00:41.518092 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:41.518097 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bbfcb238-e568-4881-9f87-6cbad5fe3674-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-fzc59\" (UID: \"bbfcb238-e568-4881-9f87-6cbad5fe3674\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-fzc59" Jul 15 17:00:41.518640 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:41.518119 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/68d72523-54cd-412a-b697-0f04e891d984-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-qtxm9\" (UID: \"68d72523-54cd-412a-b697-0f04e891d984\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qtxm9" Jul 15 17:00:41.518640 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:41.518214 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 17:00:41.518640 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:41.518237 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 17:00:41.518640 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:41.518279 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfb96618-c779-4569-8fbb-01cae188529c-samples-operator-tls podName:cfb96618-c779-4569-8fbb-01cae188529c nodeName:}" failed. No retries permitted until 2026-07-15 17:00:45.518264571 +0000 UTC m=+263.071394772 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cfb96618-c779-4569-8fbb-01cae188529c-samples-operator-tls") pod "cluster-samples-operator-65db858894-94wg5" (UID: "cfb96618-c779-4569-8fbb-01cae188529c") : secret "samples-operator-tls" not found Jul 15 17:00:41.518640 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:41.518295 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 17:00:41.518640 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:41.518304 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbfcb238-e568-4881-9f87-6cbad5fe3674-networking-console-plugin-cert podName:bbfcb238-e568-4881-9f87-6cbad5fe3674 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:45.518284792 +0000 UTC m=+263.071415013 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/bbfcb238-e568-4881-9f87-6cbad5fe3674-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-fzc59" (UID: "bbfcb238-e568-4881-9f87-6cbad5fe3674") : secret "networking-console-plugin-cert" not found Jul 15 17:00:41.518640 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:41.518353 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68d72523-54cd-412a-b697-0f04e891d984-cluster-monitoring-operator-tls podName:68d72523-54cd-412a-b697-0f04e891d984 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:45.518342221 +0000 UTC m=+263.071472414 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/68d72523-54cd-412a-b697-0f04e891d984-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-qtxm9" (UID: "68d72523-54cd-412a-b697-0f04e891d984") : secret "cluster-monitoring-operator-tls" not found Jul 15 17:00:41.730806 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:41.730773 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-hthbq" event={"ID":"f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5","Type":"ContainerStarted","Data":"89945ec82f35b05fb9c08d69df0c4ee1b7449d63dbc5e791332e2513615f8d13"} Jul 15 17:00:41.732225 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:41.732196 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p9wk7" event={"ID":"0055caa8-1ee7-4b69-8dba-d1201107d12a","Type":"ContainerStarted","Data":"153f907b053e9d6c40d94dcd1c1b134c5a114ddd3c08e1b7241b04989e104c51"} Jul 15 17:00:41.733633 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:41.733613 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-5d55j" event={"ID":"7a492e65-62e9-48e5-ae6d-f04e48964b37","Type":"ContainerStarted","Data":"e32ddbe27b46e1970db4a8e49b139315b8aea0ee3d58469ec4d8f116051a0f02"} Jul 15 17:00:41.748847 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:41.748801 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-hthbq" podStartSLOduration=2.030776584 podStartE2EDuration="4.748787215s" totalImagesPullingTime="2.718010631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:00:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:00:41.747685774 +0000 UTC m=+259.300816010" watchObservedRunningTime="2026-07-15 17:00:41.748787215 +0000 UTC m=+259.301917430" Jul 15 17:00:41.762884 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:41.762844 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-5d55j" podStartSLOduration=2.025501658 podStartE2EDuration="4.762831357s" totalImagesPullingTime="2.737329699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:00:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:00:41.762060616 +0000 UTC m=+259.315190857" watchObservedRunningTime="2026-07-15 17:00:41.762831357 +0000 UTC m=+259.315961570" Jul 15 17:00:41.782194 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:41.782100 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p9wk7" podStartSLOduration=1.219221359 podStartE2EDuration="3.782087607s" totalImagesPullingTime="2.562866248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:00:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:00:41.78147375 +0000 UTC m=+259.334603965" watchObservedRunningTime="2026-07-15 17:00:41.782087607 +0000 UTC m=+259.335217860" Jul 15 17:00:41.856559 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:41.856527 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:42.358313 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:42.355389 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-plnkx"] Jul 15 17:00:42.359744 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:42.359725 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-plnkx" Jul 15 17:00:42.362345 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:42.362321 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 15 17:00:42.362566 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:42.362550 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 15 17:00:42.363366 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:42.363348 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-n6pnl\"" Jul 15 17:00:42.367168 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:42.367148 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-plnkx"] Jul 15 17:00:42.427120 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:42.427088 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lflv\" (UniqueName: \"kubernetes.io/projected/ab04f5d8-2173-4722-8495-32636d4d91f0-kube-api-access-7lflv\") pod \"migrator-7f59b54f6c-plnkx\" (UID: \"ab04f5d8-2173-4722-8495-32636d4d91f0\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-plnkx" Jul 15 17:00:42.527830 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:42.527799 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lflv\" (UniqueName: \"kubernetes.io/projected/ab04f5d8-2173-4722-8495-32636d4d91f0-kube-api-access-7lflv\") pod \"migrator-7f59b54f6c-plnkx\" (UID: \"ab04f5d8-2173-4722-8495-32636d4d91f0\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-plnkx" Jul 15 17:00:42.536203 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:42.536172 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lflv\" (UniqueName: \"kubernetes.io/projected/ab04f5d8-2173-4722-8495-32636d4d91f0-kube-api-access-7lflv\") pod \"migrator-7f59b54f6c-plnkx\" (UID: \"ab04f5d8-2173-4722-8495-32636d4d91f0\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-plnkx" Jul 15 17:00:42.669378 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:42.669353 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-plnkx" Jul 15 17:00:42.676306 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:42.676282 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:42.798068 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:42.798039 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-plnkx"] Jul 15 17:00:42.800752 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:00:42.800726 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab04f5d8_2173_4722_8495_32636d4d91f0.slice/crio-a112a74cbdcb84952b50b9cab51d38fa0776a189fc0592faa1718eb4d693d0cd WatchSource:0}: Error finding container a112a74cbdcb84952b50b9cab51d38fa0776a189fc0592faa1718eb4d693d0cd: Status 404 returned error can't find the container with id a112a74cbdcb84952b50b9cab51d38fa0776a189fc0592faa1718eb4d693d0cd Jul 15 17:00:42.811980 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:42.811945 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:42.936166 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:42.936076 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:42.996688 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:42.996655 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-vs662"] Jul 15 17:00:43.001651 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:43.001623 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vs662" Jul 15 17:00:43.004808 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:43.004786 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 15 17:00:43.004932 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:43.004785 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 15 17:00:43.004932 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:43.004821 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tmfrj\"" Jul 15 17:00:43.019024 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:43.019002 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vs662"] Jul 15 17:00:43.033309 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:43.033283 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlfd2\" (UniqueName: \"kubernetes.io/projected/eb9d6e60-0095-487e-9c95-b26cfed57365-kube-api-access-dlfd2\") pod \"insights-runtime-extractor-vs662\" (UID: \"eb9d6e60-0095-487e-9c95-b26cfed57365\") " pod="openshift-insights/insights-runtime-extractor-vs662" Jul 15 17:00:43.033451 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:43.033325 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eb9d6e60-0095-487e-9c95-b26cfed57365-crio-socket\") pod \"insights-runtime-extractor-vs662\" (UID: \"eb9d6e60-0095-487e-9c95-b26cfed57365\") " pod="openshift-insights/insights-runtime-extractor-vs662" Jul 15 17:00:43.033451 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:43.033383 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb9d6e60-0095-487e-9c95-b26cfed57365-data-volume\") pod \"insights-runtime-extractor-vs662\" (UID: \"eb9d6e60-0095-487e-9c95-b26cfed57365\") " pod="openshift-insights/insights-runtime-extractor-vs662" Jul 15 17:00:43.033556 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:43.033490 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb9d6e60-0095-487e-9c95-b26cfed57365-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vs662\" (UID: \"eb9d6e60-0095-487e-9c95-b26cfed57365\") " pod="openshift-insights/insights-runtime-extractor-vs662" Jul 15 17:00:43.033620 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:43.033550 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb9d6e60-0095-487e-9c95-b26cfed57365-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vs662\" (UID: \"eb9d6e60-0095-487e-9c95-b26cfed57365\") " pod="openshift-insights/insights-runtime-extractor-vs662" Jul 15 17:00:43.134039 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:43.134010 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eb9d6e60-0095-487e-9c95-b26cfed57365-crio-socket\") pod \"insights-runtime-extractor-vs662\" (UID: \"eb9d6e60-0095-487e-9c95-b26cfed57365\") " pod="openshift-insights/insights-runtime-extractor-vs662" Jul 15 17:00:43.134180 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:43.134055 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb9d6e60-0095-487e-9c95-b26cfed57365-data-volume\") pod \"insights-runtime-extractor-vs662\" (UID: \"eb9d6e60-0095-487e-9c95-b26cfed57365\") " pod="openshift-insights/insights-runtime-extractor-vs662" Jul 15 17:00:43.134180 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:43.134139 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eb9d6e60-0095-487e-9c95-b26cfed57365-crio-socket\") pod \"insights-runtime-extractor-vs662\" (UID: \"eb9d6e60-0095-487e-9c95-b26cfed57365\") " pod="openshift-insights/insights-runtime-extractor-vs662" Jul 15 17:00:43.134180 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:43.134157 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb9d6e60-0095-487e-9c95-b26cfed57365-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vs662\" (UID: \"eb9d6e60-0095-487e-9c95-b26cfed57365\") " pod="openshift-insights/insights-runtime-extractor-vs662" Jul 15 17:00:43.134309 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:43.134192 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb9d6e60-0095-487e-9c95-b26cfed57365-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vs662\" (UID: \"eb9d6e60-0095-487e-9c95-b26cfed57365\") " pod="openshift-insights/insights-runtime-extractor-vs662" Jul 15 17:00:43.134309 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:43.134232 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlfd2\" (UniqueName: \"kubernetes.io/projected/eb9d6e60-0095-487e-9c95-b26cfed57365-kube-api-access-dlfd2\") pod \"insights-runtime-extractor-vs662\" (UID: \"eb9d6e60-0095-487e-9c95-b26cfed57365\") " pod="openshift-insights/insights-runtime-extractor-vs662" Jul 15 17:00:43.134309 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:43.134248 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 17:00:43.134309 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:43.134302 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb9d6e60-0095-487e-9c95-b26cfed57365-insights-runtime-extractor-tls podName:eb9d6e60-0095-487e-9c95-b26cfed57365 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:43.634285437 +0000 UTC m=+261.187415636 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eb9d6e60-0095-487e-9c95-b26cfed57365-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vs662" (UID: "eb9d6e60-0095-487e-9c95-b26cfed57365") : secret "insights-runtime-extractor-tls" not found Jul 15 17:00:43.134309 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:43.134298 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 17:00:43.134489 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:43.134342 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eb9d6e60-0095-487e-9c95-b26cfed57365-kube-rbac-proxy-cm podName:eb9d6e60-0095-487e-9c95-b26cfed57365 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:43.634333094 +0000 UTC m=+261.187463306 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eb9d6e60-0095-487e-9c95-b26cfed57365-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vs662" (UID: "eb9d6e60-0095-487e-9c95-b26cfed57365") : configmap "kube-rbac-proxy" not found Jul 15 17:00:43.134489 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:43.134417 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb9d6e60-0095-487e-9c95-b26cfed57365-data-volume\") pod \"insights-runtime-extractor-vs662\" (UID: \"eb9d6e60-0095-487e-9c95-b26cfed57365\") " pod="openshift-insights/insights-runtime-extractor-vs662" Jul 15 17:00:43.138725 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:43.138702 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:43.145019 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:43.145003 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlfd2\" (UniqueName: \"kubernetes.io/projected/eb9d6e60-0095-487e-9c95-b26cfed57365-kube-api-access-dlfd2\") pod \"insights-runtime-extractor-vs662\" (UID: \"eb9d6e60-0095-487e-9c95-b26cfed57365\") " pod="openshift-insights/insights-runtime-extractor-vs662" Jul 15 17:00:43.639614 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:43.639565 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb9d6e60-0095-487e-9c95-b26cfed57365-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vs662\" (UID: \"eb9d6e60-0095-487e-9c95-b26cfed57365\") " pod="openshift-insights/insights-runtime-extractor-vs662" Jul 15 17:00:43.640058 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:43.639623 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb9d6e60-0095-487e-9c95-b26cfed57365-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vs662\" (UID: \"eb9d6e60-0095-487e-9c95-b26cfed57365\") " pod="openshift-insights/insights-runtime-extractor-vs662" Jul 15 17:00:43.640058 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:43.639716 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 17:00:43.640058 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:43.639777 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb9d6e60-0095-487e-9c95-b26cfed57365-insights-runtime-extractor-tls podName:eb9d6e60-0095-487e-9c95-b26cfed57365 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:44.639761481 +0000 UTC m=+262.192891676 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eb9d6e60-0095-487e-9c95-b26cfed57365-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vs662" (UID: "eb9d6e60-0095-487e-9c95-b26cfed57365") : secret "insights-runtime-extractor-tls" not found Jul 15 17:00:43.640058 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:43.639716 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 17:00:43.640058 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:43.639872 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eb9d6e60-0095-487e-9c95-b26cfed57365-kube-rbac-proxy-cm podName:eb9d6e60-0095-487e-9c95-b26cfed57365 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:44.639854898 +0000 UTC m=+262.192985092 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eb9d6e60-0095-487e-9c95-b26cfed57365-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vs662" (UID: "eb9d6e60-0095-487e-9c95-b26cfed57365") : configmap "kube-rbac-proxy" not found Jul 15 17:00:43.741627 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:43.741583 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-plnkx" event={"ID":"ab04f5d8-2173-4722-8495-32636d4d91f0","Type":"ContainerStarted","Data":"a112a74cbdcb84952b50b9cab51d38fa0776a189fc0592faa1718eb4d693d0cd"} Jul 15 17:00:44.649341 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:44.649258 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb9d6e60-0095-487e-9c95-b26cfed57365-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vs662\" (UID: \"eb9d6e60-0095-487e-9c95-b26cfed57365\") " pod="openshift-insights/insights-runtime-extractor-vs662" Jul 15 17:00:44.649341 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:44.649298 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb9d6e60-0095-487e-9c95-b26cfed57365-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vs662\" (UID: \"eb9d6e60-0095-487e-9c95-b26cfed57365\") " pod="openshift-insights/insights-runtime-extractor-vs662" Jul 15 17:00:44.649775 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:44.649378 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 17:00:44.649775 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:44.649404 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 17:00:44.649775 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:44.649446 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eb9d6e60-0095-487e-9c95-b26cfed57365-kube-rbac-proxy-cm podName:eb9d6e60-0095-487e-9c95-b26cfed57365 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:46.649428976 +0000 UTC m=+264.202559172 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eb9d6e60-0095-487e-9c95-b26cfed57365-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vs662" (UID: "eb9d6e60-0095-487e-9c95-b26cfed57365") : configmap "kube-rbac-proxy" not found Jul 15 17:00:44.649775 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:44.649462 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb9d6e60-0095-487e-9c95-b26cfed57365-insights-runtime-extractor-tls podName:eb9d6e60-0095-487e-9c95-b26cfed57365 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:46.649455548 +0000 UTC m=+264.202585739 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eb9d6e60-0095-487e-9c95-b26cfed57365-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vs662" (UID: "eb9d6e60-0095-487e-9c95-b26cfed57365") : secret "insights-runtime-extractor-tls" not found Jul 15 17:00:44.693763 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:44.693731 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:44.805517 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:44.805482 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:45.556814 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:45.556782 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:45.557799 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:45.557776 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cfb96618-c779-4569-8fbb-01cae188529c-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-94wg5\" (UID: \"cfb96618-c779-4569-8fbb-01cae188529c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94wg5" Jul 15 17:00:45.557911 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:45.557815 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bbfcb238-e568-4881-9f87-6cbad5fe3674-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-fzc59\" (UID: \"bbfcb238-e568-4881-9f87-6cbad5fe3674\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-fzc59" Jul 15 17:00:45.557911 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:45.557849 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/68d72523-54cd-412a-b697-0f04e891d984-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-qtxm9\" (UID: \"68d72523-54cd-412a-b697-0f04e891d984\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qtxm9" Jul 15 17:00:45.557987 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:45.557910 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 17:00:45.557987 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:45.557957 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfb96618-c779-4569-8fbb-01cae188529c-samples-operator-tls podName:cfb96618-c779-4569-8fbb-01cae188529c nodeName:}" failed. No retries permitted until 2026-07-15 17:00:53.557943577 +0000 UTC m=+271.111073768 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cfb96618-c779-4569-8fbb-01cae188529c-samples-operator-tls") pod "cluster-samples-operator-65db858894-94wg5" (UID: "cfb96618-c779-4569-8fbb-01cae188529c") : secret "samples-operator-tls" not found Jul 15 17:00:45.558073 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:45.558005 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 17:00:45.558073 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:45.558058 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbfcb238-e568-4881-9f87-6cbad5fe3674-networking-console-plugin-cert podName:bbfcb238-e568-4881-9f87-6cbad5fe3674 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:53.558041359 +0000 UTC m=+271.111171552 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/bbfcb238-e568-4881-9f87-6cbad5fe3674-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-fzc59" (UID: "bbfcb238-e568-4881-9f87-6cbad5fe3674") : secret "networking-console-plugin-cert" not found Jul 15 17:00:45.558152 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:45.558077 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 17:00:45.558152 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:45.558110 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68d72523-54cd-412a-b697-0f04e891d984-cluster-monitoring-operator-tls podName:68d72523-54cd-412a-b697-0f04e891d984 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:53.558101946 +0000 UTC m=+271.111232142 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/68d72523-54cd-412a-b697-0f04e891d984-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-qtxm9" (UID: "68d72523-54cd-412a-b697-0f04e891d984") : secret "cluster-monitoring-operator-tls" not found Jul 15 17:00:45.573207 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:45.573185 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cvlbn_c9c2ba93-cd7a-4aa9-81f4-b5afeee112db/dns-node-resolver/0.log" Jul 15 17:00:45.750518 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:45.750481 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-plnkx" event={"ID":"ab04f5d8-2173-4722-8495-32636d4d91f0","Type":"ContainerStarted","Data":"e109f0d8863f863691bee8453efe92dc929f14ba4fd8f3c4065f750a7a3e5a08"} Jul 15 17:00:45.751020 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:45.750543 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-plnkx" event={"ID":"ab04f5d8-2173-4722-8495-32636d4d91f0","Type":"ContainerStarted","Data":"34ee605168f03c9ed4dcd701f255465fe559047cf670c8a68e0c7e5a6b591c55"} Jul 15 17:00:45.768199 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:45.768153 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-plnkx" podStartSLOduration=1.531766951 podStartE2EDuration="3.768138176s" totalImagesPullingTime="2.236371225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:00:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:00:45.76724916 +0000 UTC m=+263.320379374" watchObservedRunningTime="2026-07-15 17:00:45.768138176 +0000 UTC m=+263.321268601" Jul 15 17:00:46.667958 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:46.667922 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb9d6e60-0095-487e-9c95-b26cfed57365-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vs662\" (UID: \"eb9d6e60-0095-487e-9c95-b26cfed57365\") " pod="openshift-insights/insights-runtime-extractor-vs662" Jul 15 17:00:46.667958 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:46.667960 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb9d6e60-0095-487e-9c95-b26cfed57365-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vs662\" (UID: \"eb9d6e60-0095-487e-9c95-b26cfed57365\") " pod="openshift-insights/insights-runtime-extractor-vs662" Jul 15 17:00:46.668206 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:46.668038 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 17:00:46.668206 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:46.668083 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 17:00:46.668206 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:46.668090 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eb9d6e60-0095-487e-9c95-b26cfed57365-kube-rbac-proxy-cm podName:eb9d6e60-0095-487e-9c95-b26cfed57365 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:50.668077621 +0000 UTC m=+268.221207813 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eb9d6e60-0095-487e-9c95-b26cfed57365-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vs662" (UID: "eb9d6e60-0095-487e-9c95-b26cfed57365") : configmap "kube-rbac-proxy" not found Jul 15 17:00:46.668206 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:46.668152 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb9d6e60-0095-487e-9c95-b26cfed57365-insights-runtime-extractor-tls podName:eb9d6e60-0095-487e-9c95-b26cfed57365 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:50.668134095 +0000 UTC m=+268.221264287 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eb9d6e60-0095-487e-9c95-b26cfed57365-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vs662" (UID: "eb9d6e60-0095-487e-9c95-b26cfed57365") : secret "insights-runtime-extractor-tls" not found Jul 15 17:00:46.974541 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:46.974465 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-whrn6_d0867616-074a-4b55-94ce-df0e85196692/node-ca/0.log" Jul 15 17:00:47.772843 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:47.772818 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qsxzw_f7f08403-7f03-4072-993c-9b02881fd48f/serve-healthcheck-canary/0.log" Jul 15 17:00:47.972469 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:47.972441 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-plnkx_ab04f5d8-2173-4722-8495-32636d4d91f0/migrator/0.log" Jul 15 17:00:48.172417 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:48.172388 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-plnkx_ab04f5d8-2173-4722-8495-32636d4d91f0/graceful-termination/0.log" Jul 15 17:00:49.183984 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:49.183952 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:49.283512 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:49.283485 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:49.412763 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:49.412733 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:49.500799 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:49.500703 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:49.573109 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:49.573081 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:49.647480 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:49.647451 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:49.869117 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:49.869013 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:50.701046 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:50.701005 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb9d6e60-0095-487e-9c95-b26cfed57365-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vs662\" (UID: \"eb9d6e60-0095-487e-9c95-b26cfed57365\") " pod="openshift-insights/insights-runtime-extractor-vs662" Jul 15 17:00:50.701046 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:50.701047 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb9d6e60-0095-487e-9c95-b26cfed57365-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vs662\" (UID: \"eb9d6e60-0095-487e-9c95-b26cfed57365\") " pod="openshift-insights/insights-runtime-extractor-vs662" Jul 15 17:00:50.701507 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:50.701130 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 17:00:50.701507 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:50.701192 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eb9d6e60-0095-487e-9c95-b26cfed57365-kube-rbac-proxy-cm podName:eb9d6e60-0095-487e-9c95-b26cfed57365 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:58.701174775 +0000 UTC m=+276.254304967 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eb9d6e60-0095-487e-9c95-b26cfed57365-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vs662" (UID: "eb9d6e60-0095-487e-9c95-b26cfed57365") : configmap "kube-rbac-proxy" not found Jul 15 17:00:50.701507 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:50.701187 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 17:00:50.701507 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:50.701256 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb9d6e60-0095-487e-9c95-b26cfed57365-insights-runtime-extractor-tls podName:eb9d6e60-0095-487e-9c95-b26cfed57365 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:58.701239003 +0000 UTC m=+276.254369212 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eb9d6e60-0095-487e-9c95-b26cfed57365-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vs662" (UID: "eb9d6e60-0095-487e-9c95-b26cfed57365") : secret "insights-runtime-extractor-tls" not found Jul 15 17:00:51.193124 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:51.193084 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:51.295150 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:51.295119 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:51.394722 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:51.394667 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:51.495715 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:51.495629 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:51.674165 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:51.674132 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:51.875786 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:51.875711 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:52.873851 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:52.873816 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:53.068725 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:53.068693 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:53.625848 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:53.625812 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cfb96618-c779-4569-8fbb-01cae188529c-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-94wg5\" (UID: \"cfb96618-c779-4569-8fbb-01cae188529c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94wg5" Jul 15 17:00:53.625848 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:53.625850 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bbfcb238-e568-4881-9f87-6cbad5fe3674-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-fzc59\" (UID: \"bbfcb238-e568-4881-9f87-6cbad5fe3674\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-fzc59" Jul 15 17:00:53.626073 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:53.625881 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/68d72523-54cd-412a-b697-0f04e891d984-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-qtxm9\" (UID: \"68d72523-54cd-412a-b697-0f04e891d984\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qtxm9" Jul 15 17:00:53.626073 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:53.625969 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 17:00:53.626073 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:53.626035 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 17:00:53.626073 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:53.626049 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbfcb238-e568-4881-9f87-6cbad5fe3674-networking-console-plugin-cert podName:bbfcb238-e568-4881-9f87-6cbad5fe3674 nodeName:}" failed. No retries permitted until 2026-07-15 17:01:09.626028412 +0000 UTC m=+287.179158621 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/bbfcb238-e568-4881-9f87-6cbad5fe3674-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-fzc59" (UID: "bbfcb238-e568-4881-9f87-6cbad5fe3674") : secret "networking-console-plugin-cert" not found Jul 15 17:00:53.626073 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:53.626075 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68d72523-54cd-412a-b697-0f04e891d984-cluster-monitoring-operator-tls podName:68d72523-54cd-412a-b697-0f04e891d984 nodeName:}" failed. No retries permitted until 2026-07-15 17:01:09.62606323 +0000 UTC m=+287.179193426 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/68d72523-54cd-412a-b697-0f04e891d984-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-qtxm9" (UID: "68d72523-54cd-412a-b697-0f04e891d984") : secret "cluster-monitoring-operator-tls" not found Jul 15 17:00:53.628231 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:53.628209 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cfb96618-c779-4569-8fbb-01cae188529c-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-94wg5\" (UID: \"cfb96618-c779-4569-8fbb-01cae188529c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94wg5" Jul 15 17:00:53.786043 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:53.786015 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94wg5" Jul 15 17:00:53.909260 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:53.909236 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94wg5"] Jul 15 17:00:54.212138 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:54.212102 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:54.312448 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:54.312415 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:54.681641 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:54.681611 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:54.778381 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:54.778342 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94wg5" event={"ID":"cfb96618-c779-4569-8fbb-01cae188529c","Type":"ContainerStarted","Data":"9b1a9ad22ef281814e84979eaf64c7da1922fe9edd0bd4fd30a85f87681a0667"} Jul 15 17:00:55.275413 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:55.275373 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:55.475383 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:55.475353 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:55.540896 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:55.540824 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:55.742928 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:55.742898 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:55.870789 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:55.870721 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:56.069047 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:56.069017 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:56.164759 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:56.164690 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:56.675035 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:56.675001 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:56.786961 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:56.786930 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94wg5" event={"ID":"cfb96618-c779-4569-8fbb-01cae188529c","Type":"ContainerStarted","Data":"aeffb85b80d06fef5107e1b069a8149745d2012c73cfa130cd50b984f43caac2"} Jul 15 17:00:56.787111 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:56.787015 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94wg5" event={"ID":"cfb96618-c779-4569-8fbb-01cae188529c","Type":"ContainerStarted","Data":"ce4b9c0c866b7819537ff3fe59c0bd68e8d26cead16e4336498b49542bc9a101"} Jul 15 17:00:56.804021 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:56.803972 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94wg5" podStartSLOduration=17.869639364 podStartE2EDuration="19.803959425s" totalImagesPullingTime="1.934320061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:00:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:00:56.802949459 +0000 UTC m=+274.356079673" watchObservedRunningTime="2026-07-15 17:00:56.803959425 +0000 UTC m=+274.357089680" Jul 15 17:00:57.316554 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:57.316523 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:57.462831 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:57.462802 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:57.678770 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:57.678741 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:57.996095 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:57.996018 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:58.095744 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:58.095712 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:58.480148 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:58.480118 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:58.770537 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:58.770446 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb9d6e60-0095-487e-9c95-b26cfed57365-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vs662\" (UID: \"eb9d6e60-0095-487e-9c95-b26cfed57365\") " pod="openshift-insights/insights-runtime-extractor-vs662" Jul 15 17:00:58.770911 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:58.770620 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb9d6e60-0095-487e-9c95-b26cfed57365-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vs662\" (UID: \"eb9d6e60-0095-487e-9c95-b26cfed57365\") " pod="openshift-insights/insights-runtime-extractor-vs662" Jul 15 17:00:58.770911 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:58.770733 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 17:00:58.770911 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:00:58.770805 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb9d6e60-0095-487e-9c95-b26cfed57365-insights-runtime-extractor-tls podName:eb9d6e60-0095-487e-9c95-b26cfed57365 nodeName:}" failed. No retries permitted until 2026-07-15 17:01:14.770785527 +0000 UTC m=+292.323915723 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eb9d6e60-0095-487e-9c95-b26cfed57365-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vs662" (UID: "eb9d6e60-0095-487e-9c95-b26cfed57365") : secret "insights-runtime-extractor-tls" not found Jul 15 17:00:58.771110 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:58.771090 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb9d6e60-0095-487e-9c95-b26cfed57365-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vs662\" (UID: \"eb9d6e60-0095-487e-9c95-b26cfed57365\") " pod="openshift-insights/insights-runtime-extractor-vs662" Jul 15 17:00:59.067468 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:59.067389 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:59.471959 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:59.471922 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:59.666651 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:59.666617 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:59.867652 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:00:59.867567 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:02.432892 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:02.432859 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:02.496232 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:02.496188 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:02.595818 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:02.595774 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:02.760054 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:02.759966 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:03.141378 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:03.141288 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:03.489559 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:01:03.489522 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" podUID="f0d3cec0-475c-481e-beb3-012b1c8c215b" Jul 15 17:01:03.489559 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:01:03.489522 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-jvr75" podUID="191e57d2-b61f-443a-a7ca-a8a7ad699155" Jul 15 17:01:03.524300 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:03.524268 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:03.597079 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:03.597055 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:03.741304 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:03.741207 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:03.807113 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:03.807091 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jvr75" Jul 15 17:01:03.807241 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:03.807092 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 17:01:03.841651 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:03.841624 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:03.941516 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:03.941487 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:04.129786 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:04.129712 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:04.684131 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:04.684097 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:05.551586 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:05.551549 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:05.651661 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:05.651629 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:07.345525 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:07.345494 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-tls\") pod \"image-registry-795b87c6c7-xztkp\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 17:01:07.345920 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:07.345555 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/191e57d2-b61f-443a-a7ca-a8a7ad699155-metrics-tls\") pod \"dns-default-jvr75\" (UID: \"191e57d2-b61f-443a-a7ca-a8a7ad699155\") " pod="openshift-dns/dns-default-jvr75" Jul 15 17:01:07.347759 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:07.347736 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/191e57d2-b61f-443a-a7ca-a8a7ad699155-metrics-tls\") pod \"dns-default-jvr75\" (UID: \"191e57d2-b61f-443a-a7ca-a8a7ad699155\") " pod="openshift-dns/dns-default-jvr75" Jul 15 17:01:07.347914 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:07.347895 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-tls\") pod \"image-registry-795b87c6c7-xztkp\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 17:01:07.410192 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:07.410164 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lllhc\"" Jul 15 17:01:07.410870 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:07.410847 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hhz8f\"" Jul 15 17:01:07.417838 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:07.417825 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jvr75" Jul 15 17:01:07.417937 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:07.417920 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 17:01:07.558144 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:07.558119 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-795b87c6c7-xztkp"] Jul 15 17:01:07.560069 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:01:07.560032 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0d3cec0_475c_481e_beb3_012b1c8c215b.slice/crio-2ed241ffe59f683e1f3305d59fd38701eca73ee351637ac7e1b72ba95b3f7abe WatchSource:0}: Error finding container 2ed241ffe59f683e1f3305d59fd38701eca73ee351637ac7e1b72ba95b3f7abe: Status 404 returned error can't find the container with id 2ed241ffe59f683e1f3305d59fd38701eca73ee351637ac7e1b72ba95b3f7abe Jul 15 17:01:07.566050 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:07.565993 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jvr75"] Jul 15 17:01:07.570313 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:01:07.570294 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod191e57d2_b61f_443a_a7ca_a8a7ad699155.slice/crio-cff69f41acfba7235bdad9ffd54afdf8516b9ade31968c0ac73ae0c7bc788ed5 WatchSource:0}: Error finding container cff69f41acfba7235bdad9ffd54afdf8516b9ade31968c0ac73ae0c7bc788ed5: Status 404 returned error can't find the container with id cff69f41acfba7235bdad9ffd54afdf8516b9ade31968c0ac73ae0c7bc788ed5 Jul 15 17:01:07.821790 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:07.821757 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" event={"ID":"f0d3cec0-475c-481e-beb3-012b1c8c215b","Type":"ContainerStarted","Data":"a390673b12ff075fe4db12cd2dc7e14ebc9901e63b90e558659a3c7b1f03481f"} Jul 15 17:01:07.821961 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:07.821810 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" event={"ID":"f0d3cec0-475c-481e-beb3-012b1c8c215b","Type":"ContainerStarted","Data":"2ed241ffe59f683e1f3305d59fd38701eca73ee351637ac7e1b72ba95b3f7abe"} Jul 15 17:01:07.821961 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:07.821847 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 17:01:07.822959 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:07.822936 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jvr75" event={"ID":"191e57d2-b61f-443a-a7ca-a8a7ad699155","Type":"ContainerStarted","Data":"cff69f41acfba7235bdad9ffd54afdf8516b9ade31968c0ac73ae0c7bc788ed5"} Jul 15 17:01:07.837909 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:07.837878 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-fvqjs"] Jul 15 17:01:07.840062 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:07.840043 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-fvqjs" Jul 15 17:01:07.842702 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:07.842682 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-tpmwr\"" Jul 15 17:01:07.842801 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:07.842687 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 15 17:01:07.842861 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:07.842841 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 15 17:01:07.847865 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:07.847825 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" podStartSLOduration=284.847811491 podStartE2EDuration="4m44.847811491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:56:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:01:07.847107795 +0000 UTC m=+285.400238010" watchObservedRunningTime="2026-07-15 17:01:07.847811491 +0000 UTC m=+285.400941704" Jul 15 17:01:07.856730 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:07.856664 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-fvqjs"] Jul 15 17:01:07.923224 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:07.923194 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:07.951550 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:07.951523 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x95c\" (UniqueName: \"kubernetes.io/projected/9446a817-b692-4190-89c1-a8ae61d15bb7-kube-api-access-7x95c\") pod \"downloads-68c9cbb497-fvqjs\" (UID: \"9446a817-b692-4190-89c1-a8ae61d15bb7\") " pod="openshift-console/downloads-68c9cbb497-fvqjs" Jul 15 17:01:08.015217 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:08.015172 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:08.052166 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:08.052123 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x95c\" (UniqueName: \"kubernetes.io/projected/9446a817-b692-4190-89c1-a8ae61d15bb7-kube-api-access-7x95c\") pod \"downloads-68c9cbb497-fvqjs\" (UID: \"9446a817-b692-4190-89c1-a8ae61d15bb7\") " pod="openshift-console/downloads-68c9cbb497-fvqjs" Jul 15 17:01:08.061405 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:08.061369 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x95c\" (UniqueName: \"kubernetes.io/projected/9446a817-b692-4190-89c1-a8ae61d15bb7-kube-api-access-7x95c\") pod \"downloads-68c9cbb497-fvqjs\" (UID: \"9446a817-b692-4190-89c1-a8ae61d15bb7\") " pod="openshift-console/downloads-68c9cbb497-fvqjs" Jul 15 17:01:08.149522 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:08.149436 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-fvqjs" Jul 15 17:01:08.304911 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:08.304880 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-fvqjs"] Jul 15 17:01:08.308029 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:01:08.307998 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9446a817_b692_4190_89c1_a8ae61d15bb7.slice/crio-3f578a360d1a60d9a51c15eac8eda45d7fd97792128c23ebec81b39ffbd6e166 WatchSource:0}: Error finding container 3f578a360d1a60d9a51c15eac8eda45d7fd97792128c23ebec81b39ffbd6e166: Status 404 returned error can't find the container with id 3f578a360d1a60d9a51c15eac8eda45d7fd97792128c23ebec81b39ffbd6e166 Jul 15 17:01:08.826724 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:08.826690 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-fvqjs" event={"ID":"9446a817-b692-4190-89c1-a8ae61d15bb7","Type":"ContainerStarted","Data":"3f578a360d1a60d9a51c15eac8eda45d7fd97792128c23ebec81b39ffbd6e166"} Jul 15 17:01:09.666833 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:09.666797 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bbfcb238-e568-4881-9f87-6cbad5fe3674-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-fzc59\" (UID: \"bbfcb238-e568-4881-9f87-6cbad5fe3674\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-fzc59" Jul 15 17:01:09.667012 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:09.666847 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/68d72523-54cd-412a-b697-0f04e891d984-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-qtxm9\" (UID: \"68d72523-54cd-412a-b697-0f04e891d984\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qtxm9" Jul 15 17:01:09.669692 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:09.669667 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bbfcb238-e568-4881-9f87-6cbad5fe3674-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-fzc59\" (UID: \"bbfcb238-e568-4881-9f87-6cbad5fe3674\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-fzc59" Jul 15 17:01:09.669814 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:09.669742 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/68d72523-54cd-412a-b697-0f04e891d984-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-qtxm9\" (UID: \"68d72523-54cd-412a-b697-0f04e891d984\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qtxm9" Jul 15 17:01:09.673464 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:09.673446 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-fzc59" Jul 15 17:01:09.809014 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:09.808980 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-fzc59"] Jul 15 17:01:09.812121 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:01:09.812091 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbfcb238_e568_4881_9f87_6cbad5fe3674.slice/crio-194d5daecb03c26ebef4afd0540efe16ce8384c40a9afe0118f3c3741032e845 WatchSource:0}: Error finding container 194d5daecb03c26ebef4afd0540efe16ce8384c40a9afe0118f3c3741032e845: Status 404 returned error can't find the container with id 194d5daecb03c26ebef4afd0540efe16ce8384c40a9afe0118f3c3741032e845 Jul 15 17:01:09.831208 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:09.831178 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-fzc59" event={"ID":"bbfcb238-e568-4881-9f87-6cbad5fe3674","Type":"ContainerStarted","Data":"194d5daecb03c26ebef4afd0540efe16ce8384c40a9afe0118f3c3741032e845"} Jul 15 17:01:09.834010 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:09.833968 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jvr75" event={"ID":"191e57d2-b61f-443a-a7ca-a8a7ad699155","Type":"ContainerStarted","Data":"ac77aae20a218cf16b986b2defdbc15a88a90c47fba9d6cf81a58fb5eab6614b"} Jul 15 17:01:09.834117 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:09.834022 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jvr75" event={"ID":"191e57d2-b61f-443a-a7ca-a8a7ad699155","Type":"ContainerStarted","Data":"2b38c9f1c55f2fe758a7dba0f188bb9e9416fbeda6d36a10426abad8050fabe4"} Jul 15 17:01:09.834744 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:09.834726 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jvr75" Jul 15 17:01:09.853588 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:09.853506 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-jvr75" podStartSLOduration=251.014542747 podStartE2EDuration="4m12.853490741s" totalImagesPullingTime="1.838947994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:56:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:01:09.852027189 +0000 UTC m=+287.405157403" watchObservedRunningTime="2026-07-15 17:01:09.853490741 +0000 UTC m=+287.406620957" Jul 15 17:01:09.957769 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:09.957695 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qtxm9" Jul 15 17:01:10.097835 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:10.097809 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qtxm9"] Jul 15 17:01:10.100396 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:01:10.100370 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68d72523_54cd_412a_b697_0f04e891d984.slice/crio-e24cb296668ac265d40dec041ddf12b25ca538f1ab1b39ef805f2d26ad4f396d WatchSource:0}: Error finding container e24cb296668ac265d40dec041ddf12b25ca538f1ab1b39ef805f2d26ad4f396d: Status 404 returned error can't find the container with id e24cb296668ac265d40dec041ddf12b25ca538f1ab1b39ef805f2d26ad4f396d Jul 15 17:01:10.840411 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:10.840375 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qtxm9" event={"ID":"68d72523-54cd-412a-b697-0f04e891d984","Type":"ContainerStarted","Data":"e24cb296668ac265d40dec041ddf12b25ca538f1ab1b39ef805f2d26ad4f396d"} Jul 15 17:01:12.848537 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:12.848454 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-fzc59" event={"ID":"bbfcb238-e568-4881-9f87-6cbad5fe3674","Type":"ContainerStarted","Data":"9cdead9613788137595cd31de2f43a877673616b6788f9247bb58c2d41701b4a"} Jul 15 17:01:12.862736 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:12.862689 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-fzc59" podStartSLOduration=33.130827726 podStartE2EDuration="35.86267668s" totalImagesPullingTime="2.731848954s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:00:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:01:12.862625069 +0000 UTC m=+290.415755301" watchObservedRunningTime="2026-07-15 17:01:12.86267668 +0000 UTC m=+290.415806894" Jul 15 17:01:12.962071 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:12.962035 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:12.972199 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:12.972169 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b8564d945-6nq2w"] Jul 15 17:01:12.976507 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:12.976485 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b8564d945-6nq2w" Jul 15 17:01:12.979015 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:12.978980 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 15 17:01:12.979015 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:12.978995 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 15 17:01:12.979015 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:12.978985 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 15 17:01:12.979411 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:12.979391 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 15 17:01:12.979515 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:12.979430 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-d5smq\"" Jul 15 17:01:12.979515 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:12.979491 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 15 17:01:12.984348 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:12.984306 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b8564d945-6nq2w"] Jul 15 17:01:13.095960 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:13.095926 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a612a93-a727-4552-b21d-da7c2318980e-service-ca\") pod \"console-5b8564d945-6nq2w\" (UID: \"6a612a93-a727-4552-b21d-da7c2318980e\") " pod="openshift-console/console-5b8564d945-6nq2w" Jul 15 17:01:13.096146 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:13.095971 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a612a93-a727-4552-b21d-da7c2318980e-console-config\") pod \"console-5b8564d945-6nq2w\" (UID: \"6a612a93-a727-4552-b21d-da7c2318980e\") " pod="openshift-console/console-5b8564d945-6nq2w" Jul 15 17:01:13.096146 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:13.096024 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a612a93-a727-4552-b21d-da7c2318980e-oauth-serving-cert\") pod \"console-5b8564d945-6nq2w\" (UID: \"6a612a93-a727-4552-b21d-da7c2318980e\") " pod="openshift-console/console-5b8564d945-6nq2w" Jul 15 17:01:13.096146 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:13.096067 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a612a93-a727-4552-b21d-da7c2318980e-console-serving-cert\") pod \"console-5b8564d945-6nq2w\" (UID: \"6a612a93-a727-4552-b21d-da7c2318980e\") " pod="openshift-console/console-5b8564d945-6nq2w" Jul 15 17:01:13.096146 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:13.096097 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6xr7\" (UniqueName: \"kubernetes.io/projected/6a612a93-a727-4552-b21d-da7c2318980e-kube-api-access-b6xr7\") pod \"console-5b8564d945-6nq2w\" (UID: \"6a612a93-a727-4552-b21d-da7c2318980e\") " pod="openshift-console/console-5b8564d945-6nq2w" Jul 15 17:01:13.096146 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:13.096122 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a612a93-a727-4552-b21d-da7c2318980e-console-oauth-config\") pod \"console-5b8564d945-6nq2w\" (UID: \"6a612a93-a727-4552-b21d-da7c2318980e\") " pod="openshift-console/console-5b8564d945-6nq2w" Jul 15 17:01:13.197400 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:13.197368 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6xr7\" (UniqueName: \"kubernetes.io/projected/6a612a93-a727-4552-b21d-da7c2318980e-kube-api-access-b6xr7\") pod \"console-5b8564d945-6nq2w\" (UID: \"6a612a93-a727-4552-b21d-da7c2318980e\") " pod="openshift-console/console-5b8564d945-6nq2w" Jul 15 17:01:13.197629 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:13.197415 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a612a93-a727-4552-b21d-da7c2318980e-console-oauth-config\") pod \"console-5b8564d945-6nq2w\" (UID: \"6a612a93-a727-4552-b21d-da7c2318980e\") " pod="openshift-console/console-5b8564d945-6nq2w" Jul 15 17:01:13.197629 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:13.197505 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a612a93-a727-4552-b21d-da7c2318980e-service-ca\") pod \"console-5b8564d945-6nq2w\" (UID: \"6a612a93-a727-4552-b21d-da7c2318980e\") " pod="openshift-console/console-5b8564d945-6nq2w" Jul 15 17:01:13.197629 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:13.197538 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a612a93-a727-4552-b21d-da7c2318980e-console-config\") pod \"console-5b8564d945-6nq2w\" (UID: \"6a612a93-a727-4552-b21d-da7c2318980e\") " pod="openshift-console/console-5b8564d945-6nq2w" Jul 15 17:01:13.197629 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:13.197559 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a612a93-a727-4552-b21d-da7c2318980e-oauth-serving-cert\") pod \"console-5b8564d945-6nq2w\" (UID: \"6a612a93-a727-4552-b21d-da7c2318980e\") " pod="openshift-console/console-5b8564d945-6nq2w" Jul 15 17:01:13.197629 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:13.197609 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a612a93-a727-4552-b21d-da7c2318980e-console-serving-cert\") pod \"console-5b8564d945-6nq2w\" (UID: \"6a612a93-a727-4552-b21d-da7c2318980e\") " pod="openshift-console/console-5b8564d945-6nq2w" Jul 15 17:01:13.198560 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:13.198531 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a612a93-a727-4552-b21d-da7c2318980e-console-config\") pod \"console-5b8564d945-6nq2w\" (UID: \"6a612a93-a727-4552-b21d-da7c2318980e\") " pod="openshift-console/console-5b8564d945-6nq2w" Jul 15 17:01:13.198708 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:13.198627 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a612a93-a727-4552-b21d-da7c2318980e-service-ca\") pod \"console-5b8564d945-6nq2w\" (UID: \"6a612a93-a727-4552-b21d-da7c2318980e\") " pod="openshift-console/console-5b8564d945-6nq2w" Jul 15 17:01:13.198890 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:13.198863 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a612a93-a727-4552-b21d-da7c2318980e-oauth-serving-cert\") pod \"console-5b8564d945-6nq2w\" (UID: \"6a612a93-a727-4552-b21d-da7c2318980e\") " pod="openshift-console/console-5b8564d945-6nq2w" Jul 15 17:01:13.200703 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:13.200679 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a612a93-a727-4552-b21d-da7c2318980e-console-serving-cert\") pod \"console-5b8564d945-6nq2w\" (UID: \"6a612a93-a727-4552-b21d-da7c2318980e\") " pod="openshift-console/console-5b8564d945-6nq2w" Jul 15 17:01:13.201055 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:13.201036 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a612a93-a727-4552-b21d-da7c2318980e-console-oauth-config\") pod \"console-5b8564d945-6nq2w\" (UID: \"6a612a93-a727-4552-b21d-da7c2318980e\") " pod="openshift-console/console-5b8564d945-6nq2w" Jul 15 17:01:13.205988 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:13.205966 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6xr7\" (UniqueName: \"kubernetes.io/projected/6a612a93-a727-4552-b21d-da7c2318980e-kube-api-access-b6xr7\") pod \"console-5b8564d945-6nq2w\" (UID: \"6a612a93-a727-4552-b21d-da7c2318980e\") " pod="openshift-console/console-5b8564d945-6nq2w" Jul 15 17:01:13.287997 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:13.287964 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b8564d945-6nq2w" Jul 15 17:01:13.441348 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:13.441281 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b8564d945-6nq2w"] Jul 15 17:01:13.443794 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:01:13.443752 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a612a93_a727_4552_b21d_da7c2318980e.slice/crio-928d24935e162b6a40a219fb42e30789bc881a0d96185f603181944968378e47 WatchSource:0}: Error finding container 928d24935e162b6a40a219fb42e30789bc881a0d96185f603181944968378e47: Status 404 returned error can't find the container with id 928d24935e162b6a40a219fb42e30789bc881a0d96185f603181944968378e47 Jul 15 17:01:13.858129 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:13.857845 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b8564d945-6nq2w" event={"ID":"6a612a93-a727-4552-b21d-da7c2318980e","Type":"ContainerStarted","Data":"928d24935e162b6a40a219fb42e30789bc881a0d96185f603181944968378e47"} Jul 15 17:01:13.887489 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:13.887448 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:13.946672 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:13.946631 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:14.811445 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:14.811413 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb9d6e60-0095-487e-9c95-b26cfed57365-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vs662\" (UID: \"eb9d6e60-0095-487e-9c95-b26cfed57365\") " pod="openshift-insights/insights-runtime-extractor-vs662" Jul 15 17:01:14.814447 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:14.814423 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb9d6e60-0095-487e-9c95-b26cfed57365-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vs662\" (UID: \"eb9d6e60-0095-487e-9c95-b26cfed57365\") " pod="openshift-insights/insights-runtime-extractor-vs662" Jul 15 17:01:15.111558 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:15.111270 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vs662" Jul 15 17:01:15.143741 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:15.143272 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:15.254657 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:15.254577 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:15.284850 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:15.284816 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vs662"] Jul 15 17:01:15.287174 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:01:15.287147 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb9d6e60_0095_487e_9c95_b26cfed57365.slice/crio-52309ae54e416621bc7ef7a9d64853dc72e261bf2ae7ce2dcd2c8ff54a57bca4 WatchSource:0}: Error finding container 52309ae54e416621bc7ef7a9d64853dc72e261bf2ae7ce2dcd2c8ff54a57bca4: Status 404 returned error can't find the container with id 52309ae54e416621bc7ef7a9d64853dc72e261bf2ae7ce2dcd2c8ff54a57bca4 Jul 15 17:01:15.358637 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:15.358586 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:15.868322 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:15.868241 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vs662" event={"ID":"eb9d6e60-0095-487e-9c95-b26cfed57365","Type":"ContainerStarted","Data":"52309ae54e416621bc7ef7a9d64853dc72e261bf2ae7ce2dcd2c8ff54a57bca4"} Jul 15 17:01:15.869969 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:15.869941 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qtxm9" event={"ID":"68d72523-54cd-412a-b697-0f04e891d984","Type":"ContainerStarted","Data":"44a97890a23b6e52cc4f0093821b90d28b3e5979f035892abf50ce5c5e2a4e9b"} Jul 15 17:01:15.888026 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:15.887976 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qtxm9" podStartSLOduration=34.195530436 podStartE2EDuration="38.887957331s" totalImagesPullingTime="4.692426895s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:00:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:01:15.886850199 +0000 UTC m=+293.439980414" watchObservedRunningTime="2026-07-15 17:01:15.887957331 +0000 UTC m=+293.441087547" Jul 15 17:01:16.875307 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:16.875263 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b8564d945-6nq2w" event={"ID":"6a612a93-a727-4552-b21d-da7c2318980e","Type":"ContainerStarted","Data":"08058f17d47cf4db7e8b0c6a60ee78d22ab31f841d3a8bc40f381b6a228dc75f"} Jul 15 17:01:16.895273 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:16.895221 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b8564d945-6nq2w" podStartSLOduration=1.60316953 podStartE2EDuration="4.89520441s" totalImagesPullingTime="3.29203488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:01:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:01:16.893311614 +0000 UTC m=+294.446441843" watchObservedRunningTime="2026-07-15 17:01:16.89520441 +0000 UTC m=+294.448334686" Jul 15 17:01:16.985626 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:01:16.985567 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-62hbp" podUID="df1447df-1b9a-4dc3-ba2c-ecbab1514d09" Jul 15 17:01:17.259813 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:17.259771 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:20.846040 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:20.846009 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jvr75" Jul 15 17:01:21.018229 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:21.018190 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:21.118810 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:21.118730 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:21.187062 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:21.187018 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:22.405281 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:22.405241 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:22.470713 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:22.470677 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-rf7pz"] Jul 15 17:01:22.474337 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:22.474308 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-rf7pz" Jul 15 17:01:22.478688 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:22.478663 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 15 17:01:22.479197 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:22.479024 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 15 17:01:22.479197 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:22.479110 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 15 17:01:22.479197 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:22.479127 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-hsvf5\"" Jul 15 17:01:22.484890 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:22.484865 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-rf7pz"] Jul 15 17:01:22.492709 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:22.492639 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:22.585506 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:22.585465 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/367ca386-e717-4704-94a6-0ef7e9d5a6bf-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-rf7pz\" (UID: \"367ca386-e717-4704-94a6-0ef7e9d5a6bf\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rf7pz" Jul 15 17:01:22.585714 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:22.585535 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7pqw\" (UniqueName: \"kubernetes.io/projected/367ca386-e717-4704-94a6-0ef7e9d5a6bf-kube-api-access-w7pqw\") pod \"prometheus-operator-64785b6675-rf7pz\" (UID: \"367ca386-e717-4704-94a6-0ef7e9d5a6bf\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rf7pz" Jul 15 17:01:22.585714 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:22.585647 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/367ca386-e717-4704-94a6-0ef7e9d5a6bf-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-rf7pz\" (UID: \"367ca386-e717-4704-94a6-0ef7e9d5a6bf\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rf7pz" Jul 15 17:01:22.585714 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:22.585674 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/367ca386-e717-4704-94a6-0ef7e9d5a6bf-metrics-client-ca\") pod \"prometheus-operator-64785b6675-rf7pz\" (UID: \"367ca386-e717-4704-94a6-0ef7e9d5a6bf\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rf7pz" Jul 15 17:01:22.686484 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:22.686443 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/367ca386-e717-4704-94a6-0ef7e9d5a6bf-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-rf7pz\" (UID: \"367ca386-e717-4704-94a6-0ef7e9d5a6bf\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rf7pz" Jul 15 17:01:22.686484 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:22.686483 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/367ca386-e717-4704-94a6-0ef7e9d5a6bf-metrics-client-ca\") pod \"prometheus-operator-64785b6675-rf7pz\" (UID: \"367ca386-e717-4704-94a6-0ef7e9d5a6bf\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rf7pz" Jul 15 17:01:22.686745 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:22.686546 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/367ca386-e717-4704-94a6-0ef7e9d5a6bf-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-rf7pz\" (UID: \"367ca386-e717-4704-94a6-0ef7e9d5a6bf\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rf7pz" Jul 15 17:01:22.686745 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:22.686591 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7pqw\" (UniqueName: \"kubernetes.io/projected/367ca386-e717-4704-94a6-0ef7e9d5a6bf-kube-api-access-w7pqw\") pod \"prometheus-operator-64785b6675-rf7pz\" (UID: \"367ca386-e717-4704-94a6-0ef7e9d5a6bf\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rf7pz" Jul 15 17:01:22.686745 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:01:22.686701 2578 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 15 17:01:22.686902 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:01:22.686789 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/367ca386-e717-4704-94a6-0ef7e9d5a6bf-prometheus-operator-tls podName:367ca386-e717-4704-94a6-0ef7e9d5a6bf nodeName:}" failed. No retries permitted until 2026-07-15 17:01:23.186772932 +0000 UTC m=+300.739903125 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/367ca386-e717-4704-94a6-0ef7e9d5a6bf-prometheus-operator-tls") pod "prometheus-operator-64785b6675-rf7pz" (UID: "367ca386-e717-4704-94a6-0ef7e9d5a6bf") : secret "prometheus-operator-tls" not found Jul 15 17:01:22.687370 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:22.687346 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/367ca386-e717-4704-94a6-0ef7e9d5a6bf-metrics-client-ca\") pod \"prometheus-operator-64785b6675-rf7pz\" (UID: \"367ca386-e717-4704-94a6-0ef7e9d5a6bf\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rf7pz" Jul 15 17:01:22.689145 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:22.689119 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/367ca386-e717-4704-94a6-0ef7e9d5a6bf-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-rf7pz\" (UID: \"367ca386-e717-4704-94a6-0ef7e9d5a6bf\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rf7pz" Jul 15 17:01:22.705515 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:22.705488 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7pqw\" (UniqueName: \"kubernetes.io/projected/367ca386-e717-4704-94a6-0ef7e9d5a6bf-kube-api-access-w7pqw\") pod \"prometheus-operator-64785b6675-rf7pz\" (UID: \"367ca386-e717-4704-94a6-0ef7e9d5a6bf\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rf7pz" Jul 15 17:01:23.190982 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:23.190951 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/367ca386-e717-4704-94a6-0ef7e9d5a6bf-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-rf7pz\" (UID: \"367ca386-e717-4704-94a6-0ef7e9d5a6bf\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rf7pz" Jul 15 17:01:23.193386 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:23.193363 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/367ca386-e717-4704-94a6-0ef7e9d5a6bf-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-rf7pz\" (UID: \"367ca386-e717-4704-94a6-0ef7e9d5a6bf\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rf7pz" Jul 15 17:01:23.288289 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:23.288246 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5b8564d945-6nq2w" Jul 15 17:01:23.288459 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:23.288318 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b8564d945-6nq2w" Jul 15 17:01:23.294614 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:23.294576 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5b8564d945-6nq2w" Jul 15 17:01:23.389034 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:23.389004 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-hsvf5\"" Jul 15 17:01:23.396730 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:23.396707 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-rf7pz" Jul 15 17:01:23.774805 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:23.774769 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:23.838212 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:23.838178 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:23.910387 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:23.910355 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5b8564d945-6nq2w" Jul 15 17:01:25.634560 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:25.634526 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b8564d945-6nq2w"] Jul 15 17:01:26.563727 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:26.563690 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:27.423935 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:27.423902 2578 patch_prober.go:28] interesting pod/image-registry-795b87c6c7-xztkp container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 15 17:01:27.424392 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:27.423974 2578 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" podUID="f0d3cec0-475c-481e-beb3-012b1c8c215b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 17:01:27.921689 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:27.921571 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:27.984684 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:27.984497 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 17:01:28.011946 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:28.011913 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:28.078884 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:28.078848 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:28.832223 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:28.832153 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 17:01:29.055776 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:29.055742 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29wtx_88f8928c-237e-457f-a032-707f48111f36/ovn-acl-logging/0.log" Jul 15 17:01:29.056748 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:29.056727 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29wtx_88f8928c-237e-457f-a032-707f48111f36/ovn-acl-logging/0.log" Jul 15 17:01:29.059284 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:29.059257 2578 kubelet.go:1648] "Image garbage collection succeeded" Jul 15 17:01:29.168883 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:29.168856 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 17:01:29.170462 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:29.170439 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-rf7pz"] Jul 15 17:01:29.175211 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:01:29.175186 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod367ca386_e717_4704_94a6_0ef7e9d5a6bf.slice/crio-22d37929b868b4fce3f8bee4e568bea034bd7f5c3d26d8e38b2b7d4c1e435e4a WatchSource:0}: Error finding container 22d37929b868b4fce3f8bee4e568bea034bd7f5c3d26d8e38b2b7d4c1e435e4a: Status 404 returned error can't find the container with id 22d37929b868b4fce3f8bee4e568bea034bd7f5c3d26d8e38b2b7d4c1e435e4a Jul 15 17:01:29.197248 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:29.197191 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:29.261163 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:29.261131 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:29.486527 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:29.486493 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-795b87c6c7-xztkp"] Jul 15 17:01:29.573702 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:29.573657 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:29.675613 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:29.675564 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:29.782787 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:29.782709 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:29.938987 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:29.938648 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-fvqjs" event={"ID":"9446a817-b692-4190-89c1-a8ae61d15bb7","Type":"ContainerStarted","Data":"79fe03f1974f3c99749f78ce1ed03b0d38a89b148f4d6b8d46e691d727933375"} Jul 15 17:01:29.939432 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:29.939089 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-fvqjs" Jul 15 17:01:29.941590 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:29.940572 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-rf7pz" event={"ID":"367ca386-e717-4704-94a6-0ef7e9d5a6bf","Type":"ContainerStarted","Data":"22d37929b868b4fce3f8bee4e568bea034bd7f5c3d26d8e38b2b7d4c1e435e4a"} Jul 15 17:01:29.943934 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:29.943907 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vs662" event={"ID":"eb9d6e60-0095-487e-9c95-b26cfed57365","Type":"ContainerStarted","Data":"c833ee51946de7cb1efda5c0c3fdc10a51b46c49a67441e3eecd876a1dc45b09"} Jul 15 17:01:29.951079 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:29.951055 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-fvqjs" Jul 15 17:01:29.957451 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:29.957353 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-fvqjs" podStartSLOduration=2.243947574 podStartE2EDuration="22.957335413s" totalImagesPullingTime="20.713387839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:01:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:01:29.957019629 +0000 UTC m=+307.510149865" watchObservedRunningTime="2026-07-15 17:01:29.957335413 +0000 UTC m=+307.510465630" Jul 15 17:01:33.161482 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:33.161445 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:33.264100 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:33.264063 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:34.696710 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:34.696671 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:34.968249 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:34.968167 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-rf7pz" event={"ID":"367ca386-e717-4704-94a6-0ef7e9d5a6bf","Type":"ContainerStarted","Data":"b344659e91df1a7861f92d97ab4dab80c2722dad8422fd27f148f3c7413c7d77"} Jul 15 17:01:34.968414 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:34.968265 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-rf7pz" event={"ID":"367ca386-e717-4704-94a6-0ef7e9d5a6bf","Type":"ContainerStarted","Data":"4b3639402b898b1ead4958253a26a294e03ff717eb8c4e00d4da2c642ae215ed"} Jul 15 17:01:34.970214 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:34.970187 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vs662" event={"ID":"eb9d6e60-0095-487e-9c95-b26cfed57365","Type":"ContainerStarted","Data":"0f780d88ec327001de9eaafb8cb5d2972d79598a014727a91e36825e0c6cf923"} Jul 15 17:01:34.986877 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:34.986826 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-rf7pz" podStartSLOduration=7.830919886 podStartE2EDuration="12.986807913s" totalImagesPullingTime="5.155888027s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:01:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:01:34.985918768 +0000 UTC m=+312.539049006" watchObservedRunningTime="2026-07-15 17:01:34.986807913 +0000 UTC m=+312.539938127" Jul 15 17:01:36.555492 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:36.555443 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:36.789643 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:36.789528 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:36.833778 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:36.833707 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7bfnv"] Jul 15 17:01:36.869366 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:36.869291 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7bfnv" Jul 15 17:01:36.872468 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:36.872417 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 15 17:01:36.872468 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:36.872429 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 15 17:01:36.873401 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:36.873212 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 15 17:01:36.873401 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:36.873296 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zdfnb\"" Jul 15 17:01:37.026475 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.026443 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6e05b600-bd45-4d94-8988-48895eb43d8f-node-exporter-tls\") pod \"node-exporter-7bfnv\" (UID: \"6e05b600-bd45-4d94-8988-48895eb43d8f\") " pod="openshift-monitoring/node-exporter-7bfnv" Jul 15 17:01:37.026672 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.026515 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e05b600-bd45-4d94-8988-48895eb43d8f-sys\") pod \"node-exporter-7bfnv\" (UID: \"6e05b600-bd45-4d94-8988-48895eb43d8f\") " pod="openshift-monitoring/node-exporter-7bfnv" Jul 15 17:01:37.026672 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.026559 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e05b600-bd45-4d94-8988-48895eb43d8f-metrics-client-ca\") pod \"node-exporter-7bfnv\" (UID: \"6e05b600-bd45-4d94-8988-48895eb43d8f\") " pod="openshift-monitoring/node-exporter-7bfnv" Jul 15 17:01:37.026672 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.026583 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6e05b600-bd45-4d94-8988-48895eb43d8f-root\") pod \"node-exporter-7bfnv\" (UID: \"6e05b600-bd45-4d94-8988-48895eb43d8f\") " pod="openshift-monitoring/node-exporter-7bfnv" Jul 15 17:01:37.026672 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.026631 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6e05b600-bd45-4d94-8988-48895eb43d8f-node-exporter-wtmp\") pod \"node-exporter-7bfnv\" (UID: \"6e05b600-bd45-4d94-8988-48895eb43d8f\") " pod="openshift-monitoring/node-exporter-7bfnv" Jul 15 17:01:37.026672 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.026662 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6e05b600-bd45-4d94-8988-48895eb43d8f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7bfnv\" (UID: \"6e05b600-bd45-4d94-8988-48895eb43d8f\") " pod="openshift-monitoring/node-exporter-7bfnv" Jul 15 17:01:37.026949 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.026685 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dzzt\" (UniqueName: \"kubernetes.io/projected/6e05b600-bd45-4d94-8988-48895eb43d8f-kube-api-access-5dzzt\") pod \"node-exporter-7bfnv\" (UID: \"6e05b600-bd45-4d94-8988-48895eb43d8f\") " pod="openshift-monitoring/node-exporter-7bfnv" Jul 15 17:01:37.026949 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.026739 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6e05b600-bd45-4d94-8988-48895eb43d8f-node-exporter-accelerators-collector-config\") pod \"node-exporter-7bfnv\" (UID: \"6e05b600-bd45-4d94-8988-48895eb43d8f\") " pod="openshift-monitoring/node-exporter-7bfnv" Jul 15 17:01:37.026949 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.026782 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6e05b600-bd45-4d94-8988-48895eb43d8f-node-exporter-textfile\") pod \"node-exporter-7bfnv\" (UID: \"6e05b600-bd45-4d94-8988-48895eb43d8f\") " pod="openshift-monitoring/node-exporter-7bfnv" Jul 15 17:01:37.127761 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.127672 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6e05b600-bd45-4d94-8988-48895eb43d8f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7bfnv\" (UID: \"6e05b600-bd45-4d94-8988-48895eb43d8f\") " pod="openshift-monitoring/node-exporter-7bfnv" Jul 15 17:01:37.127761 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.127721 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dzzt\" (UniqueName: \"kubernetes.io/projected/6e05b600-bd45-4d94-8988-48895eb43d8f-kube-api-access-5dzzt\") pod \"node-exporter-7bfnv\" (UID: \"6e05b600-bd45-4d94-8988-48895eb43d8f\") " pod="openshift-monitoring/node-exporter-7bfnv" Jul 15 17:01:37.127993 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.127769 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6e05b600-bd45-4d94-8988-48895eb43d8f-node-exporter-accelerators-collector-config\") pod \"node-exporter-7bfnv\" (UID: \"6e05b600-bd45-4d94-8988-48895eb43d8f\") " pod="openshift-monitoring/node-exporter-7bfnv" Jul 15 17:01:37.128535 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.128092 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6e05b600-bd45-4d94-8988-48895eb43d8f-node-exporter-textfile\") pod \"node-exporter-7bfnv\" (UID: \"6e05b600-bd45-4d94-8988-48895eb43d8f\") " pod="openshift-monitoring/node-exporter-7bfnv" Jul 15 17:01:37.128535 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.128172 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6e05b600-bd45-4d94-8988-48895eb43d8f-node-exporter-tls\") pod \"node-exporter-7bfnv\" (UID: \"6e05b600-bd45-4d94-8988-48895eb43d8f\") " pod="openshift-monitoring/node-exporter-7bfnv" Jul 15 17:01:37.128535 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.128232 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e05b600-bd45-4d94-8988-48895eb43d8f-sys\") pod \"node-exporter-7bfnv\" (UID: \"6e05b600-bd45-4d94-8988-48895eb43d8f\") " pod="openshift-monitoring/node-exporter-7bfnv" Jul 15 17:01:37.128535 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.128272 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e05b600-bd45-4d94-8988-48895eb43d8f-metrics-client-ca\") pod \"node-exporter-7bfnv\" (UID: \"6e05b600-bd45-4d94-8988-48895eb43d8f\") " pod="openshift-monitoring/node-exporter-7bfnv" Jul 15 17:01:37.128535 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.128299 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6e05b600-bd45-4d94-8988-48895eb43d8f-root\") pod \"node-exporter-7bfnv\" (UID: \"6e05b600-bd45-4d94-8988-48895eb43d8f\") " pod="openshift-monitoring/node-exporter-7bfnv" Jul 15 17:01:37.128535 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.128324 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6e05b600-bd45-4d94-8988-48895eb43d8f-node-exporter-wtmp\") pod \"node-exporter-7bfnv\" (UID: \"6e05b600-bd45-4d94-8988-48895eb43d8f\") " pod="openshift-monitoring/node-exporter-7bfnv" Jul 15 17:01:37.128535 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.128345 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6e05b600-bd45-4d94-8988-48895eb43d8f-node-exporter-accelerators-collector-config\") pod \"node-exporter-7bfnv\" (UID: \"6e05b600-bd45-4d94-8988-48895eb43d8f\") " pod="openshift-monitoring/node-exporter-7bfnv" Jul 15 17:01:37.128535 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.128402 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6e05b600-bd45-4d94-8988-48895eb43d8f-node-exporter-textfile\") pod \"node-exporter-7bfnv\" (UID: \"6e05b600-bd45-4d94-8988-48895eb43d8f\") " pod="openshift-monitoring/node-exporter-7bfnv" Jul 15 17:01:37.128535 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.128470 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6e05b600-bd45-4d94-8988-48895eb43d8f-root\") pod \"node-exporter-7bfnv\" (UID: \"6e05b600-bd45-4d94-8988-48895eb43d8f\") " pod="openshift-monitoring/node-exporter-7bfnv" Jul 15 17:01:37.128535 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.128488 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6e05b600-bd45-4d94-8988-48895eb43d8f-node-exporter-wtmp\") pod \"node-exporter-7bfnv\" (UID: \"6e05b600-bd45-4d94-8988-48895eb43d8f\") " pod="openshift-monitoring/node-exporter-7bfnv" Jul 15 17:01:37.128535 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.128517 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e05b600-bd45-4d94-8988-48895eb43d8f-sys\") pod \"node-exporter-7bfnv\" (UID: \"6e05b600-bd45-4d94-8988-48895eb43d8f\") " pod="openshift-monitoring/node-exporter-7bfnv" Jul 15 17:01:37.129164 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.128810 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e05b600-bd45-4d94-8988-48895eb43d8f-metrics-client-ca\") pod \"node-exporter-7bfnv\" (UID: \"6e05b600-bd45-4d94-8988-48895eb43d8f\") " pod="openshift-monitoring/node-exporter-7bfnv" Jul 15 17:01:37.131108 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.131083 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6e05b600-bd45-4d94-8988-48895eb43d8f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7bfnv\" (UID: \"6e05b600-bd45-4d94-8988-48895eb43d8f\") " pod="openshift-monitoring/node-exporter-7bfnv" Jul 15 17:01:37.131462 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.131407 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6e05b600-bd45-4d94-8988-48895eb43d8f-node-exporter-tls\") pod \"node-exporter-7bfnv\" (UID: \"6e05b600-bd45-4d94-8988-48895eb43d8f\") " pod="openshift-monitoring/node-exporter-7bfnv" Jul 15 17:01:37.136975 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.136954 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dzzt\" (UniqueName: \"kubernetes.io/projected/6e05b600-bd45-4d94-8988-48895eb43d8f-kube-api-access-5dzzt\") pod \"node-exporter-7bfnv\" (UID: \"6e05b600-bd45-4d94-8988-48895eb43d8f\") " pod="openshift-monitoring/node-exporter-7bfnv" Jul 15 17:01:37.183460 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.183422 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7bfnv" Jul 15 17:01:37.763586 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.763545 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:37.861393 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.861342 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:37.879186 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.877093 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 17:01:37.961458 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.958906 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 17:01:37.961458 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.959299 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:37.963374 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.962866 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:37.965117 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.965096 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 15 17:01:37.965379 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.965349 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 15 17:01:37.965547 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.965529 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 15 17:01:37.965746 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.965728 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 15 17:01:37.965926 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.965898 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 15 17:01:37.966401 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.966363 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 15 17:01:37.966646 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.966625 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 15 17:01:37.966856 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.966835 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 15 17:01:37.967072 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.967052 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 15 17:01:37.967252 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:37.967212 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-bnspc\"" Jul 15 17:01:38.038194 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.037818 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3018848b-f926-43e9-a530-bcb25f059f95-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.038194 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.037869 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3018848b-f926-43e9-a530-bcb25f059f95-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.038194 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.037902 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3018848b-f926-43e9-a530-bcb25f059f95-config-volume\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.038194 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.037937 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3018848b-f926-43e9-a530-bcb25f059f95-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.038194 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.037996 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3018848b-f926-43e9-a530-bcb25f059f95-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.038194 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.038021 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3018848b-f926-43e9-a530-bcb25f059f95-web-config\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.038194 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.038059 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3018848b-f926-43e9-a530-bcb25f059f95-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.038194 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.038086 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3018848b-f926-43e9-a530-bcb25f059f95-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.038194 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.038166 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3018848b-f926-43e9-a530-bcb25f059f95-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.038755 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.038213 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3018848b-f926-43e9-a530-bcb25f059f95-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.038755 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.038255 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3018848b-f926-43e9-a530-bcb25f059f95-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.038755 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.038287 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3018848b-f926-43e9-a530-bcb25f059f95-config-out\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.038755 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.038319 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7694x\" (UniqueName: \"kubernetes.io/projected/3018848b-f926-43e9-a530-bcb25f059f95-kube-api-access-7694x\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.139257 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.139218 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3018848b-f926-43e9-a530-bcb25f059f95-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.139452 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.139273 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3018848b-f926-43e9-a530-bcb25f059f95-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.139452 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.139304 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3018848b-f926-43e9-a530-bcb25f059f95-config-volume\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.139452 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.139335 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3018848b-f926-43e9-a530-bcb25f059f95-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.139452 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.139397 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3018848b-f926-43e9-a530-bcb25f059f95-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.139452 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.139423 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3018848b-f926-43e9-a530-bcb25f059f95-web-config\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.139757 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.139460 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3018848b-f926-43e9-a530-bcb25f059f95-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.139757 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.139485 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3018848b-f926-43e9-a530-bcb25f059f95-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.139757 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.139543 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3018848b-f926-43e9-a530-bcb25f059f95-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.139757 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.139570 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3018848b-f926-43e9-a530-bcb25f059f95-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.139757 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.139637 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3018848b-f926-43e9-a530-bcb25f059f95-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.139757 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.139669 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3018848b-f926-43e9-a530-bcb25f059f95-config-out\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.139757 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.139704 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7694x\" (UniqueName: \"kubernetes.io/projected/3018848b-f926-43e9-a530-bcb25f059f95-kube-api-access-7694x\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.140223 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.140191 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3018848b-f926-43e9-a530-bcb25f059f95-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.141818 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.141494 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3018848b-f926-43e9-a530-bcb25f059f95-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.142662 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.142613 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3018848b-f926-43e9-a530-bcb25f059f95-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.149070 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.146852 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3018848b-f926-43e9-a530-bcb25f059f95-config-out\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.149070 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.147089 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3018848b-f926-43e9-a530-bcb25f059f95-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.149070 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.147512 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3018848b-f926-43e9-a530-bcb25f059f95-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.149070 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.147982 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3018848b-f926-43e9-a530-bcb25f059f95-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.149070 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.148911 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3018848b-f926-43e9-a530-bcb25f059f95-config-volume\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.149374 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.149001 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3018848b-f926-43e9-a530-bcb25f059f95-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.152654 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.152483 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3018848b-f926-43e9-a530-bcb25f059f95-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.160497 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.160473 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3018848b-f926-43e9-a530-bcb25f059f95-web-config\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.161675 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.161647 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3018848b-f926-43e9-a530-bcb25f059f95-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.163114 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.162879 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7694x\" (UniqueName: \"kubernetes.io/projected/3018848b-f926-43e9-a530-bcb25f059f95-kube-api-access-7694x\") pod \"alertmanager-main-0\" (UID: \"3018848b-f926-43e9-a530-bcb25f059f95\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.276326 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.276241 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:01:38.651686 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.651540 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:38.749349 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:38.749316 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:39.128504 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:01:39.128362 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e05b600_bd45_4d94_8988_48895eb43d8f.slice/crio-739ca788916d862a5039a2de817c6ea36b68204a3667401f54b7ce66b9859281 WatchSource:0}: Error finding container 739ca788916d862a5039a2de817c6ea36b68204a3667401f54b7ce66b9859281: Status 404 returned error can't find the container with id 739ca788916d862a5039a2de817c6ea36b68204a3667401f54b7ce66b9859281 Jul 15 17:01:39.275340 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:39.275308 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 17:01:39.275673 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:01:39.275651 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3018848b_f926_43e9_a530_bcb25f059f95.slice/crio-c185507d221170f9b9f26eb4c70e229882d963cdc0649d06d2d9aa7dfcf81782 WatchSource:0}: Error finding container c185507d221170f9b9f26eb4c70e229882d963cdc0649d06d2d9aa7dfcf81782: Status 404 returned error can't find the container with id c185507d221170f9b9f26eb4c70e229882d963cdc0649d06d2d9aa7dfcf81782 Jul 15 17:01:39.283765 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:39.283742 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:39.374569 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:39.374541 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:39.793077 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:39.793047 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:39.852425 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:39.852346 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:39.970728 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:39.970643 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:39.992234 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:39.992200 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7bfnv" event={"ID":"6e05b600-bd45-4d94-8988-48895eb43d8f","Type":"ContainerStarted","Data":"739ca788916d862a5039a2de817c6ea36b68204a3667401f54b7ce66b9859281"} Jul 15 17:01:39.993656 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:39.993623 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3018848b-f926-43e9-a530-bcb25f059f95","Type":"ContainerStarted","Data":"c185507d221170f9b9f26eb4c70e229882d963cdc0649d06d2d9aa7dfcf81782"} Jul 15 17:01:39.995963 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:39.995929 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vs662" event={"ID":"eb9d6e60-0095-487e-9c95-b26cfed57365","Type":"ContainerStarted","Data":"43d5a960b3414947786752b45d34a48a06799e8041977045ac3ffb51633717b7"} Jul 15 17:01:40.020231 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:40.020172 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-vs662" podStartSLOduration=33.936492787 podStartE2EDuration="58.020152127s" totalImagesPullingTime="24.08365934s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 17:00:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:01:40.019531692 +0000 UTC m=+317.572661907" watchObservedRunningTime="2026-07-15 17:01:40.020152127 +0000 UTC m=+317.573282323" Jul 15 17:01:40.072112 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:40.072072 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:41.110702 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.110681 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:41.210502 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.210473 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:41.273810 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.273784 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:41.626633 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.626583 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-6w89l"] Jul 15 17:01:41.648452 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.648405 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:41.656700 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.656666 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d8fbcbc47-hwn7m"] Jul 15 17:01:41.656856 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.656778 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6w89l" Jul 15 17:01:41.663238 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.663101 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 15 17:01:41.663238 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.663132 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-mvnbl\"" Jul 15 17:01:41.668381 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.668359 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-6w89l"] Jul 15 17:01:41.668474 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.668386 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d8fbcbc47-hwn7m"] Jul 15 17:01:41.668474 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.668457 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d8fbcbc47-hwn7m" Jul 15 17:01:41.676405 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.676386 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 15 17:01:41.726709 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.726677 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:41.772750 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.772717 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63edfb69-4852-4663-a6d7-35b77581b129-trusted-ca-bundle\") pod \"console-5d8fbcbc47-hwn7m\" (UID: \"63edfb69-4852-4663-a6d7-35b77581b129\") " pod="openshift-console/console-5d8fbcbc47-hwn7m" Jul 15 17:01:41.772927 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.772765 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/63edfb69-4852-4663-a6d7-35b77581b129-console-config\") pod \"console-5d8fbcbc47-hwn7m\" (UID: \"63edfb69-4852-4663-a6d7-35b77581b129\") " pod="openshift-console/console-5d8fbcbc47-hwn7m" Jul 15 17:01:41.772927 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.772789 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/63edfb69-4852-4663-a6d7-35b77581b129-oauth-serving-cert\") pod \"console-5d8fbcbc47-hwn7m\" (UID: \"63edfb69-4852-4663-a6d7-35b77581b129\") " pod="openshift-console/console-5d8fbcbc47-hwn7m" Jul 15 17:01:41.772927 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.772888 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/63edfb69-4852-4663-a6d7-35b77581b129-console-oauth-config\") pod \"console-5d8fbcbc47-hwn7m\" (UID: \"63edfb69-4852-4663-a6d7-35b77581b129\") " pod="openshift-console/console-5d8fbcbc47-hwn7m" Jul 15 17:01:41.773060 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.772933 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5n528\" (UniqueName: \"kubernetes.io/projected/56d144de-0dbf-4f44-ad3e-3277889a317c-kube-api-access-5n528\") pod \"monitoring-plugin-5f68df7f58-6w89l\" (UID: \"56d144de-0dbf-4f44-ad3e-3277889a317c\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6w89l" Jul 15 17:01:41.773060 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.772968 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt6x2\" (UniqueName: \"kubernetes.io/projected/63edfb69-4852-4663-a6d7-35b77581b129-kube-api-access-mt6x2\") pod \"console-5d8fbcbc47-hwn7m\" (UID: \"63edfb69-4852-4663-a6d7-35b77581b129\") " pod="openshift-console/console-5d8fbcbc47-hwn7m" Jul 15 17:01:41.773060 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.773018 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/63edfb69-4852-4663-a6d7-35b77581b129-service-ca\") pod \"console-5d8fbcbc47-hwn7m\" (UID: \"63edfb69-4852-4663-a6d7-35b77581b129\") " pod="openshift-console/console-5d8fbcbc47-hwn7m" Jul 15 17:01:41.773060 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.773043 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/63edfb69-4852-4663-a6d7-35b77581b129-console-serving-cert\") pod \"console-5d8fbcbc47-hwn7m\" (UID: \"63edfb69-4852-4663-a6d7-35b77581b129\") " pod="openshift-console/console-5d8fbcbc47-hwn7m" Jul 15 17:01:41.773194 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.773081 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/56d144de-0dbf-4f44-ad3e-3277889a317c-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-6w89l\" (UID: \"56d144de-0dbf-4f44-ad3e-3277889a317c\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6w89l" Jul 15 17:01:41.791494 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.791466 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:41.874440 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.874409 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/63edfb69-4852-4663-a6d7-35b77581b129-console-config\") pod \"console-5d8fbcbc47-hwn7m\" (UID: \"63edfb69-4852-4663-a6d7-35b77581b129\") " pod="openshift-console/console-5d8fbcbc47-hwn7m" Jul 15 17:01:41.874440 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.874441 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/63edfb69-4852-4663-a6d7-35b77581b129-oauth-serving-cert\") pod \"console-5d8fbcbc47-hwn7m\" (UID: \"63edfb69-4852-4663-a6d7-35b77581b129\") " pod="openshift-console/console-5d8fbcbc47-hwn7m" Jul 15 17:01:41.874665 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.874468 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/63edfb69-4852-4663-a6d7-35b77581b129-console-oauth-config\") pod \"console-5d8fbcbc47-hwn7m\" (UID: \"63edfb69-4852-4663-a6d7-35b77581b129\") " pod="openshift-console/console-5d8fbcbc47-hwn7m" Jul 15 17:01:41.874665 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.874616 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5n528\" (UniqueName: \"kubernetes.io/projected/56d144de-0dbf-4f44-ad3e-3277889a317c-kube-api-access-5n528\") pod \"monitoring-plugin-5f68df7f58-6w89l\" (UID: \"56d144de-0dbf-4f44-ad3e-3277889a317c\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6w89l" Jul 15 17:01:41.874665 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.874655 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt6x2\" (UniqueName: \"kubernetes.io/projected/63edfb69-4852-4663-a6d7-35b77581b129-kube-api-access-mt6x2\") pod \"console-5d8fbcbc47-hwn7m\" (UID: \"63edfb69-4852-4663-a6d7-35b77581b129\") " pod="openshift-console/console-5d8fbcbc47-hwn7m" Jul 15 17:01:41.874781 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.874697 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/63edfb69-4852-4663-a6d7-35b77581b129-service-ca\") pod \"console-5d8fbcbc47-hwn7m\" (UID: \"63edfb69-4852-4663-a6d7-35b77581b129\") " pod="openshift-console/console-5d8fbcbc47-hwn7m" Jul 15 17:01:41.874781 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.874730 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/63edfb69-4852-4663-a6d7-35b77581b129-console-serving-cert\") pod \"console-5d8fbcbc47-hwn7m\" (UID: \"63edfb69-4852-4663-a6d7-35b77581b129\") " pod="openshift-console/console-5d8fbcbc47-hwn7m" Jul 15 17:01:41.874849 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.874786 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/56d144de-0dbf-4f44-ad3e-3277889a317c-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-6w89l\" (UID: \"56d144de-0dbf-4f44-ad3e-3277889a317c\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6w89l" Jul 15 17:01:41.874892 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.874865 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63edfb69-4852-4663-a6d7-35b77581b129-trusted-ca-bundle\") pod \"console-5d8fbcbc47-hwn7m\" (UID: \"63edfb69-4852-4663-a6d7-35b77581b129\") " pod="openshift-console/console-5d8fbcbc47-hwn7m" Jul 15 17:01:41.875031 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:01:41.875013 2578 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 15 17:01:41.875085 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:01:41.875076 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56d144de-0dbf-4f44-ad3e-3277889a317c-monitoring-plugin-cert podName:56d144de-0dbf-4f44-ad3e-3277889a317c nodeName:}" failed. No retries permitted until 2026-07-15 17:01:42.375057903 +0000 UTC m=+319.928188097 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/56d144de-0dbf-4f44-ad3e-3277889a317c-monitoring-plugin-cert") pod "monitoring-plugin-5f68df7f58-6w89l" (UID: "56d144de-0dbf-4f44-ad3e-3277889a317c") : secret "monitoring-plugin-cert" not found Jul 15 17:01:41.875145 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.875084 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/63edfb69-4852-4663-a6d7-35b77581b129-oauth-serving-cert\") pod \"console-5d8fbcbc47-hwn7m\" (UID: \"63edfb69-4852-4663-a6d7-35b77581b129\") " pod="openshift-console/console-5d8fbcbc47-hwn7m" Jul 15 17:01:41.875281 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.875262 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/63edfb69-4852-4663-a6d7-35b77581b129-console-config\") pod \"console-5d8fbcbc47-hwn7m\" (UID: \"63edfb69-4852-4663-a6d7-35b77581b129\") " pod="openshift-console/console-5d8fbcbc47-hwn7m" Jul 15 17:01:41.875481 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.875461 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/63edfb69-4852-4663-a6d7-35b77581b129-service-ca\") pod \"console-5d8fbcbc47-hwn7m\" (UID: \"63edfb69-4852-4663-a6d7-35b77581b129\") " pod="openshift-console/console-5d8fbcbc47-hwn7m" Jul 15 17:01:41.876246 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.876218 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63edfb69-4852-4663-a6d7-35b77581b129-trusted-ca-bundle\") pod \"console-5d8fbcbc47-hwn7m\" (UID: \"63edfb69-4852-4663-a6d7-35b77581b129\") " pod="openshift-console/console-5d8fbcbc47-hwn7m" Jul 15 17:01:41.876998 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.876955 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/63edfb69-4852-4663-a6d7-35b77581b129-console-oauth-config\") pod \"console-5d8fbcbc47-hwn7m\" (UID: \"63edfb69-4852-4663-a6d7-35b77581b129\") " pod="openshift-console/console-5d8fbcbc47-hwn7m" Jul 15 17:01:41.877316 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.877301 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/63edfb69-4852-4663-a6d7-35b77581b129-console-serving-cert\") pod \"console-5d8fbcbc47-hwn7m\" (UID: \"63edfb69-4852-4663-a6d7-35b77581b129\") " pod="openshift-console/console-5d8fbcbc47-hwn7m" Jul 15 17:01:41.892947 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.892924 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5n528\" (UniqueName: \"kubernetes.io/projected/56d144de-0dbf-4f44-ad3e-3277889a317c-kube-api-access-5n528\") pod \"monitoring-plugin-5f68df7f58-6w89l\" (UID: \"56d144de-0dbf-4f44-ad3e-3277889a317c\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6w89l" Jul 15 17:01:41.893026 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.892989 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt6x2\" (UniqueName: \"kubernetes.io/projected/63edfb69-4852-4663-a6d7-35b77581b129-kube-api-access-mt6x2\") pod \"console-5d8fbcbc47-hwn7m\" (UID: \"63edfb69-4852-4663-a6d7-35b77581b129\") " pod="openshift-console/console-5d8fbcbc47-hwn7m" Jul 15 17:01:41.976847 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:41.976818 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d8fbcbc47-hwn7m" Jul 15 17:01:42.004872 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:42.004843 2578 generic.go:356] "Generic (PLEG): container finished" podID="3018848b-f926-43e9-a530-bcb25f059f95" containerID="b704f13c4b290354c6ed2944da362d95ada554ef0fe18849de019ac73c0cb9bb" exitCode=0 Jul 15 17:01:42.005009 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:42.004876 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3018848b-f926-43e9-a530-bcb25f059f95","Type":"ContainerDied","Data":"b704f13c4b290354c6ed2944da362d95ada554ef0fe18849de019ac73c0cb9bb"} Jul 15 17:01:42.006800 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:42.006780 2578 generic.go:356] "Generic (PLEG): container finished" podID="6e05b600-bd45-4d94-8988-48895eb43d8f" containerID="4ae52f01b6fd5574b6dcdda0054d130010fed50ce840615f9bb1962c56e2d75d" exitCode=0 Jul 15 17:01:42.006896 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:42.006808 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7bfnv" event={"ID":"6e05b600-bd45-4d94-8988-48895eb43d8f","Type":"ContainerDied","Data":"4ae52f01b6fd5574b6dcdda0054d130010fed50ce840615f9bb1962c56e2d75d"} Jul 15 17:01:42.139703 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:42.139673 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d8fbcbc47-hwn7m"] Jul 15 17:01:42.142019 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:01:42.141995 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63edfb69_4852_4663_a6d7_35b77581b129.slice/crio-170f6648890833a37fc986b9a5480bcb6e0073f4daaf39386d6a1c384984e6e3 WatchSource:0}: Error finding container 170f6648890833a37fc986b9a5480bcb6e0073f4daaf39386d6a1c384984e6e3: Status 404 returned error can't find the container with id 170f6648890833a37fc986b9a5480bcb6e0073f4daaf39386d6a1c384984e6e3 Jul 15 17:01:42.196751 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:42.196725 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:42.379846 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:42.379808 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/56d144de-0dbf-4f44-ad3e-3277889a317c-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-6w89l\" (UID: \"56d144de-0dbf-4f44-ad3e-3277889a317c\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6w89l" Jul 15 17:01:42.382181 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:42.382154 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/56d144de-0dbf-4f44-ad3e-3277889a317c-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-6w89l\" (UID: \"56d144de-0dbf-4f44-ad3e-3277889a317c\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6w89l" Jul 15 17:01:42.565717 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:42.565683 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6w89l" Jul 15 17:01:42.712808 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:42.712780 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-6w89l"] Jul 15 17:01:42.714902 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:01:42.714867 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56d144de_0dbf_4f44_ad3e_3277889a317c.slice/crio-d7b61d0d1fa8c35265713faa33e0afb107a25428cf58068689be0bded055c29d WatchSource:0}: Error finding container d7b61d0d1fa8c35265713faa33e0afb107a25428cf58068689be0bded055c29d: Status 404 returned error can't find the container with id d7b61d0d1fa8c35265713faa33e0afb107a25428cf58068689be0bded055c29d Jul 15 17:01:43.012730 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:43.012692 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6w89l" event={"ID":"56d144de-0dbf-4f44-ad3e-3277889a317c","Type":"ContainerStarted","Data":"d7b61d0d1fa8c35265713faa33e0afb107a25428cf58068689be0bded055c29d"} Jul 15 17:01:43.014786 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:43.014680 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d8fbcbc47-hwn7m" event={"ID":"63edfb69-4852-4663-a6d7-35b77581b129","Type":"ContainerStarted","Data":"e029307ee23180a05f4f23db7cbf59632b0f37b610d1d461fc26bc308da2df51"} Jul 15 17:01:43.014786 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:43.014727 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d8fbcbc47-hwn7m" event={"ID":"63edfb69-4852-4663-a6d7-35b77581b129","Type":"ContainerStarted","Data":"170f6648890833a37fc986b9a5480bcb6e0073f4daaf39386d6a1c384984e6e3"} Jul 15 17:01:43.018007 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:43.017808 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7bfnv" event={"ID":"6e05b600-bd45-4d94-8988-48895eb43d8f","Type":"ContainerStarted","Data":"907adc1499b0ec7c65d9482d6d71d83f8851b48d757dcb12db076f7f892e1a11"} Jul 15 17:01:43.018007 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:43.017859 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7bfnv" event={"ID":"6e05b600-bd45-4d94-8988-48895eb43d8f","Type":"ContainerStarted","Data":"5d74b38d1d980e4287d0324777f88afa831825b86b9b2340ebd8111273539f03"} Jul 15 17:01:43.040523 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:43.040459 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5d8fbcbc47-hwn7m" podStartSLOduration=2.040439247 podStartE2EDuration="2.040439247s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:01:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:01:43.037915136 +0000 UTC m=+320.591045385" watchObservedRunningTime="2026-07-15 17:01:43.040439247 +0000 UTC m=+320.593569463" Jul 15 17:01:43.065951 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:43.065891 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7bfnv" podStartSLOduration=5.066111874 podStartE2EDuration="7.065867826s" totalImagesPullingTime="1.922376849s" totalInitContainerRuntime="77.379103ms" isStatefulPod=true podCreationTimestamp="2026-07-15 17:01:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:01:43.062970087 +0000 UTC m=+320.616100304" watchObservedRunningTime="2026-07-15 17:01:43.065867826 +0000 UTC m=+320.618998041" Jul 15 17:01:43.193172 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:43.193137 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:45.027633 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:45.027577 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3018848b-f926-43e9-a530-bcb25f059f95","Type":"ContainerStarted","Data":"11e735ac6d139d085d8f9ecd4868ac561a86ed0e306b3e01d18bc4f3c68d45d5"} Jul 15 17:01:45.028029 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:45.027690 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3018848b-f926-43e9-a530-bcb25f059f95","Type":"ContainerStarted","Data":"fa6b1e051dfc138e7aec1ab558e5ba3c1914eeef48acc34277bc4ad48ac7ba0a"} Jul 15 17:01:45.029317 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:45.029290 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6w89l" event={"ID":"56d144de-0dbf-4f44-ad3e-3277889a317c","Type":"ContainerStarted","Data":"57c31101947779bb639b7343c165768ee390c73790db41cd1f857b0fb51c19f4"} Jul 15 17:01:45.029481 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:45.029462 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6w89l" Jul 15 17:01:45.034782 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:45.034764 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6w89l" Jul 15 17:01:45.058296 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:45.058255 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6w89l" podStartSLOduration=1.898955724 podStartE2EDuration="4.058240805s" totalImagesPullingTime="2.159285081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:01:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:01:45.057701202 +0000 UTC m=+322.610831437" watchObservedRunningTime="2026-07-15 17:01:45.058240805 +0000 UTC m=+322.611371058" Jul 15 17:01:46.036047 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:46.036015 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3018848b-f926-43e9-a530-bcb25f059f95","Type":"ContainerStarted","Data":"d88433f10fb04b711c60d084110bf2efe504182d3cb58e68c7735d40fc28ecc9"} Jul 15 17:01:46.036406 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:46.036077 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3018848b-f926-43e9-a530-bcb25f059f95","Type":"ContainerStarted","Data":"b5fb0c906f0b376be165b7703679914048e147aa6e24fdb03185bfb6c2422de2"} Jul 15 17:01:46.036406 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:46.036108 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3018848b-f926-43e9-a530-bcb25f059f95","Type":"ContainerStarted","Data":"d136ce9c181d61ac2d68ca813ddc57da6a4bf8192a48f24e8da2b9b554281e00"} Jul 15 17:01:48.046858 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:48.046814 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3018848b-f926-43e9-a530-bcb25f059f95","Type":"ContainerStarted","Data":"92c591f1ef5d643eaa9ae9fc809f9ffdf2001dd07ab32be7b99d2e72eb6f68c1"} Jul 15 17:01:48.075672 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:48.075612 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.831032145 podStartE2EDuration="11.075577807s" totalImagesPullingTime="6.227657865s" totalInitContainerRuntime="16.887797ms" isStatefulPod=false podCreationTimestamp="2026-07-15 17:01:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:01:48.075278608 +0000 UTC m=+325.628408842" watchObservedRunningTime="2026-07-15 17:01:48.075577807 +0000 UTC m=+325.628708044" Jul 15 17:01:48.339328 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:48.339291 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:48.397609 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:48.397570 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:50.935506 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:50.935464 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5b8564d945-6nq2w" podUID="6a612a93-a727-4552-b21d-da7c2318980e" containerName="console" containerID="cri-o://08058f17d47cf4db7e8b0c6a60ee78d22ab31f841d3a8bc40f381b6a228dc75f" gracePeriod=15 Jul 15 17:01:51.059618 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:51.059561 2578 generic.go:356] "Generic (PLEG): container finished" podID="6a612a93-a727-4552-b21d-da7c2318980e" containerID="08058f17d47cf4db7e8b0c6a60ee78d22ab31f841d3a8bc40f381b6a228dc75f" exitCode=0 Jul 15 17:01:51.059812 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:51.059632 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b8564d945-6nq2w" event={"ID":"6a612a93-a727-4552-b21d-da7c2318980e","Type":"ContainerDied","Data":"08058f17d47cf4db7e8b0c6a60ee78d22ab31f841d3a8bc40f381b6a228dc75f"} Jul 15 17:01:51.225816 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:51.225495 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b8564d945-6nq2w" Jul 15 17:01:51.259797 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:51.259770 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a612a93-a727-4552-b21d-da7c2318980e-service-ca\") pod \"6a612a93-a727-4552-b21d-da7c2318980e\" (UID: \"6a612a93-a727-4552-b21d-da7c2318980e\") " Jul 15 17:01:51.260004 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:51.259817 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a612a93-a727-4552-b21d-da7c2318980e-console-oauth-config\") pod \"6a612a93-a727-4552-b21d-da7c2318980e\" (UID: \"6a612a93-a727-4552-b21d-da7c2318980e\") " Jul 15 17:01:51.260004 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:51.259866 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a612a93-a727-4552-b21d-da7c2318980e-console-serving-cert\") pod \"6a612a93-a727-4552-b21d-da7c2318980e\" (UID: \"6a612a93-a727-4552-b21d-da7c2318980e\") " Jul 15 17:01:51.260004 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:51.259887 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6xr7\" (UniqueName: \"kubernetes.io/projected/6a612a93-a727-4552-b21d-da7c2318980e-kube-api-access-b6xr7\") pod \"6a612a93-a727-4552-b21d-da7c2318980e\" (UID: \"6a612a93-a727-4552-b21d-da7c2318980e\") " Jul 15 17:01:51.260004 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:51.259924 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a612a93-a727-4552-b21d-da7c2318980e-oauth-serving-cert\") pod \"6a612a93-a727-4552-b21d-da7c2318980e\" (UID: \"6a612a93-a727-4552-b21d-da7c2318980e\") " Jul 15 17:01:51.260004 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:51.259954 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a612a93-a727-4552-b21d-da7c2318980e-console-config\") pod \"6a612a93-a727-4552-b21d-da7c2318980e\" (UID: \"6a612a93-a727-4552-b21d-da7c2318980e\") " Jul 15 17:01:51.260263 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:51.260176 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a612a93-a727-4552-b21d-da7c2318980e-service-ca" (OuterVolumeSpecName: "service-ca") pod "6a612a93-a727-4552-b21d-da7c2318980e" (UID: "6a612a93-a727-4552-b21d-da7c2318980e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:01:51.260458 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:51.260433 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a612a93-a727-4552-b21d-da7c2318980e-console-config" (OuterVolumeSpecName: "console-config") pod "6a612a93-a727-4552-b21d-da7c2318980e" (UID: "6a612a93-a727-4552-b21d-da7c2318980e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:01:51.260616 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:51.260447 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a612a93-a727-4552-b21d-da7c2318980e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6a612a93-a727-4552-b21d-da7c2318980e" (UID: "6a612a93-a727-4552-b21d-da7c2318980e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:01:51.262321 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:51.262293 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a612a93-a727-4552-b21d-da7c2318980e-kube-api-access-b6xr7" (OuterVolumeSpecName: "kube-api-access-b6xr7") pod "6a612a93-a727-4552-b21d-da7c2318980e" (UID: "6a612a93-a727-4552-b21d-da7c2318980e"). InnerVolumeSpecName "kube-api-access-b6xr7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:01:51.262454 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:51.262430 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a612a93-a727-4552-b21d-da7c2318980e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6a612a93-a727-4552-b21d-da7c2318980e" (UID: "6a612a93-a727-4552-b21d-da7c2318980e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:01:51.262454 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:51.262445 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a612a93-a727-4552-b21d-da7c2318980e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6a612a93-a727-4552-b21d-da7c2318980e" (UID: "6a612a93-a727-4552-b21d-da7c2318980e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:01:51.361160 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:51.361126 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a612a93-a727-4552-b21d-da7c2318980e-console-oauth-config\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:01:51.361160 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:51.361154 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a612a93-a727-4552-b21d-da7c2318980e-console-serving-cert\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:01:51.361160 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:51.361163 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6xr7\" (UniqueName: \"kubernetes.io/projected/6a612a93-a727-4552-b21d-da7c2318980e-kube-api-access-b6xr7\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:01:51.361394 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:51.361172 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a612a93-a727-4552-b21d-da7c2318980e-oauth-serving-cert\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:01:51.361394 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:51.361182 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a612a93-a727-4552-b21d-da7c2318980e-console-config\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:01:51.361394 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:51.361190 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a612a93-a727-4552-b21d-da7c2318980e-service-ca\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:01:51.977711 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:51.977683 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d8fbcbc47-hwn7m" Jul 15 17:01:51.978125 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:51.977756 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d8fbcbc47-hwn7m" Jul 15 17:01:51.982689 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:51.982666 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d8fbcbc47-hwn7m" Jul 15 17:01:52.064665 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:52.064633 2578 generic.go:356] "Generic (PLEG): container finished" podID="0055caa8-1ee7-4b69-8dba-d1201107d12a" containerID="153f907b053e9d6c40d94dcd1c1b134c5a114ddd3c08e1b7241b04989e104c51" exitCode=0 Jul 15 17:01:52.064833 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:52.064708 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p9wk7" event={"ID":"0055caa8-1ee7-4b69-8dba-d1201107d12a","Type":"ContainerDied","Data":"153f907b053e9d6c40d94dcd1c1b134c5a114ddd3c08e1b7241b04989e104c51"} Jul 15 17:01:52.065059 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:52.065036 2578 scope.go:117] "RemoveContainer" containerID="153f907b053e9d6c40d94dcd1c1b134c5a114ddd3c08e1b7241b04989e104c51" Jul 15 17:01:52.065927 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:52.065906 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b8564d945-6nq2w" event={"ID":"6a612a93-a727-4552-b21d-da7c2318980e","Type":"ContainerDied","Data":"928d24935e162b6a40a219fb42e30789bc881a0d96185f603181944968378e47"} Jul 15 17:01:52.066027 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:52.065926 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b8564d945-6nq2w" Jul 15 17:01:52.066027 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:52.065951 2578 scope.go:117] "RemoveContainer" containerID="08058f17d47cf4db7e8b0c6a60ee78d22ab31f841d3a8bc40f381b6a228dc75f" Jul 15 17:01:52.070719 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:52.070669 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d8fbcbc47-hwn7m" Jul 15 17:01:52.095381 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:52.095351 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b8564d945-6nq2w"] Jul 15 17:01:52.102187 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:52.102159 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5b8564d945-6nq2w"] Jul 15 17:01:52.312647 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:52.312547 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:52.413061 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:52.413031 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:52.993661 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:52.993631 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a612a93-a727-4552-b21d-da7c2318980e" path="/var/lib/kubelet/pods/6a612a93-a727-4552-b21d-da7c2318980e/volumes" Jul 15 17:01:53.070853 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:53.070818 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p9wk7" event={"ID":"0055caa8-1ee7-4b69-8dba-d1201107d12a","Type":"ContainerStarted","Data":"77b565ac565ee2ba8989cc6094f4ccc46e9dc04e84473b4a6f9baaff2175b59a"} Jul 15 17:01:54.511649 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:54.511590 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" podUID="f0d3cec0-475c-481e-beb3-012b1c8c215b" containerName="registry" containerID="cri-o://a390673b12ff075fe4db12cd2dc7e14ebc9901e63b90e558659a3c7b1f03481f" gracePeriod=30 Jul 15 17:01:54.807070 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:54.807042 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 17:01:54.894135 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:54.894103 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f0d3cec0-475c-481e-beb3-012b1c8c215b-installation-pull-secrets\") pod \"f0d3cec0-475c-481e-beb3-012b1c8c215b\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " Jul 15 17:01:54.894135 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:54.894140 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f0d3cec0-475c-481e-beb3-012b1c8c215b-image-registry-private-configuration\") pod \"f0d3cec0-475c-481e-beb3-012b1c8c215b\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " Jul 15 17:01:54.894355 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:54.894168 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wk9j7\" (UniqueName: \"kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-kube-api-access-wk9j7\") pod \"f0d3cec0-475c-481e-beb3-012b1c8c215b\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " Jul 15 17:01:54.894355 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:54.894199 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-bound-sa-token\") pod \"f0d3cec0-475c-481e-beb3-012b1c8c215b\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " Jul 15 17:01:54.894355 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:54.894328 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-tls\") pod \"f0d3cec0-475c-481e-beb3-012b1c8c215b\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " Jul 15 17:01:54.894503 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:54.894375 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f0d3cec0-475c-481e-beb3-012b1c8c215b-trusted-ca\") pod \"f0d3cec0-475c-481e-beb3-012b1c8c215b\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " Jul 15 17:01:54.894503 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:54.894403 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-certificates\") pod \"f0d3cec0-475c-481e-beb3-012b1c8c215b\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " Jul 15 17:01:54.894622 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:54.894518 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f0d3cec0-475c-481e-beb3-012b1c8c215b-ca-trust-extracted\") pod \"f0d3cec0-475c-481e-beb3-012b1c8c215b\" (UID: \"f0d3cec0-475c-481e-beb3-012b1c8c215b\") " Jul 15 17:01:54.895010 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:54.894867 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f0d3cec0-475c-481e-beb3-012b1c8c215b" (UID: "f0d3cec0-475c-481e-beb3-012b1c8c215b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:01:54.895119 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:54.895030 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0d3cec0-475c-481e-beb3-012b1c8c215b-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f0d3cec0-475c-481e-beb3-012b1c8c215b" (UID: "f0d3cec0-475c-481e-beb3-012b1c8c215b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:01:54.896872 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:54.896844 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-kube-api-access-wk9j7" (OuterVolumeSpecName: "kube-api-access-wk9j7") pod "f0d3cec0-475c-481e-beb3-012b1c8c215b" (UID: "f0d3cec0-475c-481e-beb3-012b1c8c215b"). InnerVolumeSpecName "kube-api-access-wk9j7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:01:54.897005 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:54.896951 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0d3cec0-475c-481e-beb3-012b1c8c215b-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f0d3cec0-475c-481e-beb3-012b1c8c215b" (UID: "f0d3cec0-475c-481e-beb3-012b1c8c215b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:01:54.897005 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:54.896990 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f0d3cec0-475c-481e-beb3-012b1c8c215b" (UID: "f0d3cec0-475c-481e-beb3-012b1c8c215b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:01:54.897099 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:54.897003 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f0d3cec0-475c-481e-beb3-012b1c8c215b" (UID: "f0d3cec0-475c-481e-beb3-012b1c8c215b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:01:54.897099 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:54.897013 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0d3cec0-475c-481e-beb3-012b1c8c215b-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f0d3cec0-475c-481e-beb3-012b1c8c215b" (UID: "f0d3cec0-475c-481e-beb3-012b1c8c215b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:01:54.905656 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:54.905633 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0d3cec0-475c-481e-beb3-012b1c8c215b-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f0d3cec0-475c-481e-beb3-012b1c8c215b" (UID: "f0d3cec0-475c-481e-beb3-012b1c8c215b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:01:54.996022 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:54.995992 2578 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f0d3cec0-475c-481e-beb3-012b1c8c215b-ca-trust-extracted\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:01:54.996145 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:54.996021 2578 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f0d3cec0-475c-481e-beb3-012b1c8c215b-installation-pull-secrets\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:01:54.996145 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:54.996048 2578 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f0d3cec0-475c-481e-beb3-012b1c8c215b-image-registry-private-configuration\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:01:54.996145 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:54.996061 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wk9j7\" (UniqueName: \"kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-kube-api-access-wk9j7\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:01:54.996145 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:54.996073 2578 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-bound-sa-token\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:01:54.996145 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:54.996085 2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-tls\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:01:54.996145 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:54.996096 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f0d3cec0-475c-481e-beb3-012b1c8c215b-trusted-ca\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:01:54.996145 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:54.996107 2578 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f0d3cec0-475c-481e-beb3-012b1c8c215b-registry-certificates\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:01:55.080326 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:55.080242 2578 generic.go:356] "Generic (PLEG): container finished" podID="f0d3cec0-475c-481e-beb3-012b1c8c215b" containerID="a390673b12ff075fe4db12cd2dc7e14ebc9901e63b90e558659a3c7b1f03481f" exitCode=0 Jul 15 17:01:55.080326 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:55.080285 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" Jul 15 17:01:55.080326 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:55.080320 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" event={"ID":"f0d3cec0-475c-481e-beb3-012b1c8c215b","Type":"ContainerDied","Data":"a390673b12ff075fe4db12cd2dc7e14ebc9901e63b90e558659a3c7b1f03481f"} Jul 15 17:01:55.080559 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:55.080365 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-795b87c6c7-xztkp" event={"ID":"f0d3cec0-475c-481e-beb3-012b1c8c215b","Type":"ContainerDied","Data":"2ed241ffe59f683e1f3305d59fd38701eca73ee351637ac7e1b72ba95b3f7abe"} Jul 15 17:01:55.080559 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:55.080386 2578 scope.go:117] "RemoveContainer" containerID="a390673b12ff075fe4db12cd2dc7e14ebc9901e63b90e558659a3c7b1f03481f" Jul 15 17:01:55.088942 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:55.088911 2578 scope.go:117] "RemoveContainer" containerID="a390673b12ff075fe4db12cd2dc7e14ebc9901e63b90e558659a3c7b1f03481f" Jul 15 17:01:55.089183 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:01:55.089167 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a390673b12ff075fe4db12cd2dc7e14ebc9901e63b90e558659a3c7b1f03481f\": container with ID starting with a390673b12ff075fe4db12cd2dc7e14ebc9901e63b90e558659a3c7b1f03481f not found: ID does not exist" containerID="a390673b12ff075fe4db12cd2dc7e14ebc9901e63b90e558659a3c7b1f03481f" Jul 15 17:01:55.089228 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:55.089193 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a390673b12ff075fe4db12cd2dc7e14ebc9901e63b90e558659a3c7b1f03481f"} err="failed to get container status \"a390673b12ff075fe4db12cd2dc7e14ebc9901e63b90e558659a3c7b1f03481f\": rpc error: code = NotFound desc = could not find container \"a390673b12ff075fe4db12cd2dc7e14ebc9901e63b90e558659a3c7b1f03481f\": container with ID starting with a390673b12ff075fe4db12cd2dc7e14ebc9901e63b90e558659a3c7b1f03481f not found: ID does not exist" Jul 15 17:01:55.097375 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:55.097354 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-795b87c6c7-xztkp"] Jul 15 17:01:55.101560 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:55.101541 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-795b87c6c7-xztkp"] Jul 15 17:01:56.104772 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:56.104741 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:56.994678 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:56.994640 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0d3cec0-475c-481e-beb3-012b1c8c215b" path="/var/lib/kubelet/pods/f0d3cec0-475c-481e-beb3-012b1c8c215b/volumes" Jul 15 17:01:59.592823 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:01:59.592781 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:02:00.636348 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:02:00.636304 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:02:03.886368 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:02:03.886338 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:02:07.131252 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:02:07.131218 2578 generic.go:356] "Generic (PLEG): container finished" podID="f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5" containerID="89945ec82f35b05fb9c08d69df0c4ee1b7449d63dbc5e791332e2513615f8d13" exitCode=0 Jul 15 17:02:07.131737 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:02:07.131264 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-hthbq" event={"ID":"f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5","Type":"ContainerDied","Data":"89945ec82f35b05fb9c08d69df0c4ee1b7449d63dbc5e791332e2513615f8d13"} Jul 15 17:02:07.131737 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:02:07.131672 2578 scope.go:117] "RemoveContainer" containerID="89945ec82f35b05fb9c08d69df0c4ee1b7449d63dbc5e791332e2513615f8d13" Jul 15 17:02:08.138535 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:02:08.138502 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-hthbq" event={"ID":"f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5","Type":"ContainerStarted","Data":"fc83fdabc56610874cf911a200171b7ddbbbd52156a7db324754241ad2352658"} Jul 15 17:02:11.106924 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:02:11.106884 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:02:11.162011 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:02:11.161983 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:02:35.745643 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:02:35.745586 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-metrics-certs\") pod \"network-metrics-daemon-62hbp\" (UID: \"df1447df-1b9a-4dc3-ba2c-ecbab1514d09\") " pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 17:02:35.747861 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:02:35.747841 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df1447df-1b9a-4dc3-ba2c-ecbab1514d09-metrics-certs\") pod \"network-metrics-daemon-62hbp\" (UID: \"df1447df-1b9a-4dc3-ba2c-ecbab1514d09\") " pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 17:02:35.787639 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:02:35.787614 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pqk7d\"" Jul 15 17:02:35.795878 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:02:35.795860 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-62hbp" Jul 15 17:02:35.923494 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:02:35.923470 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-62hbp"] Jul 15 17:02:35.926038 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:02:35.926012 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf1447df_1b9a_4dc3_ba2c_ecbab1514d09.slice/crio-fe24628cad7f30ff26663bf4f5a4a2206263a4f77935c81f46fb787e2e7b4822 WatchSource:0}: Error finding container fe24628cad7f30ff26663bf4f5a4a2206263a4f77935c81f46fb787e2e7b4822: Status 404 returned error can't find the container with id fe24628cad7f30ff26663bf4f5a4a2206263a4f77935c81f46fb787e2e7b4822 Jul 15 17:02:36.248206 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:02:36.248170 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-62hbp" event={"ID":"df1447df-1b9a-4dc3-ba2c-ecbab1514d09","Type":"ContainerStarted","Data":"fe24628cad7f30ff26663bf4f5a4a2206263a4f77935c81f46fb787e2e7b4822"} Jul 15 17:02:37.254629 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:02:37.254581 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-62hbp" event={"ID":"df1447df-1b9a-4dc3-ba2c-ecbab1514d09","Type":"ContainerStarted","Data":"bfaa0908bc2322db5d64e417ee4a6b4dbb1a12d43f5df7ce9eb9b5a8c07be2c6"} Jul 15 17:02:37.254998 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:02:37.254655 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-62hbp" event={"ID":"df1447df-1b9a-4dc3-ba2c-ecbab1514d09","Type":"ContainerStarted","Data":"38c39405b5fc60e50cc5cb0e231212b71bd099bf133862cd78822e11f983cee3"} Jul 15 17:02:37.272745 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:02:37.272701 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-62hbp" podStartSLOduration=374.302989876 podStartE2EDuration="6m15.272686582s" totalImagesPullingTime="969.696706ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:56:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:02:37.27141274 +0000 UTC m=+374.824542954" watchObservedRunningTime="2026-07-15 17:02:37.272686582 +0000 UTC m=+374.825816848" Jul 15 17:02:37.372956 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:02:37.372928 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:02:37.474451 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:02:37.474421 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:02:38.944240 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:02:38.944200 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:02:39.019519 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:02:39.019490 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:02:39.942669 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:02:39.942634 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:02:40.008373 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:02:40.008340 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:02:40.062778 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:02:40.062750 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:02:56.262734 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:02:56.262701 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:02:56.388320 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:02:56.388288 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:01.389719 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.389680 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6dbdd47cdc-gc2g6"] Jul 15 17:03:01.390670 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.390647 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0d3cec0-475c-481e-beb3-012b1c8c215b" containerName="registry" Jul 15 17:03:01.390791 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.390740 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0d3cec0-475c-481e-beb3-012b1c8c215b" containerName="registry" Jul 15 17:03:01.390791 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.390784 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a612a93-a727-4552-b21d-da7c2318980e" containerName="console" Jul 15 17:03:01.390899 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.390793 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a612a93-a727-4552-b21d-da7c2318980e" containerName="console" Jul 15 17:03:01.390899 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.390882 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0d3cec0-475c-481e-beb3-012b1c8c215b" containerName="registry" Jul 15 17:03:01.390899 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.390893 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a612a93-a727-4552-b21d-da7c2318980e" containerName="console" Jul 15 17:03:01.393007 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.392987 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6dbdd47cdc-gc2g6" Jul 15 17:03:01.395711 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.395688 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 15 17:03:01.395989 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.395800 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 15 17:03:01.397920 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.397898 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 15 17:03:01.398091 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.398074 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-h8mmt\"" Jul 15 17:03:01.398337 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.398313 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 15 17:03:01.398418 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.398313 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 15 17:03:01.400796 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.400776 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 15 17:03:01.407177 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.407157 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6dbdd47cdc-gc2g6"] Jul 15 17:03:01.441756 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.441723 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22c130e6-6358-403d-bdd5-8a722f066d87-metrics-client-ca\") pod \"telemeter-client-6dbdd47cdc-gc2g6\" (UID: \"22c130e6-6358-403d-bdd5-8a722f066d87\") " pod="openshift-monitoring/telemeter-client-6dbdd47cdc-gc2g6" Jul 15 17:03:01.441917 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.441763 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22c130e6-6358-403d-bdd5-8a722f066d87-serving-certs-ca-bundle\") pod \"telemeter-client-6dbdd47cdc-gc2g6\" (UID: \"22c130e6-6358-403d-bdd5-8a722f066d87\") " pod="openshift-monitoring/telemeter-client-6dbdd47cdc-gc2g6" Jul 15 17:03:01.441917 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.441800 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/22c130e6-6358-403d-bdd5-8a722f066d87-telemeter-client-tls\") pod \"telemeter-client-6dbdd47cdc-gc2g6\" (UID: \"22c130e6-6358-403d-bdd5-8a722f066d87\") " pod="openshift-monitoring/telemeter-client-6dbdd47cdc-gc2g6" Jul 15 17:03:01.441917 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.441826 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8q7x\" (UniqueName: \"kubernetes.io/projected/22c130e6-6358-403d-bdd5-8a722f066d87-kube-api-access-j8q7x\") pod \"telemeter-client-6dbdd47cdc-gc2g6\" (UID: \"22c130e6-6358-403d-bdd5-8a722f066d87\") " pod="openshift-monitoring/telemeter-client-6dbdd47cdc-gc2g6" Jul 15 17:03:01.441917 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.441863 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/22c130e6-6358-403d-bdd5-8a722f066d87-secret-telemeter-client\") pod \"telemeter-client-6dbdd47cdc-gc2g6\" (UID: \"22c130e6-6358-403d-bdd5-8a722f066d87\") " pod="openshift-monitoring/telemeter-client-6dbdd47cdc-gc2g6" Jul 15 17:03:01.442047 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.441909 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/22c130e6-6358-403d-bdd5-8a722f066d87-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6dbdd47cdc-gc2g6\" (UID: \"22c130e6-6358-403d-bdd5-8a722f066d87\") " pod="openshift-monitoring/telemeter-client-6dbdd47cdc-gc2g6" Jul 15 17:03:01.442047 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.441945 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/22c130e6-6358-403d-bdd5-8a722f066d87-federate-client-tls\") pod \"telemeter-client-6dbdd47cdc-gc2g6\" (UID: \"22c130e6-6358-403d-bdd5-8a722f066d87\") " pod="openshift-monitoring/telemeter-client-6dbdd47cdc-gc2g6" Jul 15 17:03:01.442047 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.441971 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22c130e6-6358-403d-bdd5-8a722f066d87-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6dbdd47cdc-gc2g6\" (UID: \"22c130e6-6358-403d-bdd5-8a722f066d87\") " pod="openshift-monitoring/telemeter-client-6dbdd47cdc-gc2g6" Jul 15 17:03:01.542682 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.542646 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22c130e6-6358-403d-bdd5-8a722f066d87-metrics-client-ca\") pod \"telemeter-client-6dbdd47cdc-gc2g6\" (UID: \"22c130e6-6358-403d-bdd5-8a722f066d87\") " pod="openshift-monitoring/telemeter-client-6dbdd47cdc-gc2g6" Jul 15 17:03:01.542866 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.542687 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22c130e6-6358-403d-bdd5-8a722f066d87-serving-certs-ca-bundle\") pod \"telemeter-client-6dbdd47cdc-gc2g6\" (UID: \"22c130e6-6358-403d-bdd5-8a722f066d87\") " pod="openshift-monitoring/telemeter-client-6dbdd47cdc-gc2g6" Jul 15 17:03:01.542866 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.542810 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/22c130e6-6358-403d-bdd5-8a722f066d87-telemeter-client-tls\") pod \"telemeter-client-6dbdd47cdc-gc2g6\" (UID: \"22c130e6-6358-403d-bdd5-8a722f066d87\") " pod="openshift-monitoring/telemeter-client-6dbdd47cdc-gc2g6" Jul 15 17:03:01.542866 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.542858 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8q7x\" (UniqueName: \"kubernetes.io/projected/22c130e6-6358-403d-bdd5-8a722f066d87-kube-api-access-j8q7x\") pod \"telemeter-client-6dbdd47cdc-gc2g6\" (UID: \"22c130e6-6358-403d-bdd5-8a722f066d87\") " pod="openshift-monitoring/telemeter-client-6dbdd47cdc-gc2g6" Jul 15 17:03:01.543028 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.542905 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/22c130e6-6358-403d-bdd5-8a722f066d87-secret-telemeter-client\") pod \"telemeter-client-6dbdd47cdc-gc2g6\" (UID: \"22c130e6-6358-403d-bdd5-8a722f066d87\") " pod="openshift-monitoring/telemeter-client-6dbdd47cdc-gc2g6" Jul 15 17:03:01.543028 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.542925 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/22c130e6-6358-403d-bdd5-8a722f066d87-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6dbdd47cdc-gc2g6\" (UID: \"22c130e6-6358-403d-bdd5-8a722f066d87\") " pod="openshift-monitoring/telemeter-client-6dbdd47cdc-gc2g6" Jul 15 17:03:01.543028 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.542952 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/22c130e6-6358-403d-bdd5-8a722f066d87-federate-client-tls\") pod \"telemeter-client-6dbdd47cdc-gc2g6\" (UID: \"22c130e6-6358-403d-bdd5-8a722f066d87\") " pod="openshift-monitoring/telemeter-client-6dbdd47cdc-gc2g6" Jul 15 17:03:01.543028 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.542988 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22c130e6-6358-403d-bdd5-8a722f066d87-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6dbdd47cdc-gc2g6\" (UID: \"22c130e6-6358-403d-bdd5-8a722f066d87\") " pod="openshift-monitoring/telemeter-client-6dbdd47cdc-gc2g6" Jul 15 17:03:01.543455 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.543424 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22c130e6-6358-403d-bdd5-8a722f066d87-serving-certs-ca-bundle\") pod \"telemeter-client-6dbdd47cdc-gc2g6\" (UID: \"22c130e6-6358-403d-bdd5-8a722f066d87\") " pod="openshift-monitoring/telemeter-client-6dbdd47cdc-gc2g6" Jul 15 17:03:01.543578 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.543425 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22c130e6-6358-403d-bdd5-8a722f066d87-metrics-client-ca\") pod \"telemeter-client-6dbdd47cdc-gc2g6\" (UID: \"22c130e6-6358-403d-bdd5-8a722f066d87\") " pod="openshift-monitoring/telemeter-client-6dbdd47cdc-gc2g6" Jul 15 17:03:01.543861 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.543836 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22c130e6-6358-403d-bdd5-8a722f066d87-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6dbdd47cdc-gc2g6\" (UID: \"22c130e6-6358-403d-bdd5-8a722f066d87\") " pod="openshift-monitoring/telemeter-client-6dbdd47cdc-gc2g6" Jul 15 17:03:01.545375 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.545344 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/22c130e6-6358-403d-bdd5-8a722f066d87-telemeter-client-tls\") pod \"telemeter-client-6dbdd47cdc-gc2g6\" (UID: \"22c130e6-6358-403d-bdd5-8a722f066d87\") " pod="openshift-monitoring/telemeter-client-6dbdd47cdc-gc2g6" Jul 15 17:03:01.545552 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.545529 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/22c130e6-6358-403d-bdd5-8a722f066d87-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6dbdd47cdc-gc2g6\" (UID: \"22c130e6-6358-403d-bdd5-8a722f066d87\") " pod="openshift-monitoring/telemeter-client-6dbdd47cdc-gc2g6" Jul 15 17:03:01.545642 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.545582 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/22c130e6-6358-403d-bdd5-8a722f066d87-secret-telemeter-client\") pod \"telemeter-client-6dbdd47cdc-gc2g6\" (UID: \"22c130e6-6358-403d-bdd5-8a722f066d87\") " pod="openshift-monitoring/telemeter-client-6dbdd47cdc-gc2g6" Jul 15 17:03:01.545642 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.545588 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/22c130e6-6358-403d-bdd5-8a722f066d87-federate-client-tls\") pod \"telemeter-client-6dbdd47cdc-gc2g6\" (UID: \"22c130e6-6358-403d-bdd5-8a722f066d87\") " pod="openshift-monitoring/telemeter-client-6dbdd47cdc-gc2g6" Jul 15 17:03:01.551808 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.551788 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8q7x\" (UniqueName: \"kubernetes.io/projected/22c130e6-6358-403d-bdd5-8a722f066d87-kube-api-access-j8q7x\") pod \"telemeter-client-6dbdd47cdc-gc2g6\" (UID: \"22c130e6-6358-403d-bdd5-8a722f066d87\") " pod="openshift-monitoring/telemeter-client-6dbdd47cdc-gc2g6" Jul 15 17:03:01.705033 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.705002 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6dbdd47cdc-gc2g6" Jul 15 17:03:01.836627 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:01.836580 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6dbdd47cdc-gc2g6"] Jul 15 17:03:01.840122 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:03:01.840097 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22c130e6_6358_403d_bdd5_8a722f066d87.slice/crio-71585602678f1ad405dcfcedaf72eb6029b946da3299f9408522777019024bc3 WatchSource:0}: Error finding container 71585602678f1ad405dcfcedaf72eb6029b946da3299f9408522777019024bc3: Status 404 returned error can't find the container with id 71585602678f1ad405dcfcedaf72eb6029b946da3299f9408522777019024bc3 Jul 15 17:03:02.356112 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:02.356075 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6dbdd47cdc-gc2g6" event={"ID":"22c130e6-6358-403d-bdd5-8a722f066d87","Type":"ContainerStarted","Data":"71585602678f1ad405dcfcedaf72eb6029b946da3299f9408522777019024bc3"} Jul 15 17:03:04.365938 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:04.365896 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6dbdd47cdc-gc2g6" event={"ID":"22c130e6-6358-403d-bdd5-8a722f066d87","Type":"ContainerStarted","Data":"8b5899df73cccb8619d8ccd7753f149f8ac058db8ddaa6b45356f15bb443e573"} Jul 15 17:03:04.366271 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:04.365970 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6dbdd47cdc-gc2g6" event={"ID":"22c130e6-6358-403d-bdd5-8a722f066d87","Type":"ContainerStarted","Data":"b8158e262313d03ff739dcd985689d4b9816e69831ca67a7f89002b66e8a47a7"} Jul 15 17:03:04.366271 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:04.365998 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6dbdd47cdc-gc2g6" event={"ID":"22c130e6-6358-403d-bdd5-8a722f066d87","Type":"ContainerStarted","Data":"82530083082f2a5f3539d33e8a36ab0cea6682caf799e05b1bb5baa10e6110a2"} Jul 15 17:03:04.392759 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:04.392713 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6dbdd47cdc-gc2g6" podStartSLOduration=1.426870976 podStartE2EDuration="3.392696184s" totalImagesPullingTime="1.965825208s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:03:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:03:04.391003769 +0000 UTC m=+401.944133997" watchObservedRunningTime="2026-07-15 17:03:04.392696184 +0000 UTC m=+401.945826398" Jul 15 17:03:05.201538 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:05.201509 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77b4455dc-xdbnk"] Jul 15 17:03:05.203764 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:05.203749 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b4455dc-xdbnk" Jul 15 17:03:05.221712 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:05.221686 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77b4455dc-xdbnk"] Jul 15 17:03:05.272664 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:05.272634 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b8daacc-3b47-4357-adba-747884156a9e-console-serving-cert\") pod \"console-77b4455dc-xdbnk\" (UID: \"1b8daacc-3b47-4357-adba-747884156a9e\") " pod="openshift-console/console-77b4455dc-xdbnk" Jul 15 17:03:05.272664 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:05.272666 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b8daacc-3b47-4357-adba-747884156a9e-oauth-serving-cert\") pod \"console-77b4455dc-xdbnk\" (UID: \"1b8daacc-3b47-4357-adba-747884156a9e\") " pod="openshift-console/console-77b4455dc-xdbnk" Jul 15 17:03:05.272837 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:05.272730 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b8daacc-3b47-4357-adba-747884156a9e-console-config\") pod \"console-77b4455dc-xdbnk\" (UID: \"1b8daacc-3b47-4357-adba-747884156a9e\") " pod="openshift-console/console-77b4455dc-xdbnk" Jul 15 17:03:05.272837 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:05.272770 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jc9sc\" (UniqueName: \"kubernetes.io/projected/1b8daacc-3b47-4357-adba-747884156a9e-kube-api-access-jc9sc\") pod \"console-77b4455dc-xdbnk\" (UID: \"1b8daacc-3b47-4357-adba-747884156a9e\") " pod="openshift-console/console-77b4455dc-xdbnk" Jul 15 17:03:05.272906 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:05.272828 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b8daacc-3b47-4357-adba-747884156a9e-service-ca\") pod \"console-77b4455dc-xdbnk\" (UID: \"1b8daacc-3b47-4357-adba-747884156a9e\") " pod="openshift-console/console-77b4455dc-xdbnk" Jul 15 17:03:05.272906 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:05.272867 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b8daacc-3b47-4357-adba-747884156a9e-trusted-ca-bundle\") pod \"console-77b4455dc-xdbnk\" (UID: \"1b8daacc-3b47-4357-adba-747884156a9e\") " pod="openshift-console/console-77b4455dc-xdbnk" Jul 15 17:03:05.272906 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:05.272884 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b8daacc-3b47-4357-adba-747884156a9e-console-oauth-config\") pod \"console-77b4455dc-xdbnk\" (UID: \"1b8daacc-3b47-4357-adba-747884156a9e\") " pod="openshift-console/console-77b4455dc-xdbnk" Jul 15 17:03:05.275472 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:05.275451 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:05.336673 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:05.336648 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:05.374990 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:05.373710 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b8daacc-3b47-4357-adba-747884156a9e-console-config\") pod \"console-77b4455dc-xdbnk\" (UID: \"1b8daacc-3b47-4357-adba-747884156a9e\") " pod="openshift-console/console-77b4455dc-xdbnk" Jul 15 17:03:05.374990 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:05.373755 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jc9sc\" (UniqueName: \"kubernetes.io/projected/1b8daacc-3b47-4357-adba-747884156a9e-kube-api-access-jc9sc\") pod \"console-77b4455dc-xdbnk\" (UID: \"1b8daacc-3b47-4357-adba-747884156a9e\") " pod="openshift-console/console-77b4455dc-xdbnk" Jul 15 17:03:05.374990 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:05.373800 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b8daacc-3b47-4357-adba-747884156a9e-service-ca\") pod \"console-77b4455dc-xdbnk\" (UID: \"1b8daacc-3b47-4357-adba-747884156a9e\") " pod="openshift-console/console-77b4455dc-xdbnk" Jul 15 17:03:05.374990 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:05.373824 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b8daacc-3b47-4357-adba-747884156a9e-trusted-ca-bundle\") pod \"console-77b4455dc-xdbnk\" (UID: \"1b8daacc-3b47-4357-adba-747884156a9e\") " pod="openshift-console/console-77b4455dc-xdbnk" Jul 15 17:03:05.374990 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:05.373848 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b8daacc-3b47-4357-adba-747884156a9e-console-oauth-config\") pod \"console-77b4455dc-xdbnk\" (UID: \"1b8daacc-3b47-4357-adba-747884156a9e\") " pod="openshift-console/console-77b4455dc-xdbnk" Jul 15 17:03:05.374990 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:05.373894 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b8daacc-3b47-4357-adba-747884156a9e-console-serving-cert\") pod \"console-77b4455dc-xdbnk\" (UID: \"1b8daacc-3b47-4357-adba-747884156a9e\") " pod="openshift-console/console-77b4455dc-xdbnk" Jul 15 17:03:05.374990 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:05.373915 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b8daacc-3b47-4357-adba-747884156a9e-oauth-serving-cert\") pod \"console-77b4455dc-xdbnk\" (UID: \"1b8daacc-3b47-4357-adba-747884156a9e\") " pod="openshift-console/console-77b4455dc-xdbnk" Jul 15 17:03:05.374990 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:05.374528 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b8daacc-3b47-4357-adba-747884156a9e-console-config\") pod \"console-77b4455dc-xdbnk\" (UID: \"1b8daacc-3b47-4357-adba-747884156a9e\") " pod="openshift-console/console-77b4455dc-xdbnk" Jul 15 17:03:05.374990 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:05.374574 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b8daacc-3b47-4357-adba-747884156a9e-oauth-serving-cert\") pod \"console-77b4455dc-xdbnk\" (UID: \"1b8daacc-3b47-4357-adba-747884156a9e\") " pod="openshift-console/console-77b4455dc-xdbnk" Jul 15 17:03:05.375755 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:05.375292 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b8daacc-3b47-4357-adba-747884156a9e-trusted-ca-bundle\") pod \"console-77b4455dc-xdbnk\" (UID: \"1b8daacc-3b47-4357-adba-747884156a9e\") " pod="openshift-console/console-77b4455dc-xdbnk" Jul 15 17:03:05.376198 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:05.376168 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b8daacc-3b47-4357-adba-747884156a9e-service-ca\") pod \"console-77b4455dc-xdbnk\" (UID: \"1b8daacc-3b47-4357-adba-747884156a9e\") " pod="openshift-console/console-77b4455dc-xdbnk" Jul 15 17:03:05.377315 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:05.377291 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b8daacc-3b47-4357-adba-747884156a9e-console-oauth-config\") pod \"console-77b4455dc-xdbnk\" (UID: \"1b8daacc-3b47-4357-adba-747884156a9e\") " pod="openshift-console/console-77b4455dc-xdbnk" Jul 15 17:03:05.377943 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:05.377924 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b8daacc-3b47-4357-adba-747884156a9e-console-serving-cert\") pod \"console-77b4455dc-xdbnk\" (UID: \"1b8daacc-3b47-4357-adba-747884156a9e\") " pod="openshift-console/console-77b4455dc-xdbnk" Jul 15 17:03:05.385194 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:05.385172 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jc9sc\" (UniqueName: \"kubernetes.io/projected/1b8daacc-3b47-4357-adba-747884156a9e-kube-api-access-jc9sc\") pod \"console-77b4455dc-xdbnk\" (UID: \"1b8daacc-3b47-4357-adba-747884156a9e\") " pod="openshift-console/console-77b4455dc-xdbnk" Jul 15 17:03:05.406416 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:05.406389 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:05.468889 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:05.468783 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:05.513273 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:05.513243 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b4455dc-xdbnk" Jul 15 17:03:05.653621 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:05.653481 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77b4455dc-xdbnk"] Jul 15 17:03:05.657076 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:03:05.657042 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b8daacc_3b47_4357_adba_747884156a9e.slice/crio-20a72fc3b2060fe5f0f7a8cf43e0ead2c931a9629e46ccc8c2aa06112ce01c82 WatchSource:0}: Error finding container 20a72fc3b2060fe5f0f7a8cf43e0ead2c931a9629e46ccc8c2aa06112ce01c82: Status 404 returned error can't find the container with id 20a72fc3b2060fe5f0f7a8cf43e0ead2c931a9629e46ccc8c2aa06112ce01c82 Jul 15 17:03:06.009266 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:06.009171 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:06.334113 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:06.334014 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:06.378835 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:06.378798 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b4455dc-xdbnk" event={"ID":"1b8daacc-3b47-4357-adba-747884156a9e","Type":"ContainerStarted","Data":"27a8aa92a5dd48f1e447f02955d782f019e73df48fd0dfe91d0c1b0970723f05"} Jul 15 17:03:06.379208 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:06.378880 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b4455dc-xdbnk" event={"ID":"1b8daacc-3b47-4357-adba-747884156a9e","Type":"ContainerStarted","Data":"20a72fc3b2060fe5f0f7a8cf43e0ead2c931a9629e46ccc8c2aa06112ce01c82"} Jul 15 17:03:06.397360 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:06.397307 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77b4455dc-xdbnk" podStartSLOduration=1.397293653 podStartE2EDuration="1.397293653s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:03:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:03:06.395102608 +0000 UTC m=+403.948232840" watchObservedRunningTime="2026-07-15 17:03:06.397293653 +0000 UTC m=+403.950423867" Jul 15 17:03:08.819316 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:08.819282 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:11.499676 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:11.499640 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:11.563040 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:11.563010 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:12.932516 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:12.932489 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77b4455dc-xdbnk"] Jul 15 17:03:15.514316 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:15.514276 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77b4455dc-xdbnk" Jul 15 17:03:23.433230 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:23.433191 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:23.498275 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:23.498243 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:24.028430 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:24.028399 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d8fbcbc47-hwn7m"] Jul 15 17:03:24.128845 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:24.128812 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:24.237708 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:24.237672 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:24.303208 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:24.303120 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:26.496672 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:26.496640 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:26.597086 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:26.597053 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:37.953230 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:37.953168 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-77b4455dc-xdbnk" podUID="1b8daacc-3b47-4357-adba-747884156a9e" containerName="console" containerID="cri-o://27a8aa92a5dd48f1e447f02955d782f019e73df48fd0dfe91d0c1b0970723f05" gracePeriod=15 Jul 15 17:03:38.202273 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:38.202249 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b4455dc-xdbnk" Jul 15 17:03:38.253364 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:38.253293 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b8daacc-3b47-4357-adba-747884156a9e-trusted-ca-bundle\") pod \"1b8daacc-3b47-4357-adba-747884156a9e\" (UID: \"1b8daacc-3b47-4357-adba-747884156a9e\") " Jul 15 17:03:38.253364 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:38.253327 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b8daacc-3b47-4357-adba-747884156a9e-console-config\") pod \"1b8daacc-3b47-4357-adba-747884156a9e\" (UID: \"1b8daacc-3b47-4357-adba-747884156a9e\") " Jul 15 17:03:38.253544 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:38.253368 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jc9sc\" (UniqueName: \"kubernetes.io/projected/1b8daacc-3b47-4357-adba-747884156a9e-kube-api-access-jc9sc\") pod \"1b8daacc-3b47-4357-adba-747884156a9e\" (UID: \"1b8daacc-3b47-4357-adba-747884156a9e\") " Jul 15 17:03:38.253544 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:38.253414 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b8daacc-3b47-4357-adba-747884156a9e-console-oauth-config\") pod \"1b8daacc-3b47-4357-adba-747884156a9e\" (UID: \"1b8daacc-3b47-4357-adba-747884156a9e\") " Jul 15 17:03:38.253544 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:38.253456 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b8daacc-3b47-4357-adba-747884156a9e-oauth-serving-cert\") pod \"1b8daacc-3b47-4357-adba-747884156a9e\" (UID: \"1b8daacc-3b47-4357-adba-747884156a9e\") " Jul 15 17:03:38.253544 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:38.253492 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b8daacc-3b47-4357-adba-747884156a9e-console-serving-cert\") pod \"1b8daacc-3b47-4357-adba-747884156a9e\" (UID: \"1b8daacc-3b47-4357-adba-747884156a9e\") " Jul 15 17:03:38.253544 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:38.253533 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b8daacc-3b47-4357-adba-747884156a9e-service-ca\") pod \"1b8daacc-3b47-4357-adba-747884156a9e\" (UID: \"1b8daacc-3b47-4357-adba-747884156a9e\") " Jul 15 17:03:38.253821 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:38.253739 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b8daacc-3b47-4357-adba-747884156a9e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1b8daacc-3b47-4357-adba-747884156a9e" (UID: "1b8daacc-3b47-4357-adba-747884156a9e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:03:38.253882 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:38.253821 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b8daacc-3b47-4357-adba-747884156a9e-console-config" (OuterVolumeSpecName: "console-config") pod "1b8daacc-3b47-4357-adba-747884156a9e" (UID: "1b8daacc-3b47-4357-adba-747884156a9e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:03:38.253882 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:38.253831 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b8daacc-3b47-4357-adba-747884156a9e-trusted-ca-bundle\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:03:38.254042 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:38.254020 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b8daacc-3b47-4357-adba-747884156a9e-service-ca" (OuterVolumeSpecName: "service-ca") pod "1b8daacc-3b47-4357-adba-747884156a9e" (UID: "1b8daacc-3b47-4357-adba-747884156a9e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:03:38.254108 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:38.254088 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b8daacc-3b47-4357-adba-747884156a9e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1b8daacc-3b47-4357-adba-747884156a9e" (UID: "1b8daacc-3b47-4357-adba-747884156a9e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:03:38.255626 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:38.255568 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b8daacc-3b47-4357-adba-747884156a9e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1b8daacc-3b47-4357-adba-747884156a9e" (UID: "1b8daacc-3b47-4357-adba-747884156a9e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:03:38.255735 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:38.255659 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b8daacc-3b47-4357-adba-747884156a9e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1b8daacc-3b47-4357-adba-747884156a9e" (UID: "1b8daacc-3b47-4357-adba-747884156a9e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:03:38.255813 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:38.255788 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b8daacc-3b47-4357-adba-747884156a9e-kube-api-access-jc9sc" (OuterVolumeSpecName: "kube-api-access-jc9sc") pod "1b8daacc-3b47-4357-adba-747884156a9e" (UID: "1b8daacc-3b47-4357-adba-747884156a9e"). InnerVolumeSpecName "kube-api-access-jc9sc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:03:38.355143 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:38.355107 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b8daacc-3b47-4357-adba-747884156a9e-console-config\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:03:38.355143 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:38.355135 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jc9sc\" (UniqueName: \"kubernetes.io/projected/1b8daacc-3b47-4357-adba-747884156a9e-kube-api-access-jc9sc\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:03:38.355143 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:38.355145 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b8daacc-3b47-4357-adba-747884156a9e-console-oauth-config\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:03:38.355143 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:38.355154 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b8daacc-3b47-4357-adba-747884156a9e-oauth-serving-cert\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:03:38.355396 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:38.355162 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b8daacc-3b47-4357-adba-747884156a9e-console-serving-cert\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:03:38.355396 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:38.355171 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b8daacc-3b47-4357-adba-747884156a9e-service-ca\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:03:38.495609 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:38.495573 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:38.502936 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:38.502912 2578 generic.go:356] "Generic (PLEG): container finished" podID="1b8daacc-3b47-4357-adba-747884156a9e" containerID="27a8aa92a5dd48f1e447f02955d782f019e73df48fd0dfe91d0c1b0970723f05" exitCode=0 Jul 15 17:03:38.503067 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:38.502957 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b4455dc-xdbnk" Jul 15 17:03:38.503067 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:38.502995 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b4455dc-xdbnk" event={"ID":"1b8daacc-3b47-4357-adba-747884156a9e","Type":"ContainerDied","Data":"27a8aa92a5dd48f1e447f02955d782f019e73df48fd0dfe91d0c1b0970723f05"} Jul 15 17:03:38.503067 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:38.503044 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b4455dc-xdbnk" event={"ID":"1b8daacc-3b47-4357-adba-747884156a9e","Type":"ContainerDied","Data":"20a72fc3b2060fe5f0f7a8cf43e0ead2c931a9629e46ccc8c2aa06112ce01c82"} Jul 15 17:03:38.503177 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:38.503070 2578 scope.go:117] "RemoveContainer" containerID="27a8aa92a5dd48f1e447f02955d782f019e73df48fd0dfe91d0c1b0970723f05" Jul 15 17:03:38.511557 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:38.511538 2578 scope.go:117] "RemoveContainer" containerID="27a8aa92a5dd48f1e447f02955d782f019e73df48fd0dfe91d0c1b0970723f05" Jul 15 17:03:38.511828 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:03:38.511807 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27a8aa92a5dd48f1e447f02955d782f019e73df48fd0dfe91d0c1b0970723f05\": container with ID starting with 27a8aa92a5dd48f1e447f02955d782f019e73df48fd0dfe91d0c1b0970723f05 not found: ID does not exist" containerID="27a8aa92a5dd48f1e447f02955d782f019e73df48fd0dfe91d0c1b0970723f05" Jul 15 17:03:38.511902 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:38.511839 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27a8aa92a5dd48f1e447f02955d782f019e73df48fd0dfe91d0c1b0970723f05"} err="failed to get container status \"27a8aa92a5dd48f1e447f02955d782f019e73df48fd0dfe91d0c1b0970723f05\": rpc error: code = NotFound desc = could not find container \"27a8aa92a5dd48f1e447f02955d782f019e73df48fd0dfe91d0c1b0970723f05\": container with ID starting with 27a8aa92a5dd48f1e447f02955d782f019e73df48fd0dfe91d0c1b0970723f05 not found: ID does not exist" Jul 15 17:03:38.525865 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:38.525841 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77b4455dc-xdbnk"] Jul 15 17:03:38.532102 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:38.532082 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77b4455dc-xdbnk"] Jul 15 17:03:38.595919 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:38.595893 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:38.989690 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:38.989660 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b8daacc-3b47-4357-adba-747884156a9e" path="/var/lib/kubelet/pods/1b8daacc-3b47-4357-adba-747884156a9e/volumes" Jul 15 17:03:49.054727 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:49.054680 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5d8fbcbc47-hwn7m" podUID="63edfb69-4852-4663-a6d7-35b77581b129" containerName="console" containerID="cri-o://e029307ee23180a05f4f23db7cbf59632b0f37b610d1d461fc26bc308da2df51" gracePeriod=15 Jul 15 17:03:49.303178 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:49.303155 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d8fbcbc47-hwn7m" Jul 15 17:03:49.336031 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:49.335966 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/63edfb69-4852-4663-a6d7-35b77581b129-service-ca\") pod \"63edfb69-4852-4663-a6d7-35b77581b129\" (UID: \"63edfb69-4852-4663-a6d7-35b77581b129\") " Jul 15 17:03:49.336031 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:49.336017 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/63edfb69-4852-4663-a6d7-35b77581b129-console-oauth-config\") pod \"63edfb69-4852-4663-a6d7-35b77581b129\" (UID: \"63edfb69-4852-4663-a6d7-35b77581b129\") " Jul 15 17:03:49.336225 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:49.336045 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mt6x2\" (UniqueName: \"kubernetes.io/projected/63edfb69-4852-4663-a6d7-35b77581b129-kube-api-access-mt6x2\") pod \"63edfb69-4852-4663-a6d7-35b77581b129\" (UID: \"63edfb69-4852-4663-a6d7-35b77581b129\") " Jul 15 17:03:49.336338 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:49.336215 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63edfb69-4852-4663-a6d7-35b77581b129-trusted-ca-bundle\") pod \"63edfb69-4852-4663-a6d7-35b77581b129\" (UID: \"63edfb69-4852-4663-a6d7-35b77581b129\") " Jul 15 17:03:49.336338 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:49.336269 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/63edfb69-4852-4663-a6d7-35b77581b129-console-config\") pod \"63edfb69-4852-4663-a6d7-35b77581b129\" (UID: \"63edfb69-4852-4663-a6d7-35b77581b129\") " Jul 15 17:03:49.336338 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:49.336317 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/63edfb69-4852-4663-a6d7-35b77581b129-oauth-serving-cert\") pod \"63edfb69-4852-4663-a6d7-35b77581b129\" (UID: \"63edfb69-4852-4663-a6d7-35b77581b129\") " Jul 15 17:03:49.336499 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:49.336366 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/63edfb69-4852-4663-a6d7-35b77581b129-console-serving-cert\") pod \"63edfb69-4852-4663-a6d7-35b77581b129\" (UID: \"63edfb69-4852-4663-a6d7-35b77581b129\") " Jul 15 17:03:49.336499 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:49.336368 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63edfb69-4852-4663-a6d7-35b77581b129-service-ca" (OuterVolumeSpecName: "service-ca") pod "63edfb69-4852-4663-a6d7-35b77581b129" (UID: "63edfb69-4852-4663-a6d7-35b77581b129"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:03:49.336614 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:49.336585 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/63edfb69-4852-4663-a6d7-35b77581b129-service-ca\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:03:49.336771 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:49.336747 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63edfb69-4852-4663-a6d7-35b77581b129-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "63edfb69-4852-4663-a6d7-35b77581b129" (UID: "63edfb69-4852-4663-a6d7-35b77581b129"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:03:49.336771 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:49.336755 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63edfb69-4852-4663-a6d7-35b77581b129-console-config" (OuterVolumeSpecName: "console-config") pod "63edfb69-4852-4663-a6d7-35b77581b129" (UID: "63edfb69-4852-4663-a6d7-35b77581b129"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:03:49.336951 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:49.336929 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63edfb69-4852-4663-a6d7-35b77581b129-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "63edfb69-4852-4663-a6d7-35b77581b129" (UID: "63edfb69-4852-4663-a6d7-35b77581b129"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:03:49.338101 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:49.338076 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63edfb69-4852-4663-a6d7-35b77581b129-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "63edfb69-4852-4663-a6d7-35b77581b129" (UID: "63edfb69-4852-4663-a6d7-35b77581b129"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:03:49.338210 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:49.338146 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63edfb69-4852-4663-a6d7-35b77581b129-kube-api-access-mt6x2" (OuterVolumeSpecName: "kube-api-access-mt6x2") pod "63edfb69-4852-4663-a6d7-35b77581b129" (UID: "63edfb69-4852-4663-a6d7-35b77581b129"). InnerVolumeSpecName "kube-api-access-mt6x2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:03:49.338275 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:49.338220 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63edfb69-4852-4663-a6d7-35b77581b129-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "63edfb69-4852-4663-a6d7-35b77581b129" (UID: "63edfb69-4852-4663-a6d7-35b77581b129"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:03:49.437888 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:49.437853 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/63edfb69-4852-4663-a6d7-35b77581b129-console-serving-cert\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:03:49.437888 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:49.437882 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/63edfb69-4852-4663-a6d7-35b77581b129-console-oauth-config\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:03:49.437888 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:49.437893 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mt6x2\" (UniqueName: \"kubernetes.io/projected/63edfb69-4852-4663-a6d7-35b77581b129-kube-api-access-mt6x2\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:03:49.438097 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:49.437902 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63edfb69-4852-4663-a6d7-35b77581b129-trusted-ca-bundle\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:03:49.438097 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:49.437912 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/63edfb69-4852-4663-a6d7-35b77581b129-console-config\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:03:49.438097 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:49.437920 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/63edfb69-4852-4663-a6d7-35b77581b129-oauth-serving-cert\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:03:49.549248 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:49.549212 2578 generic.go:356] "Generic (PLEG): container finished" podID="63edfb69-4852-4663-a6d7-35b77581b129" containerID="e029307ee23180a05f4f23db7cbf59632b0f37b610d1d461fc26bc308da2df51" exitCode=0 Jul 15 17:03:49.549426 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:49.549256 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d8fbcbc47-hwn7m" Jul 15 17:03:49.549426 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:49.549299 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d8fbcbc47-hwn7m" event={"ID":"63edfb69-4852-4663-a6d7-35b77581b129","Type":"ContainerDied","Data":"e029307ee23180a05f4f23db7cbf59632b0f37b610d1d461fc26bc308da2df51"} Jul 15 17:03:49.549426 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:49.549358 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d8fbcbc47-hwn7m" event={"ID":"63edfb69-4852-4663-a6d7-35b77581b129","Type":"ContainerDied","Data":"170f6648890833a37fc986b9a5480bcb6e0073f4daaf39386d6a1c384984e6e3"} Jul 15 17:03:49.549426 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:49.549385 2578 scope.go:117] "RemoveContainer" containerID="e029307ee23180a05f4f23db7cbf59632b0f37b610d1d461fc26bc308da2df51" Jul 15 17:03:49.558025 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:49.558008 2578 scope.go:117] "RemoveContainer" containerID="e029307ee23180a05f4f23db7cbf59632b0f37b610d1d461fc26bc308da2df51" Jul 15 17:03:49.558285 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:03:49.558267 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e029307ee23180a05f4f23db7cbf59632b0f37b610d1d461fc26bc308da2df51\": container with ID starting with e029307ee23180a05f4f23db7cbf59632b0f37b610d1d461fc26bc308da2df51 not found: ID does not exist" containerID="e029307ee23180a05f4f23db7cbf59632b0f37b610d1d461fc26bc308da2df51" Jul 15 17:03:49.558327 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:49.558294 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e029307ee23180a05f4f23db7cbf59632b0f37b610d1d461fc26bc308da2df51"} err="failed to get container status \"e029307ee23180a05f4f23db7cbf59632b0f37b610d1d461fc26bc308da2df51\": rpc error: code = NotFound desc = could not find container \"e029307ee23180a05f4f23db7cbf59632b0f37b610d1d461fc26bc308da2df51\": container with ID starting with e029307ee23180a05f4f23db7cbf59632b0f37b610d1d461fc26bc308da2df51 not found: ID does not exist" Jul 15 17:03:49.570420 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:49.570400 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d8fbcbc47-hwn7m"] Jul 15 17:03:49.574169 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:49.574151 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5d8fbcbc47-hwn7m"] Jul 15 17:03:50.989567 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:03:50.989535 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63edfb69-4852-4663-a6d7-35b77581b129" path="/var/lib/kubelet/pods/63edfb69-4852-4663-a6d7-35b77581b129/volumes" Jul 15 17:04:46.458895 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:04:46.458859 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:04:46.558720 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:04:46.558685 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:04:47.178827 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:04:47.178794 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqx77"] Jul 15 17:04:47.179110 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:04:47.179099 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b8daacc-3b47-4357-adba-747884156a9e" containerName="console" Jul 15 17:04:47.179154 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:04:47.179112 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b8daacc-3b47-4357-adba-747884156a9e" containerName="console" Jul 15 17:04:47.179154 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:04:47.179125 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63edfb69-4852-4663-a6d7-35b77581b129" containerName="console" Jul 15 17:04:47.179154 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:04:47.179130 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="63edfb69-4852-4663-a6d7-35b77581b129" containerName="console" Jul 15 17:04:47.179245 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:04:47.179175 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b8daacc-3b47-4357-adba-747884156a9e" containerName="console" Jul 15 17:04:47.179245 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:04:47.179182 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="63edfb69-4852-4663-a6d7-35b77581b129" containerName="console" Jul 15 17:04:47.182088 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:04:47.182072 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqx77" Jul 15 17:04:47.185927 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:04:47.185904 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 15 17:04:47.185927 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:04:47.185919 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 15 17:04:47.186058 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:04:47.185904 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-f58s2\"" Jul 15 17:04:47.192433 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:04:47.192413 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqx77"] Jul 15 17:04:47.282214 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:04:47.282175 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e38a175-fe29-455e-9b17-4dc64a89d482-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqx77\" (UID: \"7e38a175-fe29-455e-9b17-4dc64a89d482\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqx77" Jul 15 17:04:47.282214 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:04:47.282214 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e38a175-fe29-455e-9b17-4dc64a89d482-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqx77\" (UID: \"7e38a175-fe29-455e-9b17-4dc64a89d482\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqx77" Jul 15 17:04:47.282430 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:04:47.282246 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lfcr\" (UniqueName: \"kubernetes.io/projected/7e38a175-fe29-455e-9b17-4dc64a89d482-kube-api-access-2lfcr\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqx77\" (UID: \"7e38a175-fe29-455e-9b17-4dc64a89d482\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqx77" Jul 15 17:04:47.382856 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:04:47.382824 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e38a175-fe29-455e-9b17-4dc64a89d482-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqx77\" (UID: \"7e38a175-fe29-455e-9b17-4dc64a89d482\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqx77" Jul 15 17:04:47.383060 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:04:47.382864 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e38a175-fe29-455e-9b17-4dc64a89d482-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqx77\" (UID: \"7e38a175-fe29-455e-9b17-4dc64a89d482\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqx77" Jul 15 17:04:47.383060 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:04:47.382892 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lfcr\" (UniqueName: \"kubernetes.io/projected/7e38a175-fe29-455e-9b17-4dc64a89d482-kube-api-access-2lfcr\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqx77\" (UID: \"7e38a175-fe29-455e-9b17-4dc64a89d482\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqx77" Jul 15 17:04:47.383202 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:04:47.383183 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e38a175-fe29-455e-9b17-4dc64a89d482-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqx77\" (UID: \"7e38a175-fe29-455e-9b17-4dc64a89d482\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqx77" Jul 15 17:04:47.383278 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:04:47.383257 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e38a175-fe29-455e-9b17-4dc64a89d482-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqx77\" (UID: \"7e38a175-fe29-455e-9b17-4dc64a89d482\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqx77" Jul 15 17:04:47.391775 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:04:47.391745 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lfcr\" (UniqueName: \"kubernetes.io/projected/7e38a175-fe29-455e-9b17-4dc64a89d482-kube-api-access-2lfcr\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqx77\" (UID: \"7e38a175-fe29-455e-9b17-4dc64a89d482\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqx77" Jul 15 17:04:47.491162 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:04:47.491081 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqx77" Jul 15 17:04:47.616382 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:04:47.616348 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqx77"] Jul 15 17:04:47.619149 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:04:47.619115 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e38a175_fe29_455e_9b17_4dc64a89d482.slice/crio-bce2ada1288df3acdbe66d6538f7ecf7c773b34d4254c3912d9b4bb7e129650c WatchSource:0}: Error finding container bce2ada1288df3acdbe66d6538f7ecf7c773b34d4254c3912d9b4bb7e129650c: Status 404 returned error can't find the container with id bce2ada1288df3acdbe66d6538f7ecf7c773b34d4254c3912d9b4bb7e129650c Jul 15 17:04:47.772214 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:04:47.772129 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqx77" event={"ID":"7e38a175-fe29-455e-9b17-4dc64a89d482","Type":"ContainerStarted","Data":"bce2ada1288df3acdbe66d6538f7ecf7c773b34d4254c3912d9b4bb7e129650c"} Jul 15 17:04:55.806048 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:04:55.806014 2578 generic.go:356] "Generic (PLEG): container finished" podID="7e38a175-fe29-455e-9b17-4dc64a89d482" containerID="42efa09aae3fab908f36a8d4fb3603434a864ef4f69205f2135ec0c4c8acfe4c" exitCode=0 Jul 15 17:04:55.806410 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:04:55.806069 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqx77" event={"ID":"7e38a175-fe29-455e-9b17-4dc64a89d482","Type":"ContainerDied","Data":"42efa09aae3fab908f36a8d4fb3603434a864ef4f69205f2135ec0c4c8acfe4c"} Jul 15 17:04:58.820343 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:04:58.820310 2578 generic.go:356] "Generic (PLEG): container finished" podID="7e38a175-fe29-455e-9b17-4dc64a89d482" containerID="087276c1aec47225adfea6b2bb7f5910f420b0e9fe7cd044864b804a69041791" exitCode=0 Jul 15 17:04:58.820798 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:04:58.820401 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqx77" event={"ID":"7e38a175-fe29-455e-9b17-4dc64a89d482","Type":"ContainerDied","Data":"087276c1aec47225adfea6b2bb7f5910f420b0e9fe7cd044864b804a69041791"} Jul 15 17:05:06.857057 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:06.857018 2578 generic.go:356] "Generic (PLEG): container finished" podID="7e38a175-fe29-455e-9b17-4dc64a89d482" containerID="b05ceb6f2e15de7e407a89419012a434332abe2df177243223bbed0e55423689" exitCode=0 Jul 15 17:05:06.857427 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:06.857079 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqx77" event={"ID":"7e38a175-fe29-455e-9b17-4dc64a89d482","Type":"ContainerDied","Data":"b05ceb6f2e15de7e407a89419012a434332abe2df177243223bbed0e55423689"} Jul 15 17:05:07.930437 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:07.930397 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:07.983088 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:07.983066 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqx77" Jul 15 17:05:08.047930 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:08.047900 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2lfcr\" (UniqueName: \"kubernetes.io/projected/7e38a175-fe29-455e-9b17-4dc64a89d482-kube-api-access-2lfcr\") pod \"7e38a175-fe29-455e-9b17-4dc64a89d482\" (UID: \"7e38a175-fe29-455e-9b17-4dc64a89d482\") " Jul 15 17:05:08.047930 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:08.047940 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e38a175-fe29-455e-9b17-4dc64a89d482-bundle\") pod \"7e38a175-fe29-455e-9b17-4dc64a89d482\" (UID: \"7e38a175-fe29-455e-9b17-4dc64a89d482\") " Jul 15 17:05:08.048171 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:08.048073 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e38a175-fe29-455e-9b17-4dc64a89d482-util\") pod \"7e38a175-fe29-455e-9b17-4dc64a89d482\" (UID: \"7e38a175-fe29-455e-9b17-4dc64a89d482\") " Jul 15 17:05:08.048647 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:08.048625 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e38a175-fe29-455e-9b17-4dc64a89d482-bundle" (OuterVolumeSpecName: "bundle") pod "7e38a175-fe29-455e-9b17-4dc64a89d482" (UID: "7e38a175-fe29-455e-9b17-4dc64a89d482"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:05:08.050086 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:08.050063 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e38a175-fe29-455e-9b17-4dc64a89d482-kube-api-access-2lfcr" (OuterVolumeSpecName: "kube-api-access-2lfcr") pod "7e38a175-fe29-455e-9b17-4dc64a89d482" (UID: "7e38a175-fe29-455e-9b17-4dc64a89d482"). InnerVolumeSpecName "kube-api-access-2lfcr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:05:08.052323 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:08.052303 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e38a175-fe29-455e-9b17-4dc64a89d482-util" (OuterVolumeSpecName: "util") pod "7e38a175-fe29-455e-9b17-4dc64a89d482" (UID: "7e38a175-fe29-455e-9b17-4dc64a89d482"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:05:08.149121 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:08.149037 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2lfcr\" (UniqueName: \"kubernetes.io/projected/7e38a175-fe29-455e-9b17-4dc64a89d482-kube-api-access-2lfcr\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:05:08.149121 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:08.149065 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e38a175-fe29-455e-9b17-4dc64a89d482-bundle\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:05:08.149121 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:08.149077 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e38a175-fe29-455e-9b17-4dc64a89d482-util\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:05:08.865975 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:08.865944 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqx77" event={"ID":"7e38a175-fe29-455e-9b17-4dc64a89d482","Type":"ContainerDied","Data":"bce2ada1288df3acdbe66d6538f7ecf7c773b34d4254c3912d9b4bb7e129650c"} Jul 15 17:05:08.866170 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:08.865989 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bce2ada1288df3acdbe66d6538f7ecf7c773b34d4254c3912d9b4bb7e129650c" Jul 15 17:05:08.866170 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:08.865998 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nqx77" Jul 15 17:05:13.242452 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:13.242422 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:14.204823 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:14.204794 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:15.323996 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:15.323965 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-89t8h"] Jul 15 17:05:15.324354 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:15.324255 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e38a175-fe29-455e-9b17-4dc64a89d482" containerName="pull" Jul 15 17:05:15.324354 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:15.324266 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e38a175-fe29-455e-9b17-4dc64a89d482" containerName="pull" Jul 15 17:05:15.324354 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:15.324277 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e38a175-fe29-455e-9b17-4dc64a89d482" containerName="extract" Jul 15 17:05:15.324354 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:15.324282 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e38a175-fe29-455e-9b17-4dc64a89d482" containerName="extract" Jul 15 17:05:15.324354 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:15.324298 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e38a175-fe29-455e-9b17-4dc64a89d482" containerName="util" Jul 15 17:05:15.324354 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:15.324305 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e38a175-fe29-455e-9b17-4dc64a89d482" containerName="util" Jul 15 17:05:15.324543 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:15.324406 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e38a175-fe29-455e-9b17-4dc64a89d482" containerName="extract" Jul 15 17:05:15.375356 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:15.375323 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-89t8h"] Jul 15 17:05:15.375503 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:15.375426 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-89t8h" Jul 15 17:05:15.378831 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:15.378811 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 15 17:05:15.379025 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:15.379009 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-7mtq6\"" Jul 15 17:05:15.379088 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:15.379036 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 15 17:05:15.509140 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:15.509110 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a227b1a5-7a12-4e8e-ab36-85683ac43832-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-89t8h\" (UID: \"a227b1a5-7a12-4e8e-ab36-85683ac43832\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-89t8h" Jul 15 17:05:15.509291 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:15.509150 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4lms\" (UniqueName: \"kubernetes.io/projected/a227b1a5-7a12-4e8e-ab36-85683ac43832-kube-api-access-h4lms\") pod \"cert-manager-operator-controller-manager-55c444bf86-89t8h\" (UID: \"a227b1a5-7a12-4e8e-ab36-85683ac43832\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-89t8h" Jul 15 17:05:15.610487 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:15.610404 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a227b1a5-7a12-4e8e-ab36-85683ac43832-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-89t8h\" (UID: \"a227b1a5-7a12-4e8e-ab36-85683ac43832\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-89t8h" Jul 15 17:05:15.610487 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:15.610456 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4lms\" (UniqueName: \"kubernetes.io/projected/a227b1a5-7a12-4e8e-ab36-85683ac43832-kube-api-access-h4lms\") pod \"cert-manager-operator-controller-manager-55c444bf86-89t8h\" (UID: \"a227b1a5-7a12-4e8e-ab36-85683ac43832\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-89t8h" Jul 15 17:05:15.610889 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:15.610868 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a227b1a5-7a12-4e8e-ab36-85683ac43832-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-89t8h\" (UID: \"a227b1a5-7a12-4e8e-ab36-85683ac43832\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-89t8h" Jul 15 17:05:15.622188 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:15.622161 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4lms\" (UniqueName: \"kubernetes.io/projected/a227b1a5-7a12-4e8e-ab36-85683ac43832-kube-api-access-h4lms\") pod \"cert-manager-operator-controller-manager-55c444bf86-89t8h\" (UID: \"a227b1a5-7a12-4e8e-ab36-85683ac43832\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-89t8h" Jul 15 17:05:15.684155 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:15.684118 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-89t8h" Jul 15 17:05:15.847051 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:15.847020 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-89t8h"] Jul 15 17:05:15.850517 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:05:15.850490 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda227b1a5_7a12_4e8e_ab36_85683ac43832.slice/crio-7d93d367cbfc5b76f586b06b5b11bba3bebcfc2572aa563dc25c91bf84f06f63 WatchSource:0}: Error finding container 7d93d367cbfc5b76f586b06b5b11bba3bebcfc2572aa563dc25c91bf84f06f63: Status 404 returned error can't find the container with id 7d93d367cbfc5b76f586b06b5b11bba3bebcfc2572aa563dc25c91bf84f06f63 Jul 15 17:05:15.894177 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:15.894118 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-89t8h" event={"ID":"a227b1a5-7a12-4e8e-ab36-85683ac43832","Type":"ContainerStarted","Data":"7d93d367cbfc5b76f586b06b5b11bba3bebcfc2572aa563dc25c91bf84f06f63"} Jul 15 17:05:16.893179 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:16.893148 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:16.991056 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:16.991024 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:17.906203 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:17.906169 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-89t8h" event={"ID":"a227b1a5-7a12-4e8e-ab36-85683ac43832","Type":"ContainerStarted","Data":"bad7e5a38b3c8e92db486534a933bb09f8f4a0c1a1c76f0df9511f38e0171182"} Jul 15 17:05:17.931505 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:17.931461 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-89t8h" podStartSLOduration=1.064380335 podStartE2EDuration="2.931447549s" totalImagesPullingTime="1.867067214s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:05:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:05:17.930931427 +0000 UTC m=+535.484061640" watchObservedRunningTime="2026-07-15 17:05:17.931447549 +0000 UTC m=+535.484577766" Jul 15 17:05:18.280149 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:18.280116 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:18.382467 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:18.382436 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:18.498751 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:18.498721 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:19.468003 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:19.467969 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-dhnbl"] Jul 15 17:05:19.470159 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:19.470143 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-dhnbl" Jul 15 17:05:19.474159 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:19.474138 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-72wsc\"" Jul 15 17:05:19.474239 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:19.474142 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 15 17:05:19.474761 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:19.474747 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 15 17:05:19.487349 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:19.487324 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-dhnbl"] Jul 15 17:05:19.546140 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:19.546104 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7758d18a-f470-4371-b896-bc2f33178fe8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-dhnbl\" (UID: \"7758d18a-f470-4371-b896-bc2f33178fe8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dhnbl" Jul 15 17:05:19.546313 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:19.546159 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7qsm\" (UniqueName: \"kubernetes.io/projected/7758d18a-f470-4371-b896-bc2f33178fe8-kube-api-access-m7qsm\") pod \"cert-manager-webhook-6486ffd997-dhnbl\" (UID: \"7758d18a-f470-4371-b896-bc2f33178fe8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dhnbl" Jul 15 17:05:19.647080 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:19.647047 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7qsm\" (UniqueName: \"kubernetes.io/projected/7758d18a-f470-4371-b896-bc2f33178fe8-kube-api-access-m7qsm\") pod \"cert-manager-webhook-6486ffd997-dhnbl\" (UID: \"7758d18a-f470-4371-b896-bc2f33178fe8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dhnbl" Jul 15 17:05:19.647237 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:19.647131 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7758d18a-f470-4371-b896-bc2f33178fe8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-dhnbl\" (UID: \"7758d18a-f470-4371-b896-bc2f33178fe8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dhnbl" Jul 15 17:05:19.662565 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:19.662537 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7qsm\" (UniqueName: \"kubernetes.io/projected/7758d18a-f470-4371-b896-bc2f33178fe8-kube-api-access-m7qsm\") pod \"cert-manager-webhook-6486ffd997-dhnbl\" (UID: \"7758d18a-f470-4371-b896-bc2f33178fe8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dhnbl" Jul 15 17:05:19.673258 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:19.673237 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7758d18a-f470-4371-b896-bc2f33178fe8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-dhnbl\" (UID: \"7758d18a-f470-4371-b896-bc2f33178fe8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dhnbl" Jul 15 17:05:19.790665 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:19.790583 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdzfm"] Jul 15 17:05:19.791999 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:19.791980 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-dhnbl" Jul 15 17:05:19.793529 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:19.793506 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdzfm" Jul 15 17:05:19.796150 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:19.796129 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-f58s2\"" Jul 15 17:05:19.796666 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:19.796649 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 15 17:05:19.796757 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:19.796742 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 15 17:05:19.807402 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:19.807379 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdzfm"] Jul 15 17:05:19.848793 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:19.848760 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmcmj\" (UniqueName: \"kubernetes.io/projected/89bf2e68-5864-4f60-8faa-5a0648d59371-kube-api-access-jmcmj\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdzfm\" (UID: \"89bf2e68-5864-4f60-8faa-5a0648d59371\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdzfm" Jul 15 17:05:19.849029 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:19.849011 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/89bf2e68-5864-4f60-8faa-5a0648d59371-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdzfm\" (UID: \"89bf2e68-5864-4f60-8faa-5a0648d59371\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdzfm" Jul 15 17:05:19.849152 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:19.849138 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/89bf2e68-5864-4f60-8faa-5a0648d59371-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdzfm\" (UID: \"89bf2e68-5864-4f60-8faa-5a0648d59371\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdzfm" Jul 15 17:05:19.924477 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:19.924421 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-dhnbl"] Jul 15 17:05:19.927286 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:05:19.927264 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7758d18a_f470_4371_b896_bc2f33178fe8.slice/crio-051e0a0e2643b68a6f3d735a0009b13f2595550f8150fe004747acd5ec74b59f WatchSource:0}: Error finding container 051e0a0e2643b68a6f3d735a0009b13f2595550f8150fe004747acd5ec74b59f: Status 404 returned error can't find the container with id 051e0a0e2643b68a6f3d735a0009b13f2595550f8150fe004747acd5ec74b59f Jul 15 17:05:19.949951 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:19.949929 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/89bf2e68-5864-4f60-8faa-5a0648d59371-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdzfm\" (UID: \"89bf2e68-5864-4f60-8faa-5a0648d59371\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdzfm" Jul 15 17:05:19.950050 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:19.949962 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/89bf2e68-5864-4f60-8faa-5a0648d59371-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdzfm\" (UID: \"89bf2e68-5864-4f60-8faa-5a0648d59371\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdzfm" Jul 15 17:05:19.950050 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:19.949999 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmcmj\" (UniqueName: \"kubernetes.io/projected/89bf2e68-5864-4f60-8faa-5a0648d59371-kube-api-access-jmcmj\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdzfm\" (UID: \"89bf2e68-5864-4f60-8faa-5a0648d59371\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdzfm" Jul 15 17:05:19.950274 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:19.950258 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/89bf2e68-5864-4f60-8faa-5a0648d59371-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdzfm\" (UID: \"89bf2e68-5864-4f60-8faa-5a0648d59371\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdzfm" Jul 15 17:05:19.950340 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:19.950321 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/89bf2e68-5864-4f60-8faa-5a0648d59371-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdzfm\" (UID: \"89bf2e68-5864-4f60-8faa-5a0648d59371\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdzfm" Jul 15 17:05:19.960498 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:19.960479 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmcmj\" (UniqueName: \"kubernetes.io/projected/89bf2e68-5864-4f60-8faa-5a0648d59371-kube-api-access-jmcmj\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdzfm\" (UID: \"89bf2e68-5864-4f60-8faa-5a0648d59371\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdzfm" Jul 15 17:05:20.118148 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:20.118077 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdzfm" Jul 15 17:05:20.262271 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:20.262240 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdzfm"] Jul 15 17:05:20.265254 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:05:20.265227 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89bf2e68_5864_4f60_8faa_5a0648d59371.slice/crio-da6c9f7704878e316a1b97486b79db7beb85c9eeea34f7c5d41403afa80faa87 WatchSource:0}: Error finding container da6c9f7704878e316a1b97486b79db7beb85c9eeea34f7c5d41403afa80faa87: Status 404 returned error can't find the container with id da6c9f7704878e316a1b97486b79db7beb85c9eeea34f7c5d41403afa80faa87 Jul 15 17:05:20.719468 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:20.719439 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:20.870466 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:20.870432 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-f8rzh"] Jul 15 17:05:20.873206 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:20.873185 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f8rzh" Jul 15 17:05:20.887082 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:20.887055 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-f8rzh"] Jul 15 17:05:20.917782 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:20.917755 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:20.919469 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:20.919446 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-dhnbl" event={"ID":"7758d18a-f470-4371-b896-bc2f33178fe8","Type":"ContainerStarted","Data":"051e0a0e2643b68a6f3d735a0009b13f2595550f8150fe004747acd5ec74b59f"} Jul 15 17:05:20.920781 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:20.920760 2578 generic.go:356] "Generic (PLEG): container finished" podID="89bf2e68-5864-4f60-8faa-5a0648d59371" containerID="ec206f57538b7f05a7d9d8d95f6d460bead954907d8e56561899f904ab27c9e6" exitCode=0 Jul 15 17:05:20.920877 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:20.920825 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdzfm" event={"ID":"89bf2e68-5864-4f60-8faa-5a0648d59371","Type":"ContainerDied","Data":"ec206f57538b7f05a7d9d8d95f6d460bead954907d8e56561899f904ab27c9e6"} Jul 15 17:05:20.920877 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:20.920856 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdzfm" event={"ID":"89bf2e68-5864-4f60-8faa-5a0648d59371","Type":"ContainerStarted","Data":"da6c9f7704878e316a1b97486b79db7beb85c9eeea34f7c5d41403afa80faa87"} Jul 15 17:05:20.957925 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:20.957896 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2d89c6c4-7c6e-4e5b-a063-763879cbcf9d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-f8rzh\" (UID: \"2d89c6c4-7c6e-4e5b-a063-763879cbcf9d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f8rzh" Jul 15 17:05:20.958022 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:20.958001 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xdsz\" (UniqueName: \"kubernetes.io/projected/2d89c6c4-7c6e-4e5b-a063-763879cbcf9d-kube-api-access-2xdsz\") pod \"cert-manager-cainjector-559cdd8c87-f8rzh\" (UID: \"2d89c6c4-7c6e-4e5b-a063-763879cbcf9d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f8rzh" Jul 15 17:05:21.058632 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:21.058536 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xdsz\" (UniqueName: \"kubernetes.io/projected/2d89c6c4-7c6e-4e5b-a063-763879cbcf9d-kube-api-access-2xdsz\") pod \"cert-manager-cainjector-559cdd8c87-f8rzh\" (UID: \"2d89c6c4-7c6e-4e5b-a063-763879cbcf9d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f8rzh" Jul 15 17:05:21.058632 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:21.058571 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2d89c6c4-7c6e-4e5b-a063-763879cbcf9d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-f8rzh\" (UID: \"2d89c6c4-7c6e-4e5b-a063-763879cbcf9d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f8rzh" Jul 15 17:05:21.068732 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:21.068708 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2d89c6c4-7c6e-4e5b-a063-763879cbcf9d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-f8rzh\" (UID: \"2d89c6c4-7c6e-4e5b-a063-763879cbcf9d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f8rzh" Jul 15 17:05:21.068892 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:21.068874 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xdsz\" (UniqueName: \"kubernetes.io/projected/2d89c6c4-7c6e-4e5b-a063-763879cbcf9d-kube-api-access-2xdsz\") pod \"cert-manager-cainjector-559cdd8c87-f8rzh\" (UID: \"2d89c6c4-7c6e-4e5b-a063-763879cbcf9d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f8rzh" Jul 15 17:05:21.183194 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:21.183161 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f8rzh" Jul 15 17:05:21.309480 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:21.309453 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-f8rzh"] Jul 15 17:05:21.311216 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:05:21.311187 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2d89c6c4_7c6e_4e5b_a063_763879cbcf9d.slice/crio-5d9031840053b7430409885c06a486be6f30ec1559a892a24100afcbfb6cb637 WatchSource:0}: Error finding container 5d9031840053b7430409885c06a486be6f30ec1559a892a24100afcbfb6cb637: Status 404 returned error can't find the container with id 5d9031840053b7430409885c06a486be6f30ec1559a892a24100afcbfb6cb637 Jul 15 17:05:21.926366 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:21.926320 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f8rzh" event={"ID":"2d89c6c4-7c6e-4e5b-a063-763879cbcf9d","Type":"ContainerStarted","Data":"5d9031840053b7430409885c06a486be6f30ec1559a892a24100afcbfb6cb637"} Jul 15 17:05:23.651464 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:23.651431 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:23.720935 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:23.720899 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:24.943249 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:24.943212 2578 generic.go:356] "Generic (PLEG): container finished" podID="89bf2e68-5864-4f60-8faa-5a0648d59371" containerID="b782de4702a811e7689d7a171f1f8a1e6331bb6b24a5da4204a3985b3d20c817" exitCode=0 Jul 15 17:05:24.943717 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:24.943293 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdzfm" event={"ID":"89bf2e68-5864-4f60-8faa-5a0648d59371","Type":"ContainerDied","Data":"b782de4702a811e7689d7a171f1f8a1e6331bb6b24a5da4204a3985b3d20c817"} Jul 15 17:05:24.944901 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:24.944875 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-dhnbl" event={"ID":"7758d18a-f470-4371-b896-bc2f33178fe8","Type":"ContainerStarted","Data":"2602682708d1b8c490f4f4501ca5f094fc087da162312fa33bee69a9440ba8ec"} Jul 15 17:05:24.945020 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:24.944915 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-dhnbl" Jul 15 17:05:24.946371 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:24.946350 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f8rzh" event={"ID":"2d89c6c4-7c6e-4e5b-a063-763879cbcf9d","Type":"ContainerStarted","Data":"afbca60a42506946cd66854413044c5b84bf6a0c26b8a15a728848fe49e51979"} Jul 15 17:05:24.979331 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:24.979270 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-dhnbl" podStartSLOduration=1.525502392 podStartE2EDuration="5.979255924s" totalImagesPullingTime="4.453753532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:05:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:05:24.978366399 +0000 UTC m=+542.531496614" watchObservedRunningTime="2026-07-15 17:05:24.979255924 +0000 UTC m=+542.532386137" Jul 15 17:05:24.994763 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:24.994712 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f8rzh" podStartSLOduration=1.925312029 podStartE2EDuration="4.994697181s" totalImagesPullingTime="3.069385152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:05:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:05:24.993534325 +0000 UTC m=+542.546664542" watchObservedRunningTime="2026-07-15 17:05:24.994697181 +0000 UTC m=+542.547827395" Jul 15 17:05:25.952806 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:25.952775 2578 generic.go:356] "Generic (PLEG): container finished" podID="89bf2e68-5864-4f60-8faa-5a0648d59371" containerID="d0efc68a1642b392a32f9ed6bdbd1bc6cc0824a1dd0157b3a44c8bcb03702305" exitCode=0 Jul 15 17:05:25.953166 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:25.952864 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdzfm" event={"ID":"89bf2e68-5864-4f60-8faa-5a0648d59371","Type":"ContainerDied","Data":"d0efc68a1642b392a32f9ed6bdbd1bc6cc0824a1dd0157b3a44c8bcb03702305"} Jul 15 17:05:27.095395 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:27.095370 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdzfm" Jul 15 17:05:27.214298 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:27.214212 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmcmj\" (UniqueName: \"kubernetes.io/projected/89bf2e68-5864-4f60-8faa-5a0648d59371-kube-api-access-jmcmj\") pod \"89bf2e68-5864-4f60-8faa-5a0648d59371\" (UID: \"89bf2e68-5864-4f60-8faa-5a0648d59371\") " Jul 15 17:05:27.214298 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:27.214286 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/89bf2e68-5864-4f60-8faa-5a0648d59371-bundle\") pod \"89bf2e68-5864-4f60-8faa-5a0648d59371\" (UID: \"89bf2e68-5864-4f60-8faa-5a0648d59371\") " Jul 15 17:05:27.214540 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:27.214319 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/89bf2e68-5864-4f60-8faa-5a0648d59371-util\") pod \"89bf2e68-5864-4f60-8faa-5a0648d59371\" (UID: \"89bf2e68-5864-4f60-8faa-5a0648d59371\") " Jul 15 17:05:27.214720 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:27.214686 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89bf2e68-5864-4f60-8faa-5a0648d59371-bundle" (OuterVolumeSpecName: "bundle") pod "89bf2e68-5864-4f60-8faa-5a0648d59371" (UID: "89bf2e68-5864-4f60-8faa-5a0648d59371"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:05:27.216408 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:27.216384 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89bf2e68-5864-4f60-8faa-5a0648d59371-kube-api-access-jmcmj" (OuterVolumeSpecName: "kube-api-access-jmcmj") pod "89bf2e68-5864-4f60-8faa-5a0648d59371" (UID: "89bf2e68-5864-4f60-8faa-5a0648d59371"). InnerVolumeSpecName "kube-api-access-jmcmj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:05:27.218742 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:27.218723 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89bf2e68-5864-4f60-8faa-5a0648d59371-util" (OuterVolumeSpecName: "util") pod "89bf2e68-5864-4f60-8faa-5a0648d59371" (UID: "89bf2e68-5864-4f60-8faa-5a0648d59371"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:05:27.315167 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:27.315134 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/89bf2e68-5864-4f60-8faa-5a0648d59371-bundle\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:05:27.315167 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:27.315159 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/89bf2e68-5864-4f60-8faa-5a0648d59371-util\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:05:27.315167 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:27.315170 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmcmj\" (UniqueName: \"kubernetes.io/projected/89bf2e68-5864-4f60-8faa-5a0648d59371-kube-api-access-jmcmj\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:05:27.946445 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:27.946413 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:27.965008 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:27.964981 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdzfm" event={"ID":"89bf2e68-5864-4f60-8faa-5a0648d59371","Type":"ContainerDied","Data":"da6c9f7704878e316a1b97486b79db7beb85c9eeea34f7c5d41403afa80faa87"} Jul 15 17:05:27.965157 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:27.965023 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="da6c9f7704878e316a1b97486b79db7beb85c9eeea34f7c5d41403afa80faa87" Jul 15 17:05:27.965157 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:27.965022 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdzfm" Jul 15 17:05:28.931273 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:28.931235 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-qxwgp"] Jul 15 17:05:28.931639 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:28.931585 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89bf2e68-5864-4f60-8faa-5a0648d59371" containerName="pull" Jul 15 17:05:28.931639 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:28.931610 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="89bf2e68-5864-4f60-8faa-5a0648d59371" containerName="pull" Jul 15 17:05:28.931639 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:28.931622 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89bf2e68-5864-4f60-8faa-5a0648d59371" containerName="util" Jul 15 17:05:28.931639 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:28.931627 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="89bf2e68-5864-4f60-8faa-5a0648d59371" containerName="util" Jul 15 17:05:28.931777 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:28.931641 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89bf2e68-5864-4f60-8faa-5a0648d59371" containerName="extract" Jul 15 17:05:28.931777 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:28.931647 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="89bf2e68-5864-4f60-8faa-5a0648d59371" containerName="extract" Jul 15 17:05:28.931777 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:28.931701 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="89bf2e68-5864-4f60-8faa-5a0648d59371" containerName="extract" Jul 15 17:05:28.935466 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:28.935451 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-qxwgp" Jul 15 17:05:28.939661 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:28.939633 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-dnmrz\"" Jul 15 17:05:28.949728 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:28.949707 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-qxwgp"] Jul 15 17:05:29.030583 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:29.030553 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvvcm\" (UniqueName: \"kubernetes.io/projected/c884110c-51e7-43d6-97ea-ab46119151d8-kube-api-access-fvvcm\") pod \"cert-manager-55b4d954d5-qxwgp\" (UID: \"c884110c-51e7-43d6-97ea-ab46119151d8\") " pod="cert-manager/cert-manager-55b4d954d5-qxwgp" Jul 15 17:05:29.030762 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:29.030639 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c884110c-51e7-43d6-97ea-ab46119151d8-bound-sa-token\") pod \"cert-manager-55b4d954d5-qxwgp\" (UID: \"c884110c-51e7-43d6-97ea-ab46119151d8\") " pod="cert-manager/cert-manager-55b4d954d5-qxwgp" Jul 15 17:05:29.131667 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:29.131634 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvvcm\" (UniqueName: \"kubernetes.io/projected/c884110c-51e7-43d6-97ea-ab46119151d8-kube-api-access-fvvcm\") pod \"cert-manager-55b4d954d5-qxwgp\" (UID: \"c884110c-51e7-43d6-97ea-ab46119151d8\") " pod="cert-manager/cert-manager-55b4d954d5-qxwgp" Jul 15 17:05:29.131843 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:29.131699 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c884110c-51e7-43d6-97ea-ab46119151d8-bound-sa-token\") pod \"cert-manager-55b4d954d5-qxwgp\" (UID: \"c884110c-51e7-43d6-97ea-ab46119151d8\") " pod="cert-manager/cert-manager-55b4d954d5-qxwgp" Jul 15 17:05:29.145925 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:29.145903 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvvcm\" (UniqueName: \"kubernetes.io/projected/c884110c-51e7-43d6-97ea-ab46119151d8-kube-api-access-fvvcm\") pod \"cert-manager-55b4d954d5-qxwgp\" (UID: \"c884110c-51e7-43d6-97ea-ab46119151d8\") " pod="cert-manager/cert-manager-55b4d954d5-qxwgp" Jul 15 17:05:29.154421 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:29.154394 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c884110c-51e7-43d6-97ea-ab46119151d8-bound-sa-token\") pod \"cert-manager-55b4d954d5-qxwgp\" (UID: \"c884110c-51e7-43d6-97ea-ab46119151d8\") " pod="cert-manager/cert-manager-55b4d954d5-qxwgp" Jul 15 17:05:29.244927 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:29.244849 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-qxwgp" Jul 15 17:05:29.377478 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:29.377452 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-qxwgp"] Jul 15 17:05:29.379873 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:05:29.379848 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc884110c_51e7_43d6_97ea_ab46119151d8.slice/crio-2b31607a41586515673cd6352d2f56a053fbfb88c4db771fc0ef1e0a0efb1e86 WatchSource:0}: Error finding container 2b31607a41586515673cd6352d2f56a053fbfb88c4db771fc0ef1e0a0efb1e86: Status 404 returned error can't find the container with id 2b31607a41586515673cd6352d2f56a053fbfb88c4db771fc0ef1e0a0efb1e86 Jul 15 17:05:29.974817 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:29.974778 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-qxwgp" event={"ID":"c884110c-51e7-43d6-97ea-ab46119151d8","Type":"ContainerStarted","Data":"d6f42b963fee4657f56a3955034ae2daf608f0463d1747135324839cb46bdf6d"} Jul 15 17:05:29.974817 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:29.974822 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-qxwgp" event={"ID":"c884110c-51e7-43d6-97ea-ab46119151d8","Type":"ContainerStarted","Data":"2b31607a41586515673cd6352d2f56a053fbfb88c4db771fc0ef1e0a0efb1e86"} Jul 15 17:05:29.994040 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:29.993995 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-qxwgp" podStartSLOduration=1.9939797000000001 podStartE2EDuration="1.9939797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:05:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:05:29.992934533 +0000 UTC m=+547.546064760" watchObservedRunningTime="2026-07-15 17:05:29.9939797 +0000 UTC m=+547.547109914" Jul 15 17:05:30.955386 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:30.955353 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-dhnbl" Jul 15 17:05:30.982723 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:30.982691 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:31.076762 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:31.076729 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:36.161795 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:36.161757 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:36.264537 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:36.264507 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:38.243537 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:38.243503 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxt8pb"] Jul 15 17:05:38.245927 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:38.245911 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxt8pb" Jul 15 17:05:38.248737 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:38.248719 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 15 17:05:38.249572 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:38.249556 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 15 17:05:38.249639 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:38.249628 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-f58s2\"" Jul 15 17:05:38.255074 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:38.255049 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxt8pb"] Jul 15 17:05:38.383966 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:38.383932 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:38.404754 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:38.404728 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kfw4\" (UniqueName: \"kubernetes.io/projected/16b0c58b-5312-4d2f-a584-341a852f58e7-kube-api-access-2kfw4\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxt8pb\" (UID: \"16b0c58b-5312-4d2f-a584-341a852f58e7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxt8pb" Jul 15 17:05:38.404880 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:38.404767 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/16b0c58b-5312-4d2f-a584-341a852f58e7-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxt8pb\" (UID: \"16b0c58b-5312-4d2f-a584-341a852f58e7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxt8pb" Jul 15 17:05:38.404880 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:38.404857 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/16b0c58b-5312-4d2f-a584-341a852f58e7-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxt8pb\" (UID: \"16b0c58b-5312-4d2f-a584-341a852f58e7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxt8pb" Jul 15 17:05:38.506225 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:38.506128 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kfw4\" (UniqueName: \"kubernetes.io/projected/16b0c58b-5312-4d2f-a584-341a852f58e7-kube-api-access-2kfw4\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxt8pb\" (UID: \"16b0c58b-5312-4d2f-a584-341a852f58e7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxt8pb" Jul 15 17:05:38.506225 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:38.506187 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/16b0c58b-5312-4d2f-a584-341a852f58e7-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxt8pb\" (UID: \"16b0c58b-5312-4d2f-a584-341a852f58e7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxt8pb" Jul 15 17:05:38.506445 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:38.506250 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/16b0c58b-5312-4d2f-a584-341a852f58e7-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxt8pb\" (UID: \"16b0c58b-5312-4d2f-a584-341a852f58e7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxt8pb" Jul 15 17:05:38.506635 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:38.506585 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/16b0c58b-5312-4d2f-a584-341a852f58e7-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxt8pb\" (UID: \"16b0c58b-5312-4d2f-a584-341a852f58e7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxt8pb" Jul 15 17:05:38.506704 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:38.506648 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/16b0c58b-5312-4d2f-a584-341a852f58e7-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxt8pb\" (UID: \"16b0c58b-5312-4d2f-a584-341a852f58e7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxt8pb" Jul 15 17:05:38.515514 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:38.515490 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kfw4\" (UniqueName: \"kubernetes.io/projected/16b0c58b-5312-4d2f-a584-341a852f58e7-kube-api-access-2kfw4\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxt8pb\" (UID: \"16b0c58b-5312-4d2f-a584-341a852f58e7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxt8pb" Jul 15 17:05:38.556018 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:38.555989 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxt8pb" Jul 15 17:05:38.682848 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:38.682821 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxt8pb"] Jul 15 17:05:38.684749 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:05:38.684720 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16b0c58b_5312_4d2f_a584_341a852f58e7.slice/crio-1d1db4d7c57647466a3a500a41c2f8d3e97181abe1f6c48ecc39149a3aa7e4ef WatchSource:0}: Error finding container 1d1db4d7c57647466a3a500a41c2f8d3e97181abe1f6c48ecc39149a3aa7e4ef: Status 404 returned error can't find the container with id 1d1db4d7c57647466a3a500a41c2f8d3e97181abe1f6c48ecc39149a3aa7e4ef Jul 15 17:05:39.017744 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:39.017709 2578 generic.go:356] "Generic (PLEG): container finished" podID="16b0c58b-5312-4d2f-a584-341a852f58e7" containerID="277b1585475624ad780112dc7ea6be269cde7ffaf355e36893ce9bed65b12b6b" exitCode=0 Jul 15 17:05:39.018030 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:39.017795 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxt8pb" event={"ID":"16b0c58b-5312-4d2f-a584-341a852f58e7","Type":"ContainerDied","Data":"277b1585475624ad780112dc7ea6be269cde7ffaf355e36893ce9bed65b12b6b"} Jul 15 17:05:39.018030 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:39.017842 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxt8pb" event={"ID":"16b0c58b-5312-4d2f-a584-341a852f58e7","Type":"ContainerStarted","Data":"1d1db4d7c57647466a3a500a41c2f8d3e97181abe1f6c48ecc39149a3aa7e4ef"} Jul 15 17:05:40.024288 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:40.024255 2578 generic.go:356] "Generic (PLEG): container finished" podID="16b0c58b-5312-4d2f-a584-341a852f58e7" containerID="12ed5b369ffb15f776dc97aa81c55b9dab5f55759cab3adae95b1eef4efca600" exitCode=0 Jul 15 17:05:40.024703 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:40.024330 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxt8pb" event={"ID":"16b0c58b-5312-4d2f-a584-341a852f58e7","Type":"ContainerDied","Data":"12ed5b369ffb15f776dc97aa81c55b9dab5f55759cab3adae95b1eef4efca600"} Jul 15 17:05:41.030589 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:41.030547 2578 generic.go:356] "Generic (PLEG): container finished" podID="16b0c58b-5312-4d2f-a584-341a852f58e7" containerID="e3a944c6a5e5496a67a4ee9bed7c3d2c99032ab3e4d7ce89ab9db313c729f25a" exitCode=0 Jul 15 17:05:41.031016 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:41.030613 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxt8pb" event={"ID":"16b0c58b-5312-4d2f-a584-341a852f58e7","Type":"ContainerDied","Data":"e3a944c6a5e5496a67a4ee9bed7c3d2c99032ab3e4d7ce89ab9db313c729f25a"} Jul 15 17:05:42.160363 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:42.160337 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxt8pb" Jul 15 17:05:42.237151 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:42.237117 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2kfw4\" (UniqueName: \"kubernetes.io/projected/16b0c58b-5312-4d2f-a584-341a852f58e7-kube-api-access-2kfw4\") pod \"16b0c58b-5312-4d2f-a584-341a852f58e7\" (UID: \"16b0c58b-5312-4d2f-a584-341a852f58e7\") " Jul 15 17:05:42.237334 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:42.237162 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/16b0c58b-5312-4d2f-a584-341a852f58e7-util\") pod \"16b0c58b-5312-4d2f-a584-341a852f58e7\" (UID: \"16b0c58b-5312-4d2f-a584-341a852f58e7\") " Jul 15 17:05:42.237334 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:42.237227 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/16b0c58b-5312-4d2f-a584-341a852f58e7-bundle\") pod \"16b0c58b-5312-4d2f-a584-341a852f58e7\" (UID: \"16b0c58b-5312-4d2f-a584-341a852f58e7\") " Jul 15 17:05:42.237986 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:42.237965 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16b0c58b-5312-4d2f-a584-341a852f58e7-bundle" (OuterVolumeSpecName: "bundle") pod "16b0c58b-5312-4d2f-a584-341a852f58e7" (UID: "16b0c58b-5312-4d2f-a584-341a852f58e7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:05:42.239206 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:42.239184 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16b0c58b-5312-4d2f-a584-341a852f58e7-kube-api-access-2kfw4" (OuterVolumeSpecName: "kube-api-access-2kfw4") pod "16b0c58b-5312-4d2f-a584-341a852f58e7" (UID: "16b0c58b-5312-4d2f-a584-341a852f58e7"). InnerVolumeSpecName "kube-api-access-2kfw4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:05:42.242788 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:42.242755 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16b0c58b-5312-4d2f-a584-341a852f58e7-util" (OuterVolumeSpecName: "util") pod "16b0c58b-5312-4d2f-a584-341a852f58e7" (UID: "16b0c58b-5312-4d2f-a584-341a852f58e7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:05:42.337897 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:42.337810 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kfw4\" (UniqueName: \"kubernetes.io/projected/16b0c58b-5312-4d2f-a584-341a852f58e7-kube-api-access-2kfw4\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:05:42.337897 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:42.337839 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/16b0c58b-5312-4d2f-a584-341a852f58e7-util\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:05:42.337897 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:42.337849 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/16b0c58b-5312-4d2f-a584-341a852f58e7-bundle\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:05:43.040740 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:43.040711 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxt8pb" event={"ID":"16b0c58b-5312-4d2f-a584-341a852f58e7","Type":"ContainerDied","Data":"1d1db4d7c57647466a3a500a41c2f8d3e97181abe1f6c48ecc39149a3aa7e4ef"} Jul 15 17:05:43.040740 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:43.040740 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxt8pb" Jul 15 17:05:43.040916 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:43.040754 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1d1db4d7c57647466a3a500a41c2f8d3e97181abe1f6c48ecc39149a3aa7e4ef" Jul 15 17:05:50.108481 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:50.108445 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:50.209456 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:50.209425 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:50.757559 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:50.757529 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:50.963443 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:50.963409 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:51.050232 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.050158 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:51.124249 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.124223 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-pf7tz"] Jul 15 17:05:51.124589 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.124566 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16b0c58b-5312-4d2f-a584-341a852f58e7" containerName="extract" Jul 15 17:05:51.124589 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.124577 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="16b0c58b-5312-4d2f-a584-341a852f58e7" containerName="extract" Jul 15 17:05:51.124589 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.124589 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16b0c58b-5312-4d2f-a584-341a852f58e7" containerName="util" Jul 15 17:05:51.124719 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.124610 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="16b0c58b-5312-4d2f-a584-341a852f58e7" containerName="util" Jul 15 17:05:51.124719 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.124617 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16b0c58b-5312-4d2f-a584-341a852f58e7" containerName="pull" Jul 15 17:05:51.124719 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.124623 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="16b0c58b-5312-4d2f-a584-341a852f58e7" containerName="pull" Jul 15 17:05:51.124719 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.124686 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="16b0c58b-5312-4d2f-a584-341a852f58e7" containerName="extract" Jul 15 17:05:51.128934 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.128918 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-pf7tz" Jul 15 17:05:51.131774 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.131752 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 15 17:05:51.132016 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.132003 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 15 17:05:51.132289 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.132277 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 15 17:05:51.132335 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.132310 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-5xhsk\"" Jul 15 17:05:51.132534 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.132518 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 15 17:05:51.150135 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.150109 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-pf7tz"] Jul 15 17:05:51.198195 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.198164 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0c424b8d-1396-4915-a075-a379ca4dfad5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-pf7tz\" (UID: \"0c424b8d-1396-4915-a075-a379ca4dfad5\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-pf7tz" Jul 15 17:05:51.198195 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.198195 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0c424b8d-1396-4915-a075-a379ca4dfad5-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-pf7tz\" (UID: \"0c424b8d-1396-4915-a075-a379ca4dfad5\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-pf7tz" Jul 15 17:05:51.198379 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.198260 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdz75\" (UniqueName: \"kubernetes.io/projected/0c424b8d-1396-4915-a075-a379ca4dfad5-kube-api-access-bdz75\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-pf7tz\" (UID: \"0c424b8d-1396-4915-a075-a379ca4dfad5\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-pf7tz" Jul 15 17:05:51.198379 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.198320 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c424b8d-1396-4915-a075-a379ca4dfad5-tmp\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-pf7tz\" (UID: \"0c424b8d-1396-4915-a075-a379ca4dfad5\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-pf7tz" Jul 15 17:05:51.198379 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.198347 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0c424b8d-1396-4915-a075-a379ca4dfad5-manifests\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-pf7tz\" (UID: \"0c424b8d-1396-4915-a075-a379ca4dfad5\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-pf7tz" Jul 15 17:05:51.244812 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.244785 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:51.299210 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.299183 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdz75\" (UniqueName: \"kubernetes.io/projected/0c424b8d-1396-4915-a075-a379ca4dfad5-kube-api-access-bdz75\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-pf7tz\" (UID: \"0c424b8d-1396-4915-a075-a379ca4dfad5\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-pf7tz" Jul 15 17:05:51.299344 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.299236 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c424b8d-1396-4915-a075-a379ca4dfad5-tmp\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-pf7tz\" (UID: \"0c424b8d-1396-4915-a075-a379ca4dfad5\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-pf7tz" Jul 15 17:05:51.299344 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.299277 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0c424b8d-1396-4915-a075-a379ca4dfad5-manifests\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-pf7tz\" (UID: \"0c424b8d-1396-4915-a075-a379ca4dfad5\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-pf7tz" Jul 15 17:05:51.299344 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.299329 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0c424b8d-1396-4915-a075-a379ca4dfad5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-pf7tz\" (UID: \"0c424b8d-1396-4915-a075-a379ca4dfad5\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-pf7tz" Jul 15 17:05:51.299511 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.299353 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0c424b8d-1396-4915-a075-a379ca4dfad5-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-pf7tz\" (UID: \"0c424b8d-1396-4915-a075-a379ca4dfad5\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-pf7tz" Jul 15 17:05:51.299708 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.299690 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0c424b8d-1396-4915-a075-a379ca4dfad5-manifests\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-pf7tz\" (UID: \"0c424b8d-1396-4915-a075-a379ca4dfad5\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-pf7tz" Jul 15 17:05:51.301429 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.301371 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c424b8d-1396-4915-a075-a379ca4dfad5-tmp\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-pf7tz\" (UID: \"0c424b8d-1396-4915-a075-a379ca4dfad5\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-pf7tz" Jul 15 17:05:51.301668 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.301652 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0c424b8d-1396-4915-a075-a379ca4dfad5-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-pf7tz\" (UID: \"0c424b8d-1396-4915-a075-a379ca4dfad5\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-pf7tz" Jul 15 17:05:51.301779 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.301761 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0c424b8d-1396-4915-a075-a379ca4dfad5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-pf7tz\" (UID: \"0c424b8d-1396-4915-a075-a379ca4dfad5\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-pf7tz" Jul 15 17:05:51.311372 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.311345 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdz75\" (UniqueName: \"kubernetes.io/projected/0c424b8d-1396-4915-a075-a379ca4dfad5-kube-api-access-bdz75\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-pf7tz\" (UID: \"0c424b8d-1396-4915-a075-a379ca4dfad5\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-pf7tz" Jul 15 17:05:51.346528 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.346496 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:51.439208 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.439174 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-pf7tz" Jul 15 17:05:51.458468 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.458445 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:51.556970 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.556898 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:51.582496 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.580764 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-pf7tz"] Jul 15 17:05:51.668041 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.668010 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:51.760045 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:51.760009 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:52.083919 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:52.083880 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-pf7tz" event={"ID":"0c424b8d-1396-4915-a075-a379ca4dfad5","Type":"ContainerStarted","Data":"5f1d21532b41d2ea56d3dd769b2489b0482177cf7e7711e81e900cc0523e1868"} Jul 15 17:05:52.772243 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:52.772193 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:52.873756 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:52.873710 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:54.607419 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:54.607381 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:54.681004 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:54.680967 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:55.101702 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:55.101670 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-pf7tz" event={"ID":"0c424b8d-1396-4915-a075-a379ca4dfad5","Type":"ContainerStarted","Data":"34c569653d0d8c3d5982e41b44c765603e2903440371e209dd4590e01a5f681b"} Jul 15 17:05:56.107965 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:56.107928 2578 generic.go:356] "Generic (PLEG): container finished" podID="0c424b8d-1396-4915-a075-a379ca4dfad5" containerID="34c569653d0d8c3d5982e41b44c765603e2903440371e209dd4590e01a5f681b" exitCode=0 Jul 15 17:05:56.108319 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:56.108004 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-pf7tz" event={"ID":"0c424b8d-1396-4915-a075-a379ca4dfad5","Type":"ContainerDied","Data":"34c569653d0d8c3d5982e41b44c765603e2903440371e209dd4590e01a5f681b"} Jul 15 17:05:57.114718 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:57.114686 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-pf7tz" event={"ID":"0c424b8d-1396-4915-a075-a379ca4dfad5","Type":"ContainerStarted","Data":"954356dbdc22e405c90d3aaabf8403a0b3eaf7c4d6d0509b4ed6514daf42555b"} Jul 15 17:05:57.115135 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:57.114808 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-pf7tz" Jul 15 17:05:57.138655 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:05:57.138590 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-pf7tz" podStartSLOduration=1.836705007 podStartE2EDuration="6.138573593s" totalImagesPullingTime="3.212338165s" totalInitContainerRuntime="1.089530421s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:05:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:05:57.137830769 +0000 UTC m=+574.690960986" watchObservedRunningTime="2026-07-15 17:05:57.138573593 +0000 UTC m=+574.691703806" Jul 15 17:06:00.832969 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:00.832931 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:00.911163 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:00.911130 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:08.122701 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:08.122668 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-pf7tz" Jul 15 17:06:11.000585 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:11.000555 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c485d79f6-4szdj"] Jul 15 17:06:11.008498 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:11.008478 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-4szdj" Jul 15 17:06:11.010974 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:11.010952 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 15 17:06:11.011075 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:11.010981 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 15 17:06:11.012389 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:11.012373 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 15 17:06:11.012467 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:11.012396 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 15 17:06:11.012467 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:11.012417 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 15 17:06:11.012552 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:11.012511 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-zzmcx\"" Jul 15 17:06:11.017772 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:11.017751 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c485d79f6-4szdj"] Jul 15 17:06:11.066541 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:11.066508 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6aa5939a-5bc2-4b1c-9294-f955d142ede2-metrics-cert\") pod \"lws-controller-manager-5c485d79f6-4szdj\" (UID: \"6aa5939a-5bc2-4b1c-9294-f955d142ede2\") " pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-4szdj" Jul 15 17:06:11.066700 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:11.066563 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k66wz\" (UniqueName: \"kubernetes.io/projected/6aa5939a-5bc2-4b1c-9294-f955d142ede2-kube-api-access-k66wz\") pod \"lws-controller-manager-5c485d79f6-4szdj\" (UID: \"6aa5939a-5bc2-4b1c-9294-f955d142ede2\") " pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-4szdj" Jul 15 17:06:11.066700 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:11.066632 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6aa5939a-5bc2-4b1c-9294-f955d142ede2-cert\") pod \"lws-controller-manager-5c485d79f6-4szdj\" (UID: \"6aa5939a-5bc2-4b1c-9294-f955d142ede2\") " pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-4szdj" Jul 15 17:06:11.066700 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:11.066666 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6aa5939a-5bc2-4b1c-9294-f955d142ede2-manager-config\") pod \"lws-controller-manager-5c485d79f6-4szdj\" (UID: \"6aa5939a-5bc2-4b1c-9294-f955d142ede2\") " pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-4szdj" Jul 15 17:06:11.167911 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:11.167877 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k66wz\" (UniqueName: \"kubernetes.io/projected/6aa5939a-5bc2-4b1c-9294-f955d142ede2-kube-api-access-k66wz\") pod \"lws-controller-manager-5c485d79f6-4szdj\" (UID: \"6aa5939a-5bc2-4b1c-9294-f955d142ede2\") " pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-4szdj" Jul 15 17:06:11.167911 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:11.167919 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6aa5939a-5bc2-4b1c-9294-f955d142ede2-cert\") pod \"lws-controller-manager-5c485d79f6-4szdj\" (UID: \"6aa5939a-5bc2-4b1c-9294-f955d142ede2\") " pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-4szdj" Jul 15 17:06:11.168112 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:11.167943 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6aa5939a-5bc2-4b1c-9294-f955d142ede2-manager-config\") pod \"lws-controller-manager-5c485d79f6-4szdj\" (UID: \"6aa5939a-5bc2-4b1c-9294-f955d142ede2\") " pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-4szdj" Jul 15 17:06:11.168112 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:11.167993 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6aa5939a-5bc2-4b1c-9294-f955d142ede2-metrics-cert\") pod \"lws-controller-manager-5c485d79f6-4szdj\" (UID: \"6aa5939a-5bc2-4b1c-9294-f955d142ede2\") " pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-4szdj" Jul 15 17:06:11.168671 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:11.168649 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6aa5939a-5bc2-4b1c-9294-f955d142ede2-manager-config\") pod \"lws-controller-manager-5c485d79f6-4szdj\" (UID: \"6aa5939a-5bc2-4b1c-9294-f955d142ede2\") " pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-4szdj" Jul 15 17:06:11.170371 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:11.170344 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6aa5939a-5bc2-4b1c-9294-f955d142ede2-cert\") pod \"lws-controller-manager-5c485d79f6-4szdj\" (UID: \"6aa5939a-5bc2-4b1c-9294-f955d142ede2\") " pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-4szdj" Jul 15 17:06:11.170456 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:11.170391 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6aa5939a-5bc2-4b1c-9294-f955d142ede2-metrics-cert\") pod \"lws-controller-manager-5c485d79f6-4szdj\" (UID: \"6aa5939a-5bc2-4b1c-9294-f955d142ede2\") " pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-4szdj" Jul 15 17:06:11.198166 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:11.198136 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k66wz\" (UniqueName: \"kubernetes.io/projected/6aa5939a-5bc2-4b1c-9294-f955d142ede2-kube-api-access-k66wz\") pod \"lws-controller-manager-5c485d79f6-4szdj\" (UID: \"6aa5939a-5bc2-4b1c-9294-f955d142ede2\") " pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-4szdj" Jul 15 17:06:11.319535 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:11.319423 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-4szdj" Jul 15 17:06:11.455372 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:11.455235 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c485d79f6-4szdj"] Jul 15 17:06:11.457466 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:06:11.457439 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aa5939a_5bc2_4b1c_9294_f955d142ede2.slice/crio-6672d0a4d049e7003b26282c60450d3389dbf6c24f828aa5919cbc379e5d6aa9 WatchSource:0}: Error finding container 6672d0a4d049e7003b26282c60450d3389dbf6c24f828aa5919cbc379e5d6aa9: Status 404 returned error can't find the container with id 6672d0a4d049e7003b26282c60450d3389dbf6c24f828aa5919cbc379e5d6aa9 Jul 15 17:06:12.190420 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:12.190380 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-4szdj" event={"ID":"6aa5939a-5bc2-4b1c-9294-f955d142ede2","Type":"ContainerStarted","Data":"6672d0a4d049e7003b26282c60450d3389dbf6c24f828aa5919cbc379e5d6aa9"} Jul 15 17:06:12.296758 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:12.296724 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:12.399329 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:12.399291 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:13.048808 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:13.048778 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:13.126892 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:13.126858 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-b44776b5f-k6nnt"] Jul 15 17:06:13.130896 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:13.130871 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-b44776b5f-k6nnt" Jul 15 17:06:13.134160 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:13.133750 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-5ttsp\"" Jul 15 17:06:13.134160 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:13.133786 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 15 17:06:13.134160 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:13.134059 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 15 17:06:13.134160 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:13.134066 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 15 17:06:13.134709 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:13.134690 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 15 17:06:13.139391 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:13.139374 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-b44776b5f-k6nnt"] Jul 15 17:06:13.187167 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:13.187133 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea760cc3-88ea-467e-94c8-6a790eb96d71-tls-certs\") pod \"kube-auth-proxy-b44776b5f-k6nnt\" (UID: \"ea760cc3-88ea-467e-94c8-6a790eb96d71\") " pod="openshift-ingress/kube-auth-proxy-b44776b5f-k6nnt" Jul 15 17:06:13.187351 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:13.187204 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fphdl\" (UniqueName: \"kubernetes.io/projected/ea760cc3-88ea-467e-94c8-6a790eb96d71-kube-api-access-fphdl\") pod \"kube-auth-proxy-b44776b5f-k6nnt\" (UID: \"ea760cc3-88ea-467e-94c8-6a790eb96d71\") " pod="openshift-ingress/kube-auth-proxy-b44776b5f-k6nnt" Jul 15 17:06:13.187351 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:13.187231 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea760cc3-88ea-467e-94c8-6a790eb96d71-tmp\") pod \"kube-auth-proxy-b44776b5f-k6nnt\" (UID: \"ea760cc3-88ea-467e-94c8-6a790eb96d71\") " pod="openshift-ingress/kube-auth-proxy-b44776b5f-k6nnt" Jul 15 17:06:13.197494 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:13.197470 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-4szdj" event={"ID":"6aa5939a-5bc2-4b1c-9294-f955d142ede2","Type":"ContainerStarted","Data":"15199a8c1252685867ed4f20e98b5b252e5969a9f61a57efee91249aebd43b49"} Jul 15 17:06:13.197922 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:13.197526 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-4szdj" Jul 15 17:06:13.219658 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:13.219610 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-4szdj" podStartSLOduration=1.676658748 podStartE2EDuration="3.219579821s" totalImagesPullingTime="1.542921073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:06:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:06:13.217931513 +0000 UTC m=+590.771061757" watchObservedRunningTime="2026-07-15 17:06:13.219579821 +0000 UTC m=+590.772710036" Jul 15 17:06:13.287991 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:13.287912 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea760cc3-88ea-467e-94c8-6a790eb96d71-tls-certs\") pod \"kube-auth-proxy-b44776b5f-k6nnt\" (UID: \"ea760cc3-88ea-467e-94c8-6a790eb96d71\") " pod="openshift-ingress/kube-auth-proxy-b44776b5f-k6nnt" Jul 15 17:06:13.288147 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:13.287997 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fphdl\" (UniqueName: \"kubernetes.io/projected/ea760cc3-88ea-467e-94c8-6a790eb96d71-kube-api-access-fphdl\") pod \"kube-auth-proxy-b44776b5f-k6nnt\" (UID: \"ea760cc3-88ea-467e-94c8-6a790eb96d71\") " pod="openshift-ingress/kube-auth-proxy-b44776b5f-k6nnt" Jul 15 17:06:13.288147 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:06:13.288064 2578 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 15 17:06:13.288147 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:13.288066 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea760cc3-88ea-467e-94c8-6a790eb96d71-tmp\") pod \"kube-auth-proxy-b44776b5f-k6nnt\" (UID: \"ea760cc3-88ea-467e-94c8-6a790eb96d71\") " pod="openshift-ingress/kube-auth-proxy-b44776b5f-k6nnt" Jul 15 17:06:13.288147 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:06:13.288130 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea760cc3-88ea-467e-94c8-6a790eb96d71-tls-certs podName:ea760cc3-88ea-467e-94c8-6a790eb96d71 nodeName:}" failed. No retries permitted until 2026-07-15 17:06:13.788111867 +0000 UTC m=+591.341242059 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/ea760cc3-88ea-467e-94c8-6a790eb96d71-tls-certs") pod "kube-auth-proxy-b44776b5f-k6nnt" (UID: "ea760cc3-88ea-467e-94c8-6a790eb96d71") : secret "kube-auth-proxy-tls" not found Jul 15 17:06:13.290200 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:13.290181 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea760cc3-88ea-467e-94c8-6a790eb96d71-tmp\") pod \"kube-auth-proxy-b44776b5f-k6nnt\" (UID: \"ea760cc3-88ea-467e-94c8-6a790eb96d71\") " pod="openshift-ingress/kube-auth-proxy-b44776b5f-k6nnt" Jul 15 17:06:13.296516 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:13.296496 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fphdl\" (UniqueName: \"kubernetes.io/projected/ea760cc3-88ea-467e-94c8-6a790eb96d71-kube-api-access-fphdl\") pod \"kube-auth-proxy-b44776b5f-k6nnt\" (UID: \"ea760cc3-88ea-467e-94c8-6a790eb96d71\") " pod="openshift-ingress/kube-auth-proxy-b44776b5f-k6nnt" Jul 15 17:06:13.303757 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:13.303732 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:13.409475 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:13.409439 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:13.504646 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:13.504583 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:13.613897 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:13.613816 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:13.792658 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:13.792624 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea760cc3-88ea-467e-94c8-6a790eb96d71-tls-certs\") pod \"kube-auth-proxy-b44776b5f-k6nnt\" (UID: \"ea760cc3-88ea-467e-94c8-6a790eb96d71\") " pod="openshift-ingress/kube-auth-proxy-b44776b5f-k6nnt" Jul 15 17:06:13.794950 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:13.794922 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea760cc3-88ea-467e-94c8-6a790eb96d71-tls-certs\") pod \"kube-auth-proxy-b44776b5f-k6nnt\" (UID: \"ea760cc3-88ea-467e-94c8-6a790eb96d71\") " pod="openshift-ingress/kube-auth-proxy-b44776b5f-k6nnt" Jul 15 17:06:13.821825 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:13.821797 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:14.013784 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:14.013754 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:14.043499 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:14.043471 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-b44776b5f-k6nnt" Jul 15 17:06:14.156064 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:14.156029 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:14.198970 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:14.198659 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-b44776b5f-k6nnt"] Jul 15 17:06:14.201649 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:06:14.201620 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea760cc3_88ea_467e_94c8_6a790eb96d71.slice/crio-f43c41a4f3014618178b58df99725b503a6af4ab71450a7271ea48bccdfaa0c2 WatchSource:0}: Error finding container f43c41a4f3014618178b58df99725b503a6af4ab71450a7271ea48bccdfaa0c2: Status 404 returned error can't find the container with id f43c41a4f3014618178b58df99725b503a6af4ab71450a7271ea48bccdfaa0c2 Jul 15 17:06:14.392014 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:14.391939 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:14.681087 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:14.681054 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:14.776440 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:14.776406 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:15.208254 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:15.208221 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-b44776b5f-k6nnt" event={"ID":"ea760cc3-88ea-467e-94c8-6a790eb96d71","Type":"ContainerStarted","Data":"f43c41a4f3014618178b58df99725b503a6af4ab71450a7271ea48bccdfaa0c2"} Jul 15 17:06:15.798795 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:15.798762 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:17.220555 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:17.220527 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-b44776b5f-k6nnt" event={"ID":"ea760cc3-88ea-467e-94c8-6a790eb96d71","Type":"ContainerStarted","Data":"1fbbe44eaa1969477595306d61853fe449d6e296c76daa6abdd77c830e38816d"} Jul 15 17:06:17.247119 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:17.247060 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-b44776b5f-k6nnt" podStartSLOduration=1.291676541 podStartE2EDuration="4.247044685s" totalImagesPullingTime="2.955368144s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:06:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:06:17.244546829 +0000 UTC m=+594.797677043" watchObservedRunningTime="2026-07-15 17:06:17.247044685 +0000 UTC m=+594.800174899" Jul 15 17:06:22.047616 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:22.047559 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:22.145649 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:22.145585 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:22.516851 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:22.516820 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:22.748779 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:22.748750 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:22.855952 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:22.855884 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:24.174059 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:24.174016 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:24.204839 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:24.204810 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-4szdj" Jul 15 17:06:24.281328 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:24.281288 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:24.395727 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:24.395693 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:24.488063 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:24.487978 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:24.588566 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:24.588534 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:24.690736 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:24.690708 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:24.819769 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:24.819677 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:25.132017 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:25.131904 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:29.088015 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:29.087988 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29wtx_88f8928c-237e-457f-a032-707f48111f36/ovn-acl-logging/0.log" Jul 15 17:06:29.088411 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:29.087997 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29wtx_88f8928c-237e-457f-a032-707f48111f36/ovn-acl-logging/0.log" Jul 15 17:06:50.309656 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:50.309623 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:53.891188 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:06:53.891154 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:07:13.863239 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:13.863200 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:07:13.958307 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:13.958273 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:07:14.089301 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:14.089271 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:07:14.821785 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:14.821753 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:07:15.030698 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:15.030638 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:07:45.312764 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.312728 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn"] Jul 15 17:07:45.316183 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.316166 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn" Jul 15 17:07:45.318672 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.318649 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 15 17:07:45.318781 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.318692 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 15 17:07:45.319549 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.319528 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-fx4k8\"" Jul 15 17:07:45.324316 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.324298 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn"] Jul 15 17:07:45.374138 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.374113 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2795c4a6-2feb-470e-b29f-8ac8c291b4cc-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn\" (UID: \"2795c4a6-2feb-470e-b29f-8ac8c291b4cc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn" Jul 15 17:07:45.374264 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.374153 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2795c4a6-2feb-470e-b29f-8ac8c291b4cc-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn\" (UID: \"2795c4a6-2feb-470e-b29f-8ac8c291b4cc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn" Jul 15 17:07:45.374264 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.374212 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4pdd\" (UniqueName: \"kubernetes.io/projected/2795c4a6-2feb-470e-b29f-8ac8c291b4cc-kube-api-access-j4pdd\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn\" (UID: \"2795c4a6-2feb-470e-b29f-8ac8c291b4cc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn" Jul 15 17:07:45.412055 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.412031 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh"] Jul 15 17:07:45.415457 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.415443 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh" Jul 15 17:07:45.426456 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.426436 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh"] Jul 15 17:07:45.475285 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.475249 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/720f778f-b686-46d0-81b9-c85c5f3db32c-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh\" (UID: \"720f778f-b686-46d0-81b9-c85c5f3db32c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh" Jul 15 17:07:45.475435 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.475298 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2795c4a6-2feb-470e-b29f-8ac8c291b4cc-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn\" (UID: \"2795c4a6-2feb-470e-b29f-8ac8c291b4cc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn" Jul 15 17:07:45.475435 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.475347 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/720f778f-b686-46d0-81b9-c85c5f3db32c-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh\" (UID: \"720f778f-b686-46d0-81b9-c85c5f3db32c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh" Jul 15 17:07:45.475435 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.475388 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t88wg\" (UniqueName: \"kubernetes.io/projected/720f778f-b686-46d0-81b9-c85c5f3db32c-kube-api-access-t88wg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh\" (UID: \"720f778f-b686-46d0-81b9-c85c5f3db32c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh" Jul 15 17:07:45.475435 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.475409 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4pdd\" (UniqueName: \"kubernetes.io/projected/2795c4a6-2feb-470e-b29f-8ac8c291b4cc-kube-api-access-j4pdd\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn\" (UID: \"2795c4a6-2feb-470e-b29f-8ac8c291b4cc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn" Jul 15 17:07:45.475623 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.475459 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2795c4a6-2feb-470e-b29f-8ac8c291b4cc-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn\" (UID: \"2795c4a6-2feb-470e-b29f-8ac8c291b4cc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn" Jul 15 17:07:45.475623 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.475579 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2795c4a6-2feb-470e-b29f-8ac8c291b4cc-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn\" (UID: \"2795c4a6-2feb-470e-b29f-8ac8c291b4cc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn" Jul 15 17:07:45.475756 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.475740 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2795c4a6-2feb-470e-b29f-8ac8c291b4cc-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn\" (UID: \"2795c4a6-2feb-470e-b29f-8ac8c291b4cc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn" Jul 15 17:07:45.484204 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.484184 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4pdd\" (UniqueName: \"kubernetes.io/projected/2795c4a6-2feb-470e-b29f-8ac8c291b4cc-kube-api-access-j4pdd\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn\" (UID: \"2795c4a6-2feb-470e-b29f-8ac8c291b4cc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn" Jul 15 17:07:45.576557 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.576477 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/720f778f-b686-46d0-81b9-c85c5f3db32c-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh\" (UID: \"720f778f-b686-46d0-81b9-c85c5f3db32c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh" Jul 15 17:07:45.576557 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.576528 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/720f778f-b686-46d0-81b9-c85c5f3db32c-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh\" (UID: \"720f778f-b686-46d0-81b9-c85c5f3db32c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh" Jul 15 17:07:45.576557 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.576554 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t88wg\" (UniqueName: \"kubernetes.io/projected/720f778f-b686-46d0-81b9-c85c5f3db32c-kube-api-access-t88wg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh\" (UID: \"720f778f-b686-46d0-81b9-c85c5f3db32c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh" Jul 15 17:07:45.576870 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.576844 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/720f778f-b686-46d0-81b9-c85c5f3db32c-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh\" (UID: \"720f778f-b686-46d0-81b9-c85c5f3db32c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh" Jul 15 17:07:45.576932 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.576908 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/720f778f-b686-46d0-81b9-c85c5f3db32c-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh\" (UID: \"720f778f-b686-46d0-81b9-c85c5f3db32c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh" Jul 15 17:07:45.586549 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.586524 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t88wg\" (UniqueName: \"kubernetes.io/projected/720f778f-b686-46d0-81b9-c85c5f3db32c-kube-api-access-t88wg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh\" (UID: \"720f778f-b686-46d0-81b9-c85c5f3db32c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh" Jul 15 17:07:45.626935 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.626912 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn" Jul 15 17:07:45.716510 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.716385 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd"] Jul 15 17:07:45.720142 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.720123 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd" Jul 15 17:07:45.724470 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.724446 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh" Jul 15 17:07:45.738712 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.736055 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd"] Jul 15 17:07:45.767532 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.767465 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn"] Jul 15 17:07:45.769782 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:07:45.769752 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2795c4a6_2feb_470e_b29f_8ac8c291b4cc.slice/crio-ae6ca0d65af88b1a8a1d9818e88335a13d647ecb6aaedc66d82100720ddfc1b8 WatchSource:0}: Error finding container ae6ca0d65af88b1a8a1d9818e88335a13d647ecb6aaedc66d82100720ddfc1b8: Status 404 returned error can't find the container with id ae6ca0d65af88b1a8a1d9818e88335a13d647ecb6aaedc66d82100720ddfc1b8 Jul 15 17:07:45.777905 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.777881 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jntdf\" (UniqueName: \"kubernetes.io/projected/b454fd31-3cf9-43f3-80d8-4f38886d03ef-kube-api-access-jntdf\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd\" (UID: \"b454fd31-3cf9-43f3-80d8-4f38886d03ef\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd" Jul 15 17:07:45.778015 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.777919 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b454fd31-3cf9-43f3-80d8-4f38886d03ef-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd\" (UID: \"b454fd31-3cf9-43f3-80d8-4f38886d03ef\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd" Jul 15 17:07:45.778015 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.777947 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b454fd31-3cf9-43f3-80d8-4f38886d03ef-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd\" (UID: \"b454fd31-3cf9-43f3-80d8-4f38886d03ef\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd" Jul 15 17:07:45.879208 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.879182 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jntdf\" (UniqueName: \"kubernetes.io/projected/b454fd31-3cf9-43f3-80d8-4f38886d03ef-kube-api-access-jntdf\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd\" (UID: \"b454fd31-3cf9-43f3-80d8-4f38886d03ef\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd" Jul 15 17:07:45.879329 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.879224 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b454fd31-3cf9-43f3-80d8-4f38886d03ef-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd\" (UID: \"b454fd31-3cf9-43f3-80d8-4f38886d03ef\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd" Jul 15 17:07:45.879418 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.879394 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b454fd31-3cf9-43f3-80d8-4f38886d03ef-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd\" (UID: \"b454fd31-3cf9-43f3-80d8-4f38886d03ef\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd" Jul 15 17:07:45.879629 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.879580 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b454fd31-3cf9-43f3-80d8-4f38886d03ef-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd\" (UID: \"b454fd31-3cf9-43f3-80d8-4f38886d03ef\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd" Jul 15 17:07:45.879760 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.879724 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b454fd31-3cf9-43f3-80d8-4f38886d03ef-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd\" (UID: \"b454fd31-3cf9-43f3-80d8-4f38886d03ef\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd" Jul 15 17:07:45.883795 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.883776 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh"] Jul 15 17:07:45.885944 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:07:45.885920 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod720f778f_b686_46d0_81b9_c85c5f3db32c.slice/crio-95314c941d6419e3f59db94ca0d98cbba139ad35faef760168eccdb1987b15df WatchSource:0}: Error finding container 95314c941d6419e3f59db94ca0d98cbba139ad35faef760168eccdb1987b15df: Status 404 returned error can't find the container with id 95314c941d6419e3f59db94ca0d98cbba139ad35faef760168eccdb1987b15df Jul 15 17:07:45.888779 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:45.888758 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jntdf\" (UniqueName: \"kubernetes.io/projected/b454fd31-3cf9-43f3-80d8-4f38886d03ef-kube-api-access-jntdf\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd\" (UID: \"b454fd31-3cf9-43f3-80d8-4f38886d03ef\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd" Jul 15 17:07:46.039554 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:46.039528 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd" Jul 15 17:07:46.103569 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:46.103539 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z"] Jul 15 17:07:46.107439 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:46.107419 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z" Jul 15 17:07:46.118174 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:46.118148 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z"] Jul 15 17:07:46.174260 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:46.174237 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd"] Jul 15 17:07:46.175837 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:07:46.175812 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb454fd31_3cf9_43f3_80d8_4f38886d03ef.slice/crio-1f85fd50b3d4aeb27d61f95cb902d526401f977b63644dc742bdb2d48cd0aecb WatchSource:0}: Error finding container 1f85fd50b3d4aeb27d61f95cb902d526401f977b63644dc742bdb2d48cd0aecb: Status 404 returned error can't find the container with id 1f85fd50b3d4aeb27d61f95cb902d526401f977b63644dc742bdb2d48cd0aecb Jul 15 17:07:46.182592 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:46.182570 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/039839f4-8025-477b-b8d7-8137daafc9d9-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z\" (UID: \"039839f4-8025-477b-b8d7-8137daafc9d9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z" Jul 15 17:07:46.182689 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:46.182622 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blck2\" (UniqueName: \"kubernetes.io/projected/039839f4-8025-477b-b8d7-8137daafc9d9-kube-api-access-blck2\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z\" (UID: \"039839f4-8025-477b-b8d7-8137daafc9d9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z" Jul 15 17:07:46.182740 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:46.182720 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/039839f4-8025-477b-b8d7-8137daafc9d9-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z\" (UID: \"039839f4-8025-477b-b8d7-8137daafc9d9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z" Jul 15 17:07:46.283483 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:46.283457 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/039839f4-8025-477b-b8d7-8137daafc9d9-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z\" (UID: \"039839f4-8025-477b-b8d7-8137daafc9d9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z" Jul 15 17:07:46.283611 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:46.283521 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/039839f4-8025-477b-b8d7-8137daafc9d9-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z\" (UID: \"039839f4-8025-477b-b8d7-8137daafc9d9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z" Jul 15 17:07:46.283611 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:46.283549 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blck2\" (UniqueName: \"kubernetes.io/projected/039839f4-8025-477b-b8d7-8137daafc9d9-kube-api-access-blck2\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z\" (UID: \"039839f4-8025-477b-b8d7-8137daafc9d9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z" Jul 15 17:07:46.283892 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:46.283869 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/039839f4-8025-477b-b8d7-8137daafc9d9-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z\" (UID: \"039839f4-8025-477b-b8d7-8137daafc9d9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z" Jul 15 17:07:46.283892 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:46.283881 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/039839f4-8025-477b-b8d7-8137daafc9d9-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z\" (UID: \"039839f4-8025-477b-b8d7-8137daafc9d9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z" Jul 15 17:07:46.292486 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:46.292459 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blck2\" (UniqueName: \"kubernetes.io/projected/039839f4-8025-477b-b8d7-8137daafc9d9-kube-api-access-blck2\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z\" (UID: \"039839f4-8025-477b-b8d7-8137daafc9d9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z" Jul 15 17:07:46.416721 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:46.416692 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z" Jul 15 17:07:46.551275 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:46.551236 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z"] Jul 15 17:07:46.552632 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:07:46.552587 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod039839f4_8025_477b_b8d7_8137daafc9d9.slice/crio-4ea3c954b7f0156725aa3cfbdcb0b8f060d39b7520db1838ba17c36fac526cc7 WatchSource:0}: Error finding container 4ea3c954b7f0156725aa3cfbdcb0b8f060d39b7520db1838ba17c36fac526cc7: Status 404 returned error can't find the container with id 4ea3c954b7f0156725aa3cfbdcb0b8f060d39b7520db1838ba17c36fac526cc7 Jul 15 17:07:46.646487 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:46.646444 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z" event={"ID":"039839f4-8025-477b-b8d7-8137daafc9d9","Type":"ContainerStarted","Data":"2fa5e31eab1b1f2f893c91873b3d1f823e40a1c9a1669f6fb1669082be676565"} Jul 15 17:07:46.646674 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:46.646508 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z" event={"ID":"039839f4-8025-477b-b8d7-8137daafc9d9","Type":"ContainerStarted","Data":"4ea3c954b7f0156725aa3cfbdcb0b8f060d39b7520db1838ba17c36fac526cc7"} Jul 15 17:07:46.648216 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:46.648188 2578 generic.go:356] "Generic (PLEG): container finished" podID="2795c4a6-2feb-470e-b29f-8ac8c291b4cc" containerID="40d342e520f41b132814cdfd93315e35762721c9cdc0ff8451798b8a7d266d50" exitCode=0 Jul 15 17:07:46.648336 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:46.648269 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn" event={"ID":"2795c4a6-2feb-470e-b29f-8ac8c291b4cc","Type":"ContainerDied","Data":"40d342e520f41b132814cdfd93315e35762721c9cdc0ff8451798b8a7d266d50"} Jul 15 17:07:46.648336 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:46.648315 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn" event={"ID":"2795c4a6-2feb-470e-b29f-8ac8c291b4cc","Type":"ContainerStarted","Data":"ae6ca0d65af88b1a8a1d9818e88335a13d647ecb6aaedc66d82100720ddfc1b8"} Jul 15 17:07:46.648739 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:46.648722 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 17:07:46.650040 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:46.650015 2578 generic.go:356] "Generic (PLEG): container finished" podID="b454fd31-3cf9-43f3-80d8-4f38886d03ef" containerID="16247f16630d1c0ecbe3987ef550b4f5c139c07f6f8633eafd559dc96ebebdbd" exitCode=0 Jul 15 17:07:46.650136 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:46.650080 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd" event={"ID":"b454fd31-3cf9-43f3-80d8-4f38886d03ef","Type":"ContainerDied","Data":"16247f16630d1c0ecbe3987ef550b4f5c139c07f6f8633eafd559dc96ebebdbd"} Jul 15 17:07:46.650136 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:46.650124 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd" event={"ID":"b454fd31-3cf9-43f3-80d8-4f38886d03ef","Type":"ContainerStarted","Data":"1f85fd50b3d4aeb27d61f95cb902d526401f977b63644dc742bdb2d48cd0aecb"} Jul 15 17:07:46.651855 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:46.651834 2578 generic.go:356] "Generic (PLEG): container finished" podID="720f778f-b686-46d0-81b9-c85c5f3db32c" containerID="db206cf61fa634e817131cea375ab899bfc7a9cc070fdc957fbacc101943516a" exitCode=0 Jul 15 17:07:46.651934 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:46.651914 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh" event={"ID":"720f778f-b686-46d0-81b9-c85c5f3db32c","Type":"ContainerDied","Data":"db206cf61fa634e817131cea375ab899bfc7a9cc070fdc957fbacc101943516a"} Jul 15 17:07:46.651975 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:46.651958 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh" event={"ID":"720f778f-b686-46d0-81b9-c85c5f3db32c","Type":"ContainerStarted","Data":"95314c941d6419e3f59db94ca0d98cbba139ad35faef760168eccdb1987b15df"} Jul 15 17:07:47.659003 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:47.658975 2578 generic.go:356] "Generic (PLEG): container finished" podID="039839f4-8025-477b-b8d7-8137daafc9d9" containerID="2fa5e31eab1b1f2f893c91873b3d1f823e40a1c9a1669f6fb1669082be676565" exitCode=0 Jul 15 17:07:47.659413 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:47.659073 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z" event={"ID":"039839f4-8025-477b-b8d7-8137daafc9d9","Type":"ContainerDied","Data":"2fa5e31eab1b1f2f893c91873b3d1f823e40a1c9a1669f6fb1669082be676565"} Jul 15 17:07:47.661338 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:47.661310 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd" event={"ID":"b454fd31-3cf9-43f3-80d8-4f38886d03ef","Type":"ContainerStarted","Data":"77251833be2a40637262306b70b4069167c2bc8d7243d24802f1a8a08f3e01d9"} Jul 15 17:07:47.663557 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:47.663537 2578 generic.go:356] "Generic (PLEG): container finished" podID="720f778f-b686-46d0-81b9-c85c5f3db32c" containerID="facb876a6070dc55c59dab0d3d93c7805e4141a14d067b8e1f38f2f4c73bb17b" exitCode=0 Jul 15 17:07:47.663670 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:47.663622 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh" event={"ID":"720f778f-b686-46d0-81b9-c85c5f3db32c","Type":"ContainerDied","Data":"facb876a6070dc55c59dab0d3d93c7805e4141a14d067b8e1f38f2f4c73bb17b"} Jul 15 17:07:48.670769 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:48.670736 2578 generic.go:356] "Generic (PLEG): container finished" podID="b454fd31-3cf9-43f3-80d8-4f38886d03ef" containerID="77251833be2a40637262306b70b4069167c2bc8d7243d24802f1a8a08f3e01d9" exitCode=0 Jul 15 17:07:48.671189 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:48.670818 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd" event={"ID":"b454fd31-3cf9-43f3-80d8-4f38886d03ef","Type":"ContainerDied","Data":"77251833be2a40637262306b70b4069167c2bc8d7243d24802f1a8a08f3e01d9"} Jul 15 17:07:48.673016 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:48.672994 2578 generic.go:356] "Generic (PLEG): container finished" podID="720f778f-b686-46d0-81b9-c85c5f3db32c" containerID="2c0a126681d3a1c1dd12fc197848463f33d8e01764dd4ef039507e8001ed5723" exitCode=0 Jul 15 17:07:48.673088 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:48.673072 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh" event={"ID":"720f778f-b686-46d0-81b9-c85c5f3db32c","Type":"ContainerDied","Data":"2c0a126681d3a1c1dd12fc197848463f33d8e01764dd4ef039507e8001ed5723"} Jul 15 17:07:49.679971 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:49.679940 2578 generic.go:356] "Generic (PLEG): container finished" podID="2795c4a6-2feb-470e-b29f-8ac8c291b4cc" containerID="976395fe901ea4bd81b402646bc4f1f9259ea7e3176bdb0169d6c4d47de982d0" exitCode=0 Jul 15 17:07:49.680298 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:49.680014 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn" event={"ID":"2795c4a6-2feb-470e-b29f-8ac8c291b4cc","Type":"ContainerDied","Data":"976395fe901ea4bd81b402646bc4f1f9259ea7e3176bdb0169d6c4d47de982d0"} Jul 15 17:07:49.682111 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:49.682088 2578 generic.go:356] "Generic (PLEG): container finished" podID="b454fd31-3cf9-43f3-80d8-4f38886d03ef" containerID="98c2e40343ad1e28b2c4b6b4d94ad7a4033bd12ff46fe7002aebe2974cde4331" exitCode=0 Jul 15 17:07:49.682196 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:49.682125 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd" event={"ID":"b454fd31-3cf9-43f3-80d8-4f38886d03ef","Type":"ContainerDied","Data":"98c2e40343ad1e28b2c4b6b4d94ad7a4033bd12ff46fe7002aebe2974cde4331"} Jul 15 17:07:49.889470 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:49.889446 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh" Jul 15 17:07:49.914400 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:49.914380 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t88wg\" (UniqueName: \"kubernetes.io/projected/720f778f-b686-46d0-81b9-c85c5f3db32c-kube-api-access-t88wg\") pod \"720f778f-b686-46d0-81b9-c85c5f3db32c\" (UID: \"720f778f-b686-46d0-81b9-c85c5f3db32c\") " Jul 15 17:07:49.914495 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:49.914424 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/720f778f-b686-46d0-81b9-c85c5f3db32c-bundle\") pod \"720f778f-b686-46d0-81b9-c85c5f3db32c\" (UID: \"720f778f-b686-46d0-81b9-c85c5f3db32c\") " Jul 15 17:07:49.914495 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:49.914469 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/720f778f-b686-46d0-81b9-c85c5f3db32c-util\") pod \"720f778f-b686-46d0-81b9-c85c5f3db32c\" (UID: \"720f778f-b686-46d0-81b9-c85c5f3db32c\") " Jul 15 17:07:49.914954 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:49.914927 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/720f778f-b686-46d0-81b9-c85c5f3db32c-bundle" (OuterVolumeSpecName: "bundle") pod "720f778f-b686-46d0-81b9-c85c5f3db32c" (UID: "720f778f-b686-46d0-81b9-c85c5f3db32c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:07:49.916487 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:49.916461 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/720f778f-b686-46d0-81b9-c85c5f3db32c-kube-api-access-t88wg" (OuterVolumeSpecName: "kube-api-access-t88wg") pod "720f778f-b686-46d0-81b9-c85c5f3db32c" (UID: "720f778f-b686-46d0-81b9-c85c5f3db32c"). InnerVolumeSpecName "kube-api-access-t88wg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:07:49.923359 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:49.923335 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/720f778f-b686-46d0-81b9-c85c5f3db32c-util" (OuterVolumeSpecName: "util") pod "720f778f-b686-46d0-81b9-c85c5f3db32c" (UID: "720f778f-b686-46d0-81b9-c85c5f3db32c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:07:50.015703 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:50.015671 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/720f778f-b686-46d0-81b9-c85c5f3db32c-util\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:07:50.015703 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:50.015697 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t88wg\" (UniqueName: \"kubernetes.io/projected/720f778f-b686-46d0-81b9-c85c5f3db32c-kube-api-access-t88wg\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:07:50.015703 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:50.015708 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/720f778f-b686-46d0-81b9-c85c5f3db32c-bundle\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:07:50.690465 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:50.690430 2578 generic.go:356] "Generic (PLEG): container finished" podID="039839f4-8025-477b-b8d7-8137daafc9d9" containerID="720cd684268901f063dbef517909f0b8ce5f848e6e3150db6563d786c3acbc2c" exitCode=0 Jul 15 17:07:50.690902 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:50.690513 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z" event={"ID":"039839f4-8025-477b-b8d7-8137daafc9d9","Type":"ContainerDied","Data":"720cd684268901f063dbef517909f0b8ce5f848e6e3150db6563d786c3acbc2c"} Jul 15 17:07:50.692845 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:50.692825 2578 generic.go:356] "Generic (PLEG): container finished" podID="2795c4a6-2feb-470e-b29f-8ac8c291b4cc" containerID="0d6e41d0da6cbb47b35781da8b753ad33b1a2d0054591e7899c3abc5ed6a48a6" exitCode=0 Jul 15 17:07:50.692943 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:50.692906 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn" event={"ID":"2795c4a6-2feb-470e-b29f-8ac8c291b4cc","Type":"ContainerDied","Data":"0d6e41d0da6cbb47b35781da8b753ad33b1a2d0054591e7899c3abc5ed6a48a6"} Jul 15 17:07:50.694737 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:50.694720 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh" Jul 15 17:07:50.694841 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:50.694750 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh" event={"ID":"720f778f-b686-46d0-81b9-c85c5f3db32c","Type":"ContainerDied","Data":"95314c941d6419e3f59db94ca0d98cbba139ad35faef760168eccdb1987b15df"} Jul 15 17:07:50.694841 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:50.694782 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="95314c941d6419e3f59db94ca0d98cbba139ad35faef760168eccdb1987b15df" Jul 15 17:07:50.834419 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:50.834393 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd" Jul 15 17:07:50.923590 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:50.923520 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jntdf\" (UniqueName: \"kubernetes.io/projected/b454fd31-3cf9-43f3-80d8-4f38886d03ef-kube-api-access-jntdf\") pod \"b454fd31-3cf9-43f3-80d8-4f38886d03ef\" (UID: \"b454fd31-3cf9-43f3-80d8-4f38886d03ef\") " Jul 15 17:07:50.923748 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:50.923729 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b454fd31-3cf9-43f3-80d8-4f38886d03ef-util\") pod \"b454fd31-3cf9-43f3-80d8-4f38886d03ef\" (UID: \"b454fd31-3cf9-43f3-80d8-4f38886d03ef\") " Jul 15 17:07:50.923812 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:50.923799 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b454fd31-3cf9-43f3-80d8-4f38886d03ef-bundle\") pod \"b454fd31-3cf9-43f3-80d8-4f38886d03ef\" (UID: \"b454fd31-3cf9-43f3-80d8-4f38886d03ef\") " Jul 15 17:07:50.924188 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:50.924166 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b454fd31-3cf9-43f3-80d8-4f38886d03ef-bundle" (OuterVolumeSpecName: "bundle") pod "b454fd31-3cf9-43f3-80d8-4f38886d03ef" (UID: "b454fd31-3cf9-43f3-80d8-4f38886d03ef"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:07:50.925648 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:50.925629 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b454fd31-3cf9-43f3-80d8-4f38886d03ef-kube-api-access-jntdf" (OuterVolumeSpecName: "kube-api-access-jntdf") pod "b454fd31-3cf9-43f3-80d8-4f38886d03ef" (UID: "b454fd31-3cf9-43f3-80d8-4f38886d03ef"). InnerVolumeSpecName "kube-api-access-jntdf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:07:50.929549 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:50.929510 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b454fd31-3cf9-43f3-80d8-4f38886d03ef-util" (OuterVolumeSpecName: "util") pod "b454fd31-3cf9-43f3-80d8-4f38886d03ef" (UID: "b454fd31-3cf9-43f3-80d8-4f38886d03ef"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:07:51.024396 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:51.024363 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b454fd31-3cf9-43f3-80d8-4f38886d03ef-util\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:07:51.024396 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:51.024396 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b454fd31-3cf9-43f3-80d8-4f38886d03ef-bundle\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:07:51.024582 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:51.024407 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jntdf\" (UniqueName: \"kubernetes.io/projected/b454fd31-3cf9-43f3-80d8-4f38886d03ef-kube-api-access-jntdf\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:07:51.702294 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:51.702258 2578 generic.go:356] "Generic (PLEG): container finished" podID="039839f4-8025-477b-b8d7-8137daafc9d9" containerID="6cda04986d9d839aea4011b2f0be8a949ea626c8b18943547eedebb9ce13e5fe" exitCode=0 Jul 15 17:07:51.702685 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:51.702338 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z" event={"ID":"039839f4-8025-477b-b8d7-8137daafc9d9","Type":"ContainerDied","Data":"6cda04986d9d839aea4011b2f0be8a949ea626c8b18943547eedebb9ce13e5fe"} Jul 15 17:07:51.704278 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:51.704256 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd" Jul 15 17:07:51.704372 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:51.704285 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd" event={"ID":"b454fd31-3cf9-43f3-80d8-4f38886d03ef","Type":"ContainerDied","Data":"1f85fd50b3d4aeb27d61f95cb902d526401f977b63644dc742bdb2d48cd0aecb"} Jul 15 17:07:51.704372 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:51.704321 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1f85fd50b3d4aeb27d61f95cb902d526401f977b63644dc742bdb2d48cd0aecb" Jul 15 17:07:51.834290 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:51.834264 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn" Jul 15 17:07:51.930898 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:51.930813 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j4pdd\" (UniqueName: \"kubernetes.io/projected/2795c4a6-2feb-470e-b29f-8ac8c291b4cc-kube-api-access-j4pdd\") pod \"2795c4a6-2feb-470e-b29f-8ac8c291b4cc\" (UID: \"2795c4a6-2feb-470e-b29f-8ac8c291b4cc\") " Jul 15 17:07:51.930898 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:51.930867 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2795c4a6-2feb-470e-b29f-8ac8c291b4cc-bundle\") pod \"2795c4a6-2feb-470e-b29f-8ac8c291b4cc\" (UID: \"2795c4a6-2feb-470e-b29f-8ac8c291b4cc\") " Jul 15 17:07:51.931123 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:51.930909 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2795c4a6-2feb-470e-b29f-8ac8c291b4cc-util\") pod \"2795c4a6-2feb-470e-b29f-8ac8c291b4cc\" (UID: \"2795c4a6-2feb-470e-b29f-8ac8c291b4cc\") " Jul 15 17:07:51.931458 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:51.931436 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2795c4a6-2feb-470e-b29f-8ac8c291b4cc-bundle" (OuterVolumeSpecName: "bundle") pod "2795c4a6-2feb-470e-b29f-8ac8c291b4cc" (UID: "2795c4a6-2feb-470e-b29f-8ac8c291b4cc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:07:51.932971 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:51.932947 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2795c4a6-2feb-470e-b29f-8ac8c291b4cc-kube-api-access-j4pdd" (OuterVolumeSpecName: "kube-api-access-j4pdd") pod "2795c4a6-2feb-470e-b29f-8ac8c291b4cc" (UID: "2795c4a6-2feb-470e-b29f-8ac8c291b4cc"). InnerVolumeSpecName "kube-api-access-j4pdd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:07:51.935528 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:51.935508 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2795c4a6-2feb-470e-b29f-8ac8c291b4cc-util" (OuterVolumeSpecName: "util") pod "2795c4a6-2feb-470e-b29f-8ac8c291b4cc" (UID: "2795c4a6-2feb-470e-b29f-8ac8c291b4cc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:07:52.031755 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:52.031721 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4pdd\" (UniqueName: \"kubernetes.io/projected/2795c4a6-2feb-470e-b29f-8ac8c291b4cc-kube-api-access-j4pdd\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:07:52.031755 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:52.031750 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2795c4a6-2feb-470e-b29f-8ac8c291b4cc-bundle\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:07:52.031755 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:52.031760 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2795c4a6-2feb-470e-b29f-8ac8c291b4cc-util\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:07:52.717455 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:52.717427 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn" Jul 15 17:07:52.717829 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:52.717426 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn" event={"ID":"2795c4a6-2feb-470e-b29f-8ac8c291b4cc","Type":"ContainerDied","Data":"ae6ca0d65af88b1a8a1d9818e88335a13d647ecb6aaedc66d82100720ddfc1b8"} Jul 15 17:07:52.717829 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:52.717591 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae6ca0d65af88b1a8a1d9818e88335a13d647ecb6aaedc66d82100720ddfc1b8" Jul 15 17:07:52.846477 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:52.846451 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z" Jul 15 17:07:52.939457 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:52.939425 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/039839f4-8025-477b-b8d7-8137daafc9d9-util\") pod \"039839f4-8025-477b-b8d7-8137daafc9d9\" (UID: \"039839f4-8025-477b-b8d7-8137daafc9d9\") " Jul 15 17:07:52.939457 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:52.939456 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/039839f4-8025-477b-b8d7-8137daafc9d9-bundle\") pod \"039839f4-8025-477b-b8d7-8137daafc9d9\" (UID: \"039839f4-8025-477b-b8d7-8137daafc9d9\") " Jul 15 17:07:52.939670 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:52.939535 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-blck2\" (UniqueName: \"kubernetes.io/projected/039839f4-8025-477b-b8d7-8137daafc9d9-kube-api-access-blck2\") pod \"039839f4-8025-477b-b8d7-8137daafc9d9\" (UID: \"039839f4-8025-477b-b8d7-8137daafc9d9\") " Jul 15 17:07:52.940121 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:52.940093 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/039839f4-8025-477b-b8d7-8137daafc9d9-bundle" (OuterVolumeSpecName: "bundle") pod "039839f4-8025-477b-b8d7-8137daafc9d9" (UID: "039839f4-8025-477b-b8d7-8137daafc9d9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:07:52.941590 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:52.941569 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/039839f4-8025-477b-b8d7-8137daafc9d9-kube-api-access-blck2" (OuterVolumeSpecName: "kube-api-access-blck2") pod "039839f4-8025-477b-b8d7-8137daafc9d9" (UID: "039839f4-8025-477b-b8d7-8137daafc9d9"). InnerVolumeSpecName "kube-api-access-blck2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:07:52.944096 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:52.944072 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/039839f4-8025-477b-b8d7-8137daafc9d9-util" (OuterVolumeSpecName: "util") pod "039839f4-8025-477b-b8d7-8137daafc9d9" (UID: "039839f4-8025-477b-b8d7-8137daafc9d9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:07:53.040286 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:53.040256 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-blck2\" (UniqueName: \"kubernetes.io/projected/039839f4-8025-477b-b8d7-8137daafc9d9-kube-api-access-blck2\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:07:53.040286 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:53.040280 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/039839f4-8025-477b-b8d7-8137daafc9d9-util\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:07:53.040286 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:53.040290 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/039839f4-8025-477b-b8d7-8137daafc9d9-bundle\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:07:53.724376 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:53.724348 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z" Jul 15 17:07:53.724762 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:53.724338 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z" event={"ID":"039839f4-8025-477b-b8d7-8137daafc9d9","Type":"ContainerDied","Data":"4ea3c954b7f0156725aa3cfbdcb0b8f060d39b7520db1838ba17c36fac526cc7"} Jul 15 17:07:53.724762 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:53.724474 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4ea3c954b7f0156725aa3cfbdcb0b8f060d39b7520db1838ba17c36fac526cc7" Jul 15 17:07:53.983009 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:53.982926 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:07:57.736261 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:07:57.736223 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:01.176336 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:01.176298 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:03.122195 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:03.122156 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:04.959115 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:04.959081 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:05.110473 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:05.110437 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:08.799014 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:08.798980 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:08.898460 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:08.898413 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:11.478181 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:11.478130 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:12.715723 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:12.715690 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:12.785722 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:12.785686 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:13.095195 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:13.095110 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:13.296324 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:13.296291 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:19.266927 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:19.266854 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:19.372758 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:19.372692 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:19.438272 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:19.438240 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:20.374014 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:20.373986 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:20.438056 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:20.438023 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:21.922142 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:21.922111 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:21.988034 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:21.987995 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:24.090321 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.090284 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vk82h"] Jul 15 17:08:24.090714 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.090629 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="720f778f-b686-46d0-81b9-c85c5f3db32c" containerName="util" Jul 15 17:08:24.090714 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.090640 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="720f778f-b686-46d0-81b9-c85c5f3db32c" containerName="util" Jul 15 17:08:24.090714 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.090648 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="720f778f-b686-46d0-81b9-c85c5f3db32c" containerName="extract" Jul 15 17:08:24.090714 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.090653 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="720f778f-b686-46d0-81b9-c85c5f3db32c" containerName="extract" Jul 15 17:08:24.090714 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.090661 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2795c4a6-2feb-470e-b29f-8ac8c291b4cc" containerName="pull" Jul 15 17:08:24.090714 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.090667 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2795c4a6-2feb-470e-b29f-8ac8c291b4cc" containerName="pull" Jul 15 17:08:24.090714 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.090674 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="039839f4-8025-477b-b8d7-8137daafc9d9" containerName="util" Jul 15 17:08:24.090714 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.090681 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="039839f4-8025-477b-b8d7-8137daafc9d9" containerName="util" Jul 15 17:08:24.090714 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.090686 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b454fd31-3cf9-43f3-80d8-4f38886d03ef" containerName="pull" Jul 15 17:08:24.090714 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.090691 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b454fd31-3cf9-43f3-80d8-4f38886d03ef" containerName="pull" Jul 15 17:08:24.090714 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.090700 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="039839f4-8025-477b-b8d7-8137daafc9d9" containerName="pull" Jul 15 17:08:24.090714 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.090704 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="039839f4-8025-477b-b8d7-8137daafc9d9" containerName="pull" Jul 15 17:08:24.090714 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.090712 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="039839f4-8025-477b-b8d7-8137daafc9d9" containerName="extract" Jul 15 17:08:24.090714 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.090717 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="039839f4-8025-477b-b8d7-8137daafc9d9" containerName="extract" Jul 15 17:08:24.091148 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.090723 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="720f778f-b686-46d0-81b9-c85c5f3db32c" containerName="pull" Jul 15 17:08:24.091148 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.090728 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="720f778f-b686-46d0-81b9-c85c5f3db32c" containerName="pull" Jul 15 17:08:24.091148 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.090738 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2795c4a6-2feb-470e-b29f-8ac8c291b4cc" containerName="extract" Jul 15 17:08:24.091148 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.090744 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2795c4a6-2feb-470e-b29f-8ac8c291b4cc" containerName="extract" Jul 15 17:08:24.091148 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.090750 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b454fd31-3cf9-43f3-80d8-4f38886d03ef" containerName="extract" Jul 15 17:08:24.091148 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.090755 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b454fd31-3cf9-43f3-80d8-4f38886d03ef" containerName="extract" Jul 15 17:08:24.091148 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.090762 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2795c4a6-2feb-470e-b29f-8ac8c291b4cc" containerName="util" Jul 15 17:08:24.091148 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.090766 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2795c4a6-2feb-470e-b29f-8ac8c291b4cc" containerName="util" Jul 15 17:08:24.091148 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.090771 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b454fd31-3cf9-43f3-80d8-4f38886d03ef" containerName="util" Jul 15 17:08:24.091148 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.090776 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b454fd31-3cf9-43f3-80d8-4f38886d03ef" containerName="util" Jul 15 17:08:24.091148 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.090830 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="039839f4-8025-477b-b8d7-8137daafc9d9" containerName="extract" Jul 15 17:08:24.091148 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.090836 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="720f778f-b686-46d0-81b9-c85c5f3db32c" containerName="extract" Jul 15 17:08:24.091148 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.090843 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2795c4a6-2feb-470e-b29f-8ac8c291b4cc" containerName="extract" Jul 15 17:08:24.091148 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.090850 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b454fd31-3cf9-43f3-80d8-4f38886d03ef" containerName="extract" Jul 15 17:08:24.095617 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.095580 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vk82h" Jul 15 17:08:24.098136 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.098109 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 15 17:08:24.098264 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.098156 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 15 17:08:24.098264 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.098204 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-984tg\"" Jul 15 17:08:24.105486 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.105466 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vk82h"] Jul 15 17:08:24.213446 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.213414 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/08280924-0060-46e0-854d-ccfd543d4ab2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vk82h\" (UID: \"08280924-0060-46e0-854d-ccfd543d4ab2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vk82h" Jul 15 17:08:24.213702 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.213535 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rncz9\" (UniqueName: \"kubernetes.io/projected/08280924-0060-46e0-854d-ccfd543d4ab2-kube-api-access-rncz9\") pod \"kuadrant-operator-controller-manager-86878c769f-vk82h\" (UID: \"08280924-0060-46e0-854d-ccfd543d4ab2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vk82h" Jul 15 17:08:24.314155 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.314119 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/08280924-0060-46e0-854d-ccfd543d4ab2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vk82h\" (UID: \"08280924-0060-46e0-854d-ccfd543d4ab2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vk82h" Jul 15 17:08:24.314326 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.314198 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rncz9\" (UniqueName: \"kubernetes.io/projected/08280924-0060-46e0-854d-ccfd543d4ab2-kube-api-access-rncz9\") pod \"kuadrant-operator-controller-manager-86878c769f-vk82h\" (UID: \"08280924-0060-46e0-854d-ccfd543d4ab2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vk82h" Jul 15 17:08:24.314571 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.314549 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/08280924-0060-46e0-854d-ccfd543d4ab2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vk82h\" (UID: \"08280924-0060-46e0-854d-ccfd543d4ab2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vk82h" Jul 15 17:08:24.323076 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.323040 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rncz9\" (UniqueName: \"kubernetes.io/projected/08280924-0060-46e0-854d-ccfd543d4ab2-kube-api-access-rncz9\") pod \"kuadrant-operator-controller-manager-86878c769f-vk82h\" (UID: \"08280924-0060-46e0-854d-ccfd543d4ab2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vk82h" Jul 15 17:08:24.407133 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.407048 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vk82h" Jul 15 17:08:24.549850 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.549811 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vk82h"] Jul 15 17:08:24.553093 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:08:24.553068 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08280924_0060_46e0_854d_ccfd543d4ab2.slice/crio-e3ca18d7ad199e26bd61186c861d5b5c57b9928a4d01593e57223ba291056aa9 WatchSource:0}: Error finding container e3ca18d7ad199e26bd61186c861d5b5c57b9928a4d01593e57223ba291056aa9: Status 404 returned error can't find the container with id e3ca18d7ad199e26bd61186c861d5b5c57b9928a4d01593e57223ba291056aa9 Jul 15 17:08:24.878191 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:24.878159 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vk82h" event={"ID":"08280924-0060-46e0-854d-ccfd543d4ab2","Type":"ContainerStarted","Data":"e3ca18d7ad199e26bd61186c861d5b5c57b9928a4d01593e57223ba291056aa9"} Jul 15 17:08:25.150989 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:25.150891 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:25.242128 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:25.242097 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:29.922793 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:29.922757 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:30.023533 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:30.023501 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:30.911570 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:30.911480 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vk82h" event={"ID":"08280924-0060-46e0-854d-ccfd543d4ab2","Type":"ContainerStarted","Data":"a7a9eae24062fd67c8f5382d34bbaf049e4e379585a06449f690cf03706ef3bc"} Jul 15 17:08:30.911734 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:30.911586 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vk82h" Jul 15 17:08:30.940490 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:30.940442 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vk82h" podStartSLOduration=0.85589561 podStartE2EDuration="6.940427516s" totalImagesPullingTime="6.084531906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:08:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:08:30.937860605 +0000 UTC m=+728.490990843" watchObservedRunningTime="2026-07-15 17:08:30.940427516 +0000 UTC m=+728.493557795" Jul 15 17:08:36.038446 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:36.038410 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:36.141556 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:36.141524 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:41.335890 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:41.335856 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:41.440414 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:41.440382 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:41.919041 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:41.919006 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vk82h" Jul 15 17:08:45.538975 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:45.538940 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:45.639748 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:45.639710 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:50.405982 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:50.405944 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:50.503484 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:50.503452 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:58.401825 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:58.401794 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:58.604482 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:08:58.604445 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:09:00.314532 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:00.314500 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:09:00.436270 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:00.436241 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:09:00.520847 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:00.520814 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:09:01.195010 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:01.194977 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:09:01.288556 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:01.288523 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:09:01.292727 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:01.292699 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-lf9fp"] Jul 15 17:09:01.295044 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:01.295029 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-lf9fp" Jul 15 17:09:01.297662 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:01.297643 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-jhq54\"" Jul 15 17:09:01.305247 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:01.305226 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-lf9fp"] Jul 15 17:09:01.424201 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:01.424171 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chv4n\" (UniqueName: \"kubernetes.io/projected/07b76ab7-680c-467b-a162-7a8fc7257cff-kube-api-access-chv4n\") pod \"authorino-75d556bd77-lf9fp\" (UID: \"07b76ab7-680c-467b-a162-7a8fc7257cff\") " pod="kuadrant-system/authorino-75d556bd77-lf9fp" Jul 15 17:09:01.525590 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:01.525502 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chv4n\" (UniqueName: \"kubernetes.io/projected/07b76ab7-680c-467b-a162-7a8fc7257cff-kube-api-access-chv4n\") pod \"authorino-75d556bd77-lf9fp\" (UID: \"07b76ab7-680c-467b-a162-7a8fc7257cff\") " pod="kuadrant-system/authorino-75d556bd77-lf9fp" Jul 15 17:09:01.533903 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:01.533880 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chv4n\" (UniqueName: \"kubernetes.io/projected/07b76ab7-680c-467b-a162-7a8fc7257cff-kube-api-access-chv4n\") pod \"authorino-75d556bd77-lf9fp\" (UID: \"07b76ab7-680c-467b-a162-7a8fc7257cff\") " pod="kuadrant-system/authorino-75d556bd77-lf9fp" Jul 15 17:09:01.605885 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:01.605851 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-lf9fp" Jul 15 17:09:01.735720 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:01.735695 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-lf9fp"] Jul 15 17:09:01.737095 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:09:01.737069 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07b76ab7_680c_467b_a162_7a8fc7257cff.slice/crio-7e7b19e1db880b5b896bfce24201ea7339108fd3406a2606dd68bc140d25be3c WatchSource:0}: Error finding container 7e7b19e1db880b5b896bfce24201ea7339108fd3406a2606dd68bc140d25be3c: Status 404 returned error can't find the container with id 7e7b19e1db880b5b896bfce24201ea7339108fd3406a2606dd68bc140d25be3c Jul 15 17:09:02.073526 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:02.073486 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-lf9fp" event={"ID":"07b76ab7-680c-467b-a162-7a8fc7257cff","Type":"ContainerStarted","Data":"7e7b19e1db880b5b896bfce24201ea7339108fd3406a2606dd68bc140d25be3c"} Jul 15 17:09:05.092864 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:05.092829 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-lf9fp" event={"ID":"07b76ab7-680c-467b-a162-7a8fc7257cff","Type":"ContainerStarted","Data":"ccd324b401b9608280566a981c02f8a55861365761d5e0d83f2ae0b7051885a8"} Jul 15 17:09:05.108404 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:05.108357 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-lf9fp" podStartSLOduration=0.881888147 podStartE2EDuration="4.10834185s" totalImagesPullingTime="3.226453703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:09:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:09:05.107374568 +0000 UTC m=+762.660504783" watchObservedRunningTime="2026-07-15 17:09:05.10834185 +0000 UTC m=+762.661472063" Jul 15 17:09:21.556369 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:21.556333 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:09:21.655700 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:21.655663 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:09:22.049846 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:22.049813 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:09:34.398020 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:34.397985 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:09:34.440116 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:34.440086 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-77f46567d8-x6zj6"] Jul 15 17:09:34.443984 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:34.443962 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-77f46567d8-x6zj6" Jul 15 17:09:34.446301 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:34.446276 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 15 17:09:34.450676 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:34.450656 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-77f46567d8-x6zj6"] Jul 15 17:09:34.472876 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:34.472849 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:09:34.506935 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:34.506909 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jvxg\" (UniqueName: \"kubernetes.io/projected/096da392-ec64-43b2-8c74-d48fe112dc43-kube-api-access-6jvxg\") pod \"authorino-77f46567d8-x6zj6\" (UID: \"096da392-ec64-43b2-8c74-d48fe112dc43\") " pod="kuadrant-system/authorino-77f46567d8-x6zj6" Jul 15 17:09:34.507066 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:34.506946 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/096da392-ec64-43b2-8c74-d48fe112dc43-tls-cert\") pod \"authorino-77f46567d8-x6zj6\" (UID: \"096da392-ec64-43b2-8c74-d48fe112dc43\") " pod="kuadrant-system/authorino-77f46567d8-x6zj6" Jul 15 17:09:34.607801 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:34.607768 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jvxg\" (UniqueName: \"kubernetes.io/projected/096da392-ec64-43b2-8c74-d48fe112dc43-kube-api-access-6jvxg\") pod \"authorino-77f46567d8-x6zj6\" (UID: \"096da392-ec64-43b2-8c74-d48fe112dc43\") " pod="kuadrant-system/authorino-77f46567d8-x6zj6" Jul 15 17:09:34.607960 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:34.607812 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/096da392-ec64-43b2-8c74-d48fe112dc43-tls-cert\") pod \"authorino-77f46567d8-x6zj6\" (UID: \"096da392-ec64-43b2-8c74-d48fe112dc43\") " pod="kuadrant-system/authorino-77f46567d8-x6zj6" Jul 15 17:09:34.610192 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:34.610164 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/096da392-ec64-43b2-8c74-d48fe112dc43-tls-cert\") pod \"authorino-77f46567d8-x6zj6\" (UID: \"096da392-ec64-43b2-8c74-d48fe112dc43\") " pod="kuadrant-system/authorino-77f46567d8-x6zj6" Jul 15 17:09:34.616613 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:34.616581 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jvxg\" (UniqueName: \"kubernetes.io/projected/096da392-ec64-43b2-8c74-d48fe112dc43-kube-api-access-6jvxg\") pod \"authorino-77f46567d8-x6zj6\" (UID: \"096da392-ec64-43b2-8c74-d48fe112dc43\") " pod="kuadrant-system/authorino-77f46567d8-x6zj6" Jul 15 17:09:34.753718 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:34.753683 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-77f46567d8-x6zj6" Jul 15 17:09:34.883218 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:34.883193 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-77f46567d8-x6zj6"] Jul 15 17:09:34.884952 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:09:34.884925 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod096da392_ec64_43b2_8c74_d48fe112dc43.slice/crio-d726cb0fa827c3264d5f53135133091ee63196f9b7d5798b05205e8e66466f7d WatchSource:0}: Error finding container d726cb0fa827c3264d5f53135133091ee63196f9b7d5798b05205e8e66466f7d: Status 404 returned error can't find the container with id d726cb0fa827c3264d5f53135133091ee63196f9b7d5798b05205e8e66466f7d Jul 15 17:09:35.243422 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:35.243389 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-77f46567d8-x6zj6" event={"ID":"096da392-ec64-43b2-8c74-d48fe112dc43","Type":"ContainerStarted","Data":"d726cb0fa827c3264d5f53135133091ee63196f9b7d5798b05205e8e66466f7d"} Jul 15 17:09:38.260370 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:38.260333 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-77f46567d8-x6zj6" event={"ID":"096da392-ec64-43b2-8c74-d48fe112dc43","Type":"ContainerStarted","Data":"bd412a4457f9a151998fc58664a615b621981ef9fae9a1eaa63b2d44476540b4"} Jul 15 17:09:38.279052 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:38.279003 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-77f46567d8-x6zj6" podStartSLOduration=1.731050164 podStartE2EDuration="4.27898894s" totalImagesPullingTime="2.547938776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:09:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:09:38.27717024 +0000 UTC m=+795.830300456" watchObservedRunningTime="2026-07-15 17:09:38.27898894 +0000 UTC m=+795.832119153" Jul 15 17:09:38.307276 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:38.307246 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-lf9fp"] Jul 15 17:09:38.307474 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:38.307452 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-lf9fp" podUID="07b76ab7-680c-467b-a162-7a8fc7257cff" containerName="authorino" containerID="cri-o://ccd324b401b9608280566a981c02f8a55861365761d5e0d83f2ae0b7051885a8" gracePeriod=30 Jul 15 17:09:38.573057 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:38.573035 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-lf9fp" Jul 15 17:09:38.636637 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:38.636583 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-chv4n\" (UniqueName: \"kubernetes.io/projected/07b76ab7-680c-467b-a162-7a8fc7257cff-kube-api-access-chv4n\") pod \"07b76ab7-680c-467b-a162-7a8fc7257cff\" (UID: \"07b76ab7-680c-467b-a162-7a8fc7257cff\") " Jul 15 17:09:38.638585 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:38.638557 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07b76ab7-680c-467b-a162-7a8fc7257cff-kube-api-access-chv4n" (OuterVolumeSpecName: "kube-api-access-chv4n") pod "07b76ab7-680c-467b-a162-7a8fc7257cff" (UID: "07b76ab7-680c-467b-a162-7a8fc7257cff"). InnerVolumeSpecName "kube-api-access-chv4n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:09:38.737244 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:38.737208 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-chv4n\" (UniqueName: \"kubernetes.io/projected/07b76ab7-680c-467b-a162-7a8fc7257cff-kube-api-access-chv4n\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:09:39.266103 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:39.266019 2578 generic.go:356] "Generic (PLEG): container finished" podID="07b76ab7-680c-467b-a162-7a8fc7257cff" containerID="ccd324b401b9608280566a981c02f8a55861365761d5e0d83f2ae0b7051885a8" exitCode=0 Jul 15 17:09:39.266103 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:39.266059 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-lf9fp" Jul 15 17:09:39.266103 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:39.266076 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-lf9fp" event={"ID":"07b76ab7-680c-467b-a162-7a8fc7257cff","Type":"ContainerDied","Data":"ccd324b401b9608280566a981c02f8a55861365761d5e0d83f2ae0b7051885a8"} Jul 15 17:09:39.266667 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:39.266126 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-lf9fp" event={"ID":"07b76ab7-680c-467b-a162-7a8fc7257cff","Type":"ContainerDied","Data":"7e7b19e1db880b5b896bfce24201ea7339108fd3406a2606dd68bc140d25be3c"} Jul 15 17:09:39.266667 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:39.266146 2578 scope.go:117] "RemoveContainer" containerID="ccd324b401b9608280566a981c02f8a55861365761d5e0d83f2ae0b7051885a8" Jul 15 17:09:39.274841 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:39.274821 2578 scope.go:117] "RemoveContainer" containerID="ccd324b401b9608280566a981c02f8a55861365761d5e0d83f2ae0b7051885a8" Jul 15 17:09:39.275094 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:09:39.275059 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccd324b401b9608280566a981c02f8a55861365761d5e0d83f2ae0b7051885a8\": container with ID starting with ccd324b401b9608280566a981c02f8a55861365761d5e0d83f2ae0b7051885a8 not found: ID does not exist" containerID="ccd324b401b9608280566a981c02f8a55861365761d5e0d83f2ae0b7051885a8" Jul 15 17:09:39.275094 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:39.275081 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccd324b401b9608280566a981c02f8a55861365761d5e0d83f2ae0b7051885a8"} err="failed to get container status \"ccd324b401b9608280566a981c02f8a55861365761d5e0d83f2ae0b7051885a8\": rpc error: code = NotFound desc = could not find container \"ccd324b401b9608280566a981c02f8a55861365761d5e0d83f2ae0b7051885a8\": container with ID starting with ccd324b401b9608280566a981c02f8a55861365761d5e0d83f2ae0b7051885a8 not found: ID does not exist" Jul 15 17:09:39.283671 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:39.283650 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-lf9fp"] Jul 15 17:09:39.287677 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:39.287658 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-lf9fp"] Jul 15 17:09:40.990424 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:40.990390 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07b76ab7-680c-467b-a162-7a8fc7257cff" path="/var/lib/kubelet/pods/07b76ab7-680c-467b-a162-7a8fc7257cff/volumes" Jul 15 17:09:42.970641 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:42.970579 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:09:43.117485 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:43.117420 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:09:43.217793 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:43.217753 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:09:46.355853 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:46.355762 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:09:46.453763 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:09:46.453731 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:10:19.517755 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:10:19.517720 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:10:19.621080 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:10:19.621036 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:10:20.040155 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:10:20.040116 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:10:20.199770 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:10:20.199739 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:10:20.538614 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:10:20.538536 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:10:20.694159 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:10:20.694113 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:10:30.793434 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:10:30.793393 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:10:30.892575 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:10:30.892539 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:10:55.143440 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:10:55.143407 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:10:55.244667 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:10:55.244635 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:10:56.120211 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:10:56.120176 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:11:05.993868 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:05.993834 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:11:12.903091 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:12.903057 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:11:18.198628 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:18.198500 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64"] Jul 15 17:11:18.199050 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:18.198982 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07b76ab7-680c-467b-a162-7a8fc7257cff" containerName="authorino" Jul 15 17:11:18.199050 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:18.198996 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="07b76ab7-680c-467b-a162-7a8fc7257cff" containerName="authorino" Jul 15 17:11:18.199165 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:18.199087 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="07b76ab7-680c-467b-a162-7a8fc7257cff" containerName="authorino" Jul 15 17:11:18.201210 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:18.201190 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64" Jul 15 17:11:18.204942 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:18.204920 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-k5dwz\"" Jul 15 17:11:18.205069 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:18.204931 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 15 17:11:18.205174 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:18.205160 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 15 17:11:18.205244 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:18.205202 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 15 17:11:18.214921 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:18.214895 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64"] Jul 15 17:11:18.240881 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:18.240854 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:11:18.381110 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:18.381074 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b541d6ad-49d6-4731-8a19-3f3ca904ac51-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64\" (UID: \"b541d6ad-49d6-4731-8a19-3f3ca904ac51\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64" Jul 15 17:11:18.381110 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:18.381107 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b541d6ad-49d6-4731-8a19-3f3ca904ac51-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64\" (UID: \"b541d6ad-49d6-4731-8a19-3f3ca904ac51\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64" Jul 15 17:11:18.381325 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:18.381144 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b541d6ad-49d6-4731-8a19-3f3ca904ac51-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64\" (UID: \"b541d6ad-49d6-4731-8a19-3f3ca904ac51\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64" Jul 15 17:11:18.381325 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:18.381235 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2txr2\" (UniqueName: \"kubernetes.io/projected/b541d6ad-49d6-4731-8a19-3f3ca904ac51-kube-api-access-2txr2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64\" (UID: \"b541d6ad-49d6-4731-8a19-3f3ca904ac51\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64" Jul 15 17:11:18.381325 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:18.381274 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b541d6ad-49d6-4731-8a19-3f3ca904ac51-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64\" (UID: \"b541d6ad-49d6-4731-8a19-3f3ca904ac51\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64" Jul 15 17:11:18.381443 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:18.381343 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b541d6ad-49d6-4731-8a19-3f3ca904ac51-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64\" (UID: \"b541d6ad-49d6-4731-8a19-3f3ca904ac51\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64" Jul 15 17:11:18.481997 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:18.481899 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b541d6ad-49d6-4731-8a19-3f3ca904ac51-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64\" (UID: \"b541d6ad-49d6-4731-8a19-3f3ca904ac51\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64" Jul 15 17:11:18.481997 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:18.481943 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b541d6ad-49d6-4731-8a19-3f3ca904ac51-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64\" (UID: \"b541d6ad-49d6-4731-8a19-3f3ca904ac51\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64" Jul 15 17:11:18.482252 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:18.482063 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b541d6ad-49d6-4731-8a19-3f3ca904ac51-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64\" (UID: \"b541d6ad-49d6-4731-8a19-3f3ca904ac51\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64" Jul 15 17:11:18.482252 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:18.482120 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2txr2\" (UniqueName: \"kubernetes.io/projected/b541d6ad-49d6-4731-8a19-3f3ca904ac51-kube-api-access-2txr2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64\" (UID: \"b541d6ad-49d6-4731-8a19-3f3ca904ac51\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64" Jul 15 17:11:18.482252 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:18.482154 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b541d6ad-49d6-4731-8a19-3f3ca904ac51-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64\" (UID: \"b541d6ad-49d6-4731-8a19-3f3ca904ac51\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64" Jul 15 17:11:18.482252 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:18.482181 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b541d6ad-49d6-4731-8a19-3f3ca904ac51-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64\" (UID: \"b541d6ad-49d6-4731-8a19-3f3ca904ac51\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64" Jul 15 17:11:18.482428 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:18.482289 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b541d6ad-49d6-4731-8a19-3f3ca904ac51-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64\" (UID: \"b541d6ad-49d6-4731-8a19-3f3ca904ac51\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64" Jul 15 17:11:18.482428 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:18.482397 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b541d6ad-49d6-4731-8a19-3f3ca904ac51-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64\" (UID: \"b541d6ad-49d6-4731-8a19-3f3ca904ac51\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64" Jul 15 17:11:18.482515 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:18.482451 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b541d6ad-49d6-4731-8a19-3f3ca904ac51-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64\" (UID: \"b541d6ad-49d6-4731-8a19-3f3ca904ac51\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64" Jul 15 17:11:18.484237 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:18.484206 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b541d6ad-49d6-4731-8a19-3f3ca904ac51-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64\" (UID: \"b541d6ad-49d6-4731-8a19-3f3ca904ac51\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64" Jul 15 17:11:18.484641 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:18.484619 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b541d6ad-49d6-4731-8a19-3f3ca904ac51-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64\" (UID: \"b541d6ad-49d6-4731-8a19-3f3ca904ac51\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64" Jul 15 17:11:18.492656 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:18.492635 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2txr2\" (UniqueName: \"kubernetes.io/projected/b541d6ad-49d6-4731-8a19-3f3ca904ac51-kube-api-access-2txr2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64\" (UID: \"b541d6ad-49d6-4731-8a19-3f3ca904ac51\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64" Jul 15 17:11:18.530488 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:18.530463 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64" Jul 15 17:11:18.662879 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:18.662854 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64"] Jul 15 17:11:18.664469 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:11:18.664439 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb541d6ad_49d6_4731_8a19_3f3ca904ac51.slice/crio-7f54affe65b549e1dc6cc0f62a9e71d5d9179c549eb831c03e208ac2030eebf8 WatchSource:0}: Error finding container 7f54affe65b549e1dc6cc0f62a9e71d5d9179c549eb831c03e208ac2030eebf8: Status 404 returned error can't find the container with id 7f54affe65b549e1dc6cc0f62a9e71d5d9179c549eb831c03e208ac2030eebf8 Jul 15 17:11:18.787307 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:18.787228 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64" event={"ID":"b541d6ad-49d6-4731-8a19-3f3ca904ac51","Type":"ContainerStarted","Data":"7f54affe65b549e1dc6cc0f62a9e71d5d9179c549eb831c03e208ac2030eebf8"} Jul 15 17:11:19.794255 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:19.794217 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64" event={"ID":"b541d6ad-49d6-4731-8a19-3f3ca904ac51","Type":"ContainerStarted","Data":"56c6d9f3070b00e2fd3de2ed3ad06e8a7df56036c9e068f81b18f552f9738b5f"} Jul 15 17:11:19.811732 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:19.811687 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64" podStartSLOduration=0.783179213 podStartE2EDuration="1.811670614s" totalImagesPullingTime="1.028491401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:11:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:11:19.810463719 +0000 UTC m=+897.363593956" watchObservedRunningTime="2026-07-15 17:11:19.811670614 +0000 UTC m=+897.364800829" Jul 15 17:11:23.515612 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:23.515548 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:11:28.530986 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:28.530950 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64" Jul 15 17:11:28.533420 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:28.531017 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64" Jul 15 17:11:28.544656 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:28.544636 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64" Jul 15 17:11:28.854622 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:28.854521 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64" Jul 15 17:11:29.127489 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:29.127402 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29wtx_88f8928c-237e-457f-a032-707f48111f36/ovn-acl-logging/0.log" Jul 15 17:11:29.127759 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:29.127740 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29wtx_88f8928c-237e-457f-a032-707f48111f36/ovn-acl-logging/0.log" Jul 15 17:11:35.854222 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:35.854190 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:11:40.950677 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:11:40.950588 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:12:33.628946 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:12:33.628905 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcb78779c-85hpg"] Jul 15 17:12:33.632533 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:12:33.632507 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcb78779c-85hpg" Jul 15 17:12:33.641424 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:12:33.641402 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcb78779c-85hpg"] Jul 15 17:12:33.739062 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:12:33.739016 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2fbbec08-4030-4af6-8f8e-4ccc6c783a49-tls-cert\") pod \"authorino-7bcb78779c-85hpg\" (UID: \"2fbbec08-4030-4af6-8f8e-4ccc6c783a49\") " pod="kuadrant-system/authorino-7bcb78779c-85hpg" Jul 15 17:12:33.739263 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:12:33.739122 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2l5s\" (UniqueName: \"kubernetes.io/projected/2fbbec08-4030-4af6-8f8e-4ccc6c783a49-kube-api-access-w2l5s\") pod \"authorino-7bcb78779c-85hpg\" (UID: \"2fbbec08-4030-4af6-8f8e-4ccc6c783a49\") " pod="kuadrant-system/authorino-7bcb78779c-85hpg" Jul 15 17:12:33.839971 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:12:33.839937 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2fbbec08-4030-4af6-8f8e-4ccc6c783a49-tls-cert\") pod \"authorino-7bcb78779c-85hpg\" (UID: \"2fbbec08-4030-4af6-8f8e-4ccc6c783a49\") " pod="kuadrant-system/authorino-7bcb78779c-85hpg" Jul 15 17:12:33.840134 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:12:33.839997 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2l5s\" (UniqueName: \"kubernetes.io/projected/2fbbec08-4030-4af6-8f8e-4ccc6c783a49-kube-api-access-w2l5s\") pod \"authorino-7bcb78779c-85hpg\" (UID: \"2fbbec08-4030-4af6-8f8e-4ccc6c783a49\") " pod="kuadrant-system/authorino-7bcb78779c-85hpg" Jul 15 17:12:33.842528 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:12:33.842502 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2fbbec08-4030-4af6-8f8e-4ccc6c783a49-tls-cert\") pod \"authorino-7bcb78779c-85hpg\" (UID: \"2fbbec08-4030-4af6-8f8e-4ccc6c783a49\") " pod="kuadrant-system/authorino-7bcb78779c-85hpg" Jul 15 17:12:33.848558 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:12:33.848540 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2l5s\" (UniqueName: \"kubernetes.io/projected/2fbbec08-4030-4af6-8f8e-4ccc6c783a49-kube-api-access-w2l5s\") pod \"authorino-7bcb78779c-85hpg\" (UID: \"2fbbec08-4030-4af6-8f8e-4ccc6c783a49\") " pod="kuadrant-system/authorino-7bcb78779c-85hpg" Jul 15 17:12:33.943336 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:12:33.943295 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcb78779c-85hpg" Jul 15 17:12:34.067435 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:12:34.067406 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:12:34.083803 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:12:34.083775 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcb78779c-85hpg"] Jul 15 17:12:34.086132 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:12:34.086101 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2fbbec08_4030_4af6_8f8e_4ccc6c783a49.slice/crio-946beff15e178f41d1f5cbd2f9fef9bfd2c19dfc3527a0842e6f6ea4c83a2d11 WatchSource:0}: Error finding container 946beff15e178f41d1f5cbd2f9fef9bfd2c19dfc3527a0842e6f6ea4c83a2d11: Status 404 returned error can't find the container with id 946beff15e178f41d1f5cbd2f9fef9bfd2c19dfc3527a0842e6f6ea4c83a2d11 Jul 15 17:12:34.192326 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:12:34.192282 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcb78779c-85hpg" event={"ID":"2fbbec08-4030-4af6-8f8e-4ccc6c783a49","Type":"ContainerStarted","Data":"946beff15e178f41d1f5cbd2f9fef9bfd2c19dfc3527a0842e6f6ea4c83a2d11"} Jul 15 17:12:35.400288 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:12:35.400255 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:12:36.206701 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:12:36.206656 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcb78779c-85hpg" event={"ID":"2fbbec08-4030-4af6-8f8e-4ccc6c783a49","Type":"ContainerStarted","Data":"b6182b57f213c2cd0fac4acf5c3c98900f17033778ba05a5709d29958a65a40f"} Jul 15 17:12:36.227890 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:12:36.227836 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcb78779c-85hpg" podStartSLOduration=1.283683229 podStartE2EDuration="3.227819009s" totalImagesPullingTime="1.94413578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:12:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:12:36.2257598 +0000 UTC m=+973.778890014" watchObservedRunningTime="2026-07-15 17:12:36.227819009 +0000 UTC m=+973.780949225" Jul 15 17:12:36.259796 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:12:36.259759 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-77f46567d8-x6zj6"] Jul 15 17:12:36.260037 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:12:36.259984 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-77f46567d8-x6zj6" podUID="096da392-ec64-43b2-8c74-d48fe112dc43" containerName="authorino" containerID="cri-o://bd412a4457f9a151998fc58664a615b621981ef9fae9a1eaa63b2d44476540b4" gracePeriod=30 Jul 15 17:12:36.522062 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:12:36.522033 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-77f46567d8-x6zj6" Jul 15 17:12:36.664397 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:12:36.664357 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/096da392-ec64-43b2-8c74-d48fe112dc43-tls-cert\") pod \"096da392-ec64-43b2-8c74-d48fe112dc43\" (UID: \"096da392-ec64-43b2-8c74-d48fe112dc43\") " Jul 15 17:12:36.664578 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:12:36.664419 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6jvxg\" (UniqueName: \"kubernetes.io/projected/096da392-ec64-43b2-8c74-d48fe112dc43-kube-api-access-6jvxg\") pod \"096da392-ec64-43b2-8c74-d48fe112dc43\" (UID: \"096da392-ec64-43b2-8c74-d48fe112dc43\") " Jul 15 17:12:36.672376 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:12:36.672341 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/096da392-ec64-43b2-8c74-d48fe112dc43-kube-api-access-6jvxg" (OuterVolumeSpecName: "kube-api-access-6jvxg") pod "096da392-ec64-43b2-8c74-d48fe112dc43" (UID: "096da392-ec64-43b2-8c74-d48fe112dc43"). InnerVolumeSpecName "kube-api-access-6jvxg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:12:36.679451 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:12:36.679420 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/096da392-ec64-43b2-8c74-d48fe112dc43-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "096da392-ec64-43b2-8c74-d48fe112dc43" (UID: "096da392-ec64-43b2-8c74-d48fe112dc43"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:12:36.765270 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:12:36.765235 2578 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/096da392-ec64-43b2-8c74-d48fe112dc43-tls-cert\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:12:36.765270 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:12:36.765268 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jvxg\" (UniqueName: \"kubernetes.io/projected/096da392-ec64-43b2-8c74-d48fe112dc43-kube-api-access-6jvxg\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:12:37.213571 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:12:37.213533 2578 generic.go:356] "Generic (PLEG): container finished" podID="096da392-ec64-43b2-8c74-d48fe112dc43" containerID="bd412a4457f9a151998fc58664a615b621981ef9fae9a1eaa63b2d44476540b4" exitCode=0 Jul 15 17:12:37.213785 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:12:37.213616 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-77f46567d8-x6zj6" Jul 15 17:12:37.213785 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:12:37.213633 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-77f46567d8-x6zj6" event={"ID":"096da392-ec64-43b2-8c74-d48fe112dc43","Type":"ContainerDied","Data":"bd412a4457f9a151998fc58664a615b621981ef9fae9a1eaa63b2d44476540b4"} Jul 15 17:12:37.213785 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:12:37.213686 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-77f46567d8-x6zj6" event={"ID":"096da392-ec64-43b2-8c74-d48fe112dc43","Type":"ContainerDied","Data":"d726cb0fa827c3264d5f53135133091ee63196f9b7d5798b05205e8e66466f7d"} Jul 15 17:12:37.213785 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:12:37.213710 2578 scope.go:117] "RemoveContainer" containerID="bd412a4457f9a151998fc58664a615b621981ef9fae9a1eaa63b2d44476540b4" Jul 15 17:12:37.223416 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:12:37.223389 2578 scope.go:117] "RemoveContainer" containerID="bd412a4457f9a151998fc58664a615b621981ef9fae9a1eaa63b2d44476540b4" Jul 15 17:12:37.223789 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:12:37.223761 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd412a4457f9a151998fc58664a615b621981ef9fae9a1eaa63b2d44476540b4\": container with ID starting with bd412a4457f9a151998fc58664a615b621981ef9fae9a1eaa63b2d44476540b4 not found: ID does not exist" containerID="bd412a4457f9a151998fc58664a615b621981ef9fae9a1eaa63b2d44476540b4" Jul 15 17:12:37.223889 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:12:37.223798 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd412a4457f9a151998fc58664a615b621981ef9fae9a1eaa63b2d44476540b4"} err="failed to get container status \"bd412a4457f9a151998fc58664a615b621981ef9fae9a1eaa63b2d44476540b4\": rpc error: code = NotFound desc = could not find container \"bd412a4457f9a151998fc58664a615b621981ef9fae9a1eaa63b2d44476540b4\": container with ID starting with bd412a4457f9a151998fc58664a615b621981ef9fae9a1eaa63b2d44476540b4 not found: ID does not exist" Jul 15 17:12:37.235920 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:12:37.235894 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-77f46567d8-x6zj6"] Jul 15 17:12:37.245577 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:12:37.245556 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-77f46567d8-x6zj6"] Jul 15 17:12:38.991010 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:12:38.990978 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="096da392-ec64-43b2-8c74-d48fe112dc43" path="/var/lib/kubelet/pods/096da392-ec64-43b2-8c74-d48fe112dc43/volumes" Jul 15 17:13:08.032124 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:13:08.032093 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:13:08.107108 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:13:08.107077 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:13:12.294361 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:13:12.294331 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:13:37.137608 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:13:37.137556 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:13:47.130887 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:13:47.130854 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:13:48.409191 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:13:48.409160 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:13:58.419810 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:13:58.419774 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:13:58.520050 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:13:58.520014 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:14:13.412490 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:14:13.412450 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:14:14.006460 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:14:14.006419 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:14:33.475983 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:14:33.475888 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:14:33.575368 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:14:33.575323 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:14:33.709374 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:14:33.709342 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:14:40.509654 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:14:40.509586 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:14:40.608138 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:14:40.608105 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:14:40.709049 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:14:40.709013 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:14:40.898883 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:14:40.898798 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:14:47.373002 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:14:47.372950 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:14:47.439173 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:14:47.439137 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:15:00.157001 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:15:00.156965 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735595-29zc9"] Jul 15 17:15:00.157452 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:15:00.157351 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="096da392-ec64-43b2-8c74-d48fe112dc43" containerName="authorino" Jul 15 17:15:00.157452 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:15:00.157363 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="096da392-ec64-43b2-8c74-d48fe112dc43" containerName="authorino" Jul 15 17:15:00.157452 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:15:00.157426 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="096da392-ec64-43b2-8c74-d48fe112dc43" containerName="authorino" Jul 15 17:15:00.160571 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:15:00.160552 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735595-29zc9" Jul 15 17:15:00.163638 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:15:00.163616 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dlhgl\"" Jul 15 17:15:00.171006 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:15:00.170983 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735595-29zc9" Jul 15 17:15:00.171190 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:15:00.171168 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735595-29zc9"] Jul 15 17:15:00.310188 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:15:00.308139 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735595-29zc9"] Jul 15 17:15:00.310822 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:15:00.310786 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc033c931_9fd1_4689_a00b_002d7379af84.slice/crio-18f80c6542e5de71db4038bc71a8b99260dbbcd6d02426d84e51c73070df2a93 WatchSource:0}: Error finding container 18f80c6542e5de71db4038bc71a8b99260dbbcd6d02426d84e51c73070df2a93: Status 404 returned error can't find the container with id 18f80c6542e5de71db4038bc71a8b99260dbbcd6d02426d84e51c73070df2a93 Jul 15 17:15:00.312507 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:15:00.312492 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 17:15:00.977193 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:15:00.977158 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735595-29zc9" event={"ID":"c033c931-9fd1-4689-a00b-002d7379af84","Type":"ContainerStarted","Data":"18f80c6542e5de71db4038bc71a8b99260dbbcd6d02426d84e51c73070df2a93"} Jul 15 17:15:02.993027 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:15:02.992942 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735595-29zc9" event={"ID":"c033c931-9fd1-4689-a00b-002d7379af84","Type":"ContainerStarted","Data":"1ac3584d855472ce66e07a398fe13213cf76e865926fcef7dbe561cc8676602d"} Jul 15 17:15:03.012587 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:15:03.012529 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735595-29zc9" podStartSLOduration=0.852752337 podStartE2EDuration="3.012512264s" totalImagesPullingTime="2.159759927s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:15:03.010545219 +0000 UTC m=+1120.563675472" watchObservedRunningTime="2026-07-15 17:15:03.012512264 +0000 UTC m=+1120.565642478" Jul 15 17:15:03.999235 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:15:03.999200 2578 generic.go:356] "Generic (PLEG): container finished" podID="c033c931-9fd1-4689-a00b-002d7379af84" containerID="1ac3584d855472ce66e07a398fe13213cf76e865926fcef7dbe561cc8676602d" exitCode=0 Jul 15 17:15:03.999235 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:15:03.999239 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735595-29zc9" event={"ID":"c033c931-9fd1-4689-a00b-002d7379af84","Type":"ContainerDied","Data":"1ac3584d855472ce66e07a398fe13213cf76e865926fcef7dbe561cc8676602d"} Jul 15 17:15:05.139990 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:15:05.139959 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735595-29zc9" Jul 15 17:15:06.011224 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:15:06.011192 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735595-29zc9" Jul 15 17:15:06.011405 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:15:06.011195 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735595-29zc9" event={"ID":"c033c931-9fd1-4689-a00b-002d7379af84","Type":"ContainerDied","Data":"18f80c6542e5de71db4038bc71a8b99260dbbcd6d02426d84e51c73070df2a93"} Jul 15 17:15:06.011405 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:15:06.011315 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="18f80c6542e5de71db4038bc71a8b99260dbbcd6d02426d84e51c73070df2a93" Jul 15 17:16:29.163380 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:16:29.163351 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29wtx_88f8928c-237e-457f-a032-707f48111f36/ovn-acl-logging/0.log" Jul 15 17:16:29.165926 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:16:29.165906 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29wtx_88f8928c-237e-457f-a032-707f48111f36/ovn-acl-logging/0.log" Jul 15 17:16:34.350701 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:16:34.350665 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:17:41.859867 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:17:41.859833 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:18:13.866248 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:18:13.866203 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:18:45.317953 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:18:45.317873 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vk82h"] Jul 15 17:18:45.318457 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:18:45.318112 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vk82h" podUID="08280924-0060-46e0-854d-ccfd543d4ab2" containerName="manager" containerID="cri-o://a7a9eae24062fd67c8f5382d34bbaf049e4e379585a06449f690cf03706ef3bc" gracePeriod=10 Jul 15 17:18:47.870114 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:18:47.870090 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vk82h" Jul 15 17:18:47.903957 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:18:47.903872 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/08280924-0060-46e0-854d-ccfd543d4ab2-extensions-socket-volume\") pod \"08280924-0060-46e0-854d-ccfd543d4ab2\" (UID: \"08280924-0060-46e0-854d-ccfd543d4ab2\") " Jul 15 17:18:47.904093 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:18:47.904006 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rncz9\" (UniqueName: \"kubernetes.io/projected/08280924-0060-46e0-854d-ccfd543d4ab2-kube-api-access-rncz9\") pod \"08280924-0060-46e0-854d-ccfd543d4ab2\" (UID: \"08280924-0060-46e0-854d-ccfd543d4ab2\") " Jul 15 17:18:47.904330 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:18:47.904289 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08280924-0060-46e0-854d-ccfd543d4ab2-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "08280924-0060-46e0-854d-ccfd543d4ab2" (UID: "08280924-0060-46e0-854d-ccfd543d4ab2"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:18:47.906390 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:18:47.906363 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08280924-0060-46e0-854d-ccfd543d4ab2-kube-api-access-rncz9" (OuterVolumeSpecName: "kube-api-access-rncz9") pod "08280924-0060-46e0-854d-ccfd543d4ab2" (UID: "08280924-0060-46e0-854d-ccfd543d4ab2"). InnerVolumeSpecName "kube-api-access-rncz9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:18:48.004963 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:18:48.004919 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rncz9\" (UniqueName: \"kubernetes.io/projected/08280924-0060-46e0-854d-ccfd543d4ab2-kube-api-access-rncz9\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:18:48.004963 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:18:48.004961 2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/08280924-0060-46e0-854d-ccfd543d4ab2-extensions-socket-volume\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:18:48.202830 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:18:48.202797 2578 generic.go:356] "Generic (PLEG): container finished" podID="08280924-0060-46e0-854d-ccfd543d4ab2" containerID="a7a9eae24062fd67c8f5382d34bbaf049e4e379585a06449f690cf03706ef3bc" exitCode=0 Jul 15 17:18:48.202992 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:18:48.202839 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vk82h" Jul 15 17:18:48.202992 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:18:48.202880 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vk82h" event={"ID":"08280924-0060-46e0-854d-ccfd543d4ab2","Type":"ContainerDied","Data":"a7a9eae24062fd67c8f5382d34bbaf049e4e379585a06449f690cf03706ef3bc"} Jul 15 17:18:48.202992 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:18:48.202932 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vk82h" event={"ID":"08280924-0060-46e0-854d-ccfd543d4ab2","Type":"ContainerDied","Data":"e3ca18d7ad199e26bd61186c861d5b5c57b9928a4d01593e57223ba291056aa9"} Jul 15 17:18:48.202992 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:18:48.202955 2578 scope.go:117] "RemoveContainer" containerID="a7a9eae24062fd67c8f5382d34bbaf049e4e379585a06449f690cf03706ef3bc" Jul 15 17:18:48.213014 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:18:48.212998 2578 scope.go:117] "RemoveContainer" containerID="a7a9eae24062fd67c8f5382d34bbaf049e4e379585a06449f690cf03706ef3bc" Jul 15 17:18:48.213247 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:18:48.213232 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7a9eae24062fd67c8f5382d34bbaf049e4e379585a06449f690cf03706ef3bc\": container with ID starting with a7a9eae24062fd67c8f5382d34bbaf049e4e379585a06449f690cf03706ef3bc not found: ID does not exist" containerID="a7a9eae24062fd67c8f5382d34bbaf049e4e379585a06449f690cf03706ef3bc" Jul 15 17:18:48.213292 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:18:48.213254 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7a9eae24062fd67c8f5382d34bbaf049e4e379585a06449f690cf03706ef3bc"} err="failed to get container status \"a7a9eae24062fd67c8f5382d34bbaf049e4e379585a06449f690cf03706ef3bc\": rpc error: code = NotFound desc = could not find container \"a7a9eae24062fd67c8f5382d34bbaf049e4e379585a06449f690cf03706ef3bc\": container with ID starting with a7a9eae24062fd67c8f5382d34bbaf049e4e379585a06449f690cf03706ef3bc not found: ID does not exist" Jul 15 17:18:48.224575 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:18:48.224549 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vk82h"] Jul 15 17:18:48.228876 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:18:48.228853 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vk82h"] Jul 15 17:18:48.990723 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:18:48.990695 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08280924-0060-46e0-854d-ccfd543d4ab2" path="/var/lib/kubelet/pods/08280924-0060-46e0-854d-ccfd543d4ab2/volumes" Jul 15 17:19:46.566413 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:19:46.566376 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rk7bm"] Jul 15 17:19:46.566836 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:19:46.566754 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08280924-0060-46e0-854d-ccfd543d4ab2" containerName="manager" Jul 15 17:19:46.566836 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:19:46.566765 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="08280924-0060-46e0-854d-ccfd543d4ab2" containerName="manager" Jul 15 17:19:46.566836 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:19:46.566775 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c033c931-9fd1-4689-a00b-002d7379af84" containerName="cleanup" Jul 15 17:19:46.566836 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:19:46.566781 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c033c931-9fd1-4689-a00b-002d7379af84" containerName="cleanup" Jul 15 17:19:46.566836 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:19:46.566832 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c033c931-9fd1-4689-a00b-002d7379af84" containerName="cleanup" Jul 15 17:19:46.566999 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:19:46.566844 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="08280924-0060-46e0-854d-ccfd543d4ab2" containerName="manager" Jul 15 17:19:46.571114 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:19:46.571098 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rk7bm" Jul 15 17:19:46.573607 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:19:46.573578 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-984tg\"" Jul 15 17:19:46.582188 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:19:46.582165 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rk7bm"] Jul 15 17:19:46.587318 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:19:46.587289 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/23a87bb5-b7ed-4ccf-941e-27c800dedc92-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rk7bm\" (UID: \"23a87bb5-b7ed-4ccf-941e-27c800dedc92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rk7bm" Jul 15 17:19:46.587427 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:19:46.587406 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9bcj\" (UniqueName: \"kubernetes.io/projected/23a87bb5-b7ed-4ccf-941e-27c800dedc92-kube-api-access-n9bcj\") pod \"kuadrant-operator-controller-manager-86878c769f-rk7bm\" (UID: \"23a87bb5-b7ed-4ccf-941e-27c800dedc92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rk7bm" Jul 15 17:19:46.688533 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:19:46.688498 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/23a87bb5-b7ed-4ccf-941e-27c800dedc92-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rk7bm\" (UID: \"23a87bb5-b7ed-4ccf-941e-27c800dedc92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rk7bm" Jul 15 17:19:46.688710 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:19:46.688586 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9bcj\" (UniqueName: \"kubernetes.io/projected/23a87bb5-b7ed-4ccf-941e-27c800dedc92-kube-api-access-n9bcj\") pod \"kuadrant-operator-controller-manager-86878c769f-rk7bm\" (UID: \"23a87bb5-b7ed-4ccf-941e-27c800dedc92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rk7bm" Jul 15 17:19:46.688908 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:19:46.688889 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/23a87bb5-b7ed-4ccf-941e-27c800dedc92-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rk7bm\" (UID: \"23a87bb5-b7ed-4ccf-941e-27c800dedc92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rk7bm" Jul 15 17:19:46.697080 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:19:46.697054 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9bcj\" (UniqueName: \"kubernetes.io/projected/23a87bb5-b7ed-4ccf-941e-27c800dedc92-kube-api-access-n9bcj\") pod \"kuadrant-operator-controller-manager-86878c769f-rk7bm\" (UID: \"23a87bb5-b7ed-4ccf-941e-27c800dedc92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rk7bm" Jul 15 17:19:46.883046 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:19:46.882954 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rk7bm" Jul 15 17:19:47.022854 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:19:47.022826 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rk7bm"] Jul 15 17:19:47.024270 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:19:47.024229 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23a87bb5_b7ed_4ccf_941e_27c800dedc92.slice/crio-3f2b6128ee1449281deae59d4ab4e695733580fd841450086975633839bc40f9 WatchSource:0}: Error finding container 3f2b6128ee1449281deae59d4ab4e695733580fd841450086975633839bc40f9: Status 404 returned error can't find the container with id 3f2b6128ee1449281deae59d4ab4e695733580fd841450086975633839bc40f9 Jul 15 17:19:47.512307 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:19:47.512271 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rk7bm" event={"ID":"23a87bb5-b7ed-4ccf-941e-27c800dedc92","Type":"ContainerStarted","Data":"84d3befabf7ba61ed04f821f9380166fb3dac5f7f8cd267bd47302c024beb32f"} Jul 15 17:19:47.512499 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:19:47.512332 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rk7bm" event={"ID":"23a87bb5-b7ed-4ccf-941e-27c800dedc92","Type":"ContainerStarted","Data":"3f2b6128ee1449281deae59d4ab4e695733580fd841450086975633839bc40f9"} Jul 15 17:19:47.512499 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:19:47.512452 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rk7bm" Jul 15 17:19:47.533843 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:19:47.533788 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rk7bm" podStartSLOduration=1.53377254 podStartE2EDuration="1.53377254s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:19:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:19:47.531180901 +0000 UTC m=+1405.084311143" watchObservedRunningTime="2026-07-15 17:19:47.53377254 +0000 UTC m=+1405.086902754" Jul 15 17:19:58.522477 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:19:58.522446 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rk7bm" Jul 15 17:21:29.199550 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:21:29.199451 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29wtx_88f8928c-237e-457f-a032-707f48111f36/ovn-acl-logging/0.log" Jul 15 17:21:29.204962 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:21:29.204942 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29wtx_88f8928c-237e-457f-a032-707f48111f36/ovn-acl-logging/0.log" Jul 15 17:23:19.528330 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:23:19.528243 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:23:21.296107 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:23:21.296068 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:23:25.827999 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:23:25.827961 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:23:36.753186 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:23:36.753144 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:23:36.852879 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:23:36.852836 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:25:41.861207 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:25:41.861172 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:25:49.295226 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:25:49.295181 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:25:57.559410 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:25:57.559366 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:26:02.676087 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:02.676054 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:26:05.661541 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:05.661504 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:26:05.764137 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:05.764093 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:26:06.074239 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:06.074194 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:26:12.541716 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:12.541678 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:26:12.641490 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:12.641454 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:26:12.761216 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:12.761179 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:26:12.826240 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:12.826153 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:26:13.932120 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:13.932081 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:26:14.018987 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:14.018953 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:26:14.021188 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:14.021162 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd"] Jul 15 17:26:14.024721 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:14.024704 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd" Jul 15 17:26:14.027360 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:14.027334 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-77cxl\"" Jul 15 17:26:14.027466 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:14.027363 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-82b50f4c-serving-cert\"" Jul 15 17:26:14.027466 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:14.027379 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 17:26:14.028355 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:14.028338 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 15 17:26:14.035812 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:14.035789 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd"] Jul 15 17:26:14.091142 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:14.091109 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:26:14.174899 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:14.174859 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nwps\" (UniqueName: \"kubernetes.io/projected/7a206ebd-a423-4623-8029-097359f73b7c-kube-api-access-5nwps\") pod \"maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd\" (UID: \"7a206ebd-a423-4623-8029-097359f73b7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd" Jul 15 17:26:14.174899 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:14.174896 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a206ebd-a423-4623-8029-097359f73b7c-openshift-service-ca\") pod \"maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd\" (UID: \"7a206ebd-a423-4623-8029-097359f73b7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd" Jul 15 17:26:14.175104 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:14.174969 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a206ebd-a423-4623-8029-097359f73b7c-maas-api-tls\") pod \"maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd\" (UID: \"7a206ebd-a423-4623-8029-097359f73b7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd" Jul 15 17:26:14.276434 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:14.276344 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a206ebd-a423-4623-8029-097359f73b7c-maas-api-tls\") pod \"maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd\" (UID: \"7a206ebd-a423-4623-8029-097359f73b7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd" Jul 15 17:26:14.276585 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:26:14.276508 2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-82b50f4c-serving-cert: secret "maas-api-e2e-ait-82b50f4c-serving-cert" not found Jul 15 17:26:14.276585 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:14.276522 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nwps\" (UniqueName: \"kubernetes.io/projected/7a206ebd-a423-4623-8029-097359f73b7c-kube-api-access-5nwps\") pod \"maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd\" (UID: \"7a206ebd-a423-4623-8029-097359f73b7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd" Jul 15 17:26:14.276585 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:14.276563 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a206ebd-a423-4623-8029-097359f73b7c-openshift-service-ca\") pod \"maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd\" (UID: \"7a206ebd-a423-4623-8029-097359f73b7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd" Jul 15 17:26:14.276750 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:26:14.276624 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a206ebd-a423-4623-8029-097359f73b7c-maas-api-tls podName:7a206ebd-a423-4623-8029-097359f73b7c nodeName:}" failed. No retries permitted until 2026-07-15 17:26:14.77657408 +0000 UTC m=+1792.329704276 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7a206ebd-a423-4623-8029-097359f73b7c-maas-api-tls") pod "maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd" (UID: "7a206ebd-a423-4623-8029-097359f73b7c") : secret "maas-api-e2e-ait-82b50f4c-serving-cert" not found Jul 15 17:26:14.277200 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:14.277182 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a206ebd-a423-4623-8029-097359f73b7c-openshift-service-ca\") pod \"maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd\" (UID: \"7a206ebd-a423-4623-8029-097359f73b7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd" Jul 15 17:26:14.293582 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:14.293553 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nwps\" (UniqueName: \"kubernetes.io/projected/7a206ebd-a423-4623-8029-097359f73b7c-kube-api-access-5nwps\") pod \"maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd\" (UID: \"7a206ebd-a423-4623-8029-097359f73b7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd" Jul 15 17:26:14.781498 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:14.781462 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a206ebd-a423-4623-8029-097359f73b7c-maas-api-tls\") pod \"maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd\" (UID: \"7a206ebd-a423-4623-8029-097359f73b7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd" Jul 15 17:26:14.783913 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:14.783882 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a206ebd-a423-4623-8029-097359f73b7c-maas-api-tls\") pod \"maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd\" (UID: \"7a206ebd-a423-4623-8029-097359f73b7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd" Jul 15 17:26:14.793774 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:14.793746 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:26:14.861419 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:14.861364 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:26:14.936043 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:14.935998 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd" Jul 15 17:26:15.072619 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:15.072581 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd"] Jul 15 17:26:15.074328 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:26:15.074298 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a206ebd_a423_4623_8029_097359f73b7c.slice/crio-210a6e29eb0aa28e63ac4aea48525d7b1b0d7193250ebc8c7c79c79e06311829 WatchSource:0}: Error finding container 210a6e29eb0aa28e63ac4aea48525d7b1b0d7193250ebc8c7c79c79e06311829: Status 404 returned error can't find the container with id 210a6e29eb0aa28e63ac4aea48525d7b1b0d7193250ebc8c7c79c79e06311829 Jul 15 17:26:15.075659 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:15.075642 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 17:26:15.602044 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:15.602003 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd" event={"ID":"7a206ebd-a423-4623-8029-097359f73b7c","Type":"ContainerStarted","Data":"210a6e29eb0aa28e63ac4aea48525d7b1b0d7193250ebc8c7c79c79e06311829"} Jul 15 17:26:16.860443 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:16.860367 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:26:17.616181 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:17.616084 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd" event={"ID":"7a206ebd-a423-4623-8029-097359f73b7c","Type":"ContainerStarted","Data":"fe7f6cd5b5c6144769cbbcf651d715972f32c3c5934e65c8e45e1c491069c85f"} Jul 15 17:26:17.616181 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:17.616154 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd" Jul 15 17:26:17.634000 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:17.633952 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd" podStartSLOduration=1.381971478 podStartE2EDuration="3.633934901s" totalImagesPullingTime="2.251963423s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:26:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:26:17.631615452 +0000 UTC m=+1795.184745683" watchObservedRunningTime="2026-07-15 17:26:17.633934901 +0000 UTC m=+1795.187065115" Jul 15 17:26:18.932870 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:18.932832 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-zwmx8"] Jul 15 17:26:18.936559 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:18.936536 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-zwmx8" Jul 15 17:26:18.938875 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:18.938851 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dlhgl\"" Jul 15 17:26:18.945804 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:18.945781 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-zwmx8"] Jul 15 17:26:19.022902 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:19.022868 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6736155-c8c4-4206-b3dc-a5a56ffce63a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-82b50f4c-zwmx8\" (UID: \"f6736155-c8c4-4206-b3dc-a5a56ffce63a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-zwmx8" Jul 15 17:26:19.124184 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:19.124149 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6736155-c8c4-4206-b3dc-a5a56ffce63a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-82b50f4c-zwmx8\" (UID: \"f6736155-c8c4-4206-b3dc-a5a56ffce63a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-zwmx8" Jul 15 17:26:19.124822 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:19.124801 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6736155-c8c4-4206-b3dc-a5a56ffce63a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-82b50f4c-zwmx8\" (UID: \"f6736155-c8c4-4206-b3dc-a5a56ffce63a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-zwmx8" Jul 15 17:26:19.248238 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:19.248160 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-zwmx8" Jul 15 17:26:19.389551 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:19.389525 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-zwmx8"] Jul 15 17:26:19.391096 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:26:19.391065 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6736155_c8c4_4206_b3dc_a5a56ffce63a.slice/crio-ab6d5d748e31096e40b875e08797ae958d4234eec33cebf47e9624780daed02c WatchSource:0}: Error finding container ab6d5d748e31096e40b875e08797ae958d4234eec33cebf47e9624780daed02c: Status 404 returned error can't find the container with id ab6d5d748e31096e40b875e08797ae958d4234eec33cebf47e9624780daed02c Jul 15 17:26:19.627258 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:19.627225 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-zwmx8" event={"ID":"f6736155-c8c4-4206-b3dc-a5a56ffce63a","Type":"ContainerStarted","Data":"1d0029f4d8f7d5b40b59a3ced99e7ef56434bd6b9eadd5e5ddb163f5ec3725f9"} Jul 15 17:26:19.627419 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:19.627283 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-zwmx8" event={"ID":"f6736155-c8c4-4206-b3dc-a5a56ffce63a","Type":"ContainerStarted","Data":"ab6d5d748e31096e40b875e08797ae958d4234eec33cebf47e9624780daed02c"} Jul 15 17:26:19.647202 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:19.647153 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-zwmx8" podStartSLOduration=1.6471395869999998 podStartE2EDuration="1.647139587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:26:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:26:19.645398867 +0000 UTC m=+1797.198529082" watchObservedRunningTime="2026-07-15 17:26:19.647139587 +0000 UTC m=+1797.200269800" Jul 15 17:26:20.633983 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:20.633944 2578 generic.go:356] "Generic (PLEG): container finished" podID="f6736155-c8c4-4206-b3dc-a5a56ffce63a" containerID="1d0029f4d8f7d5b40b59a3ced99e7ef56434bd6b9eadd5e5ddb163f5ec3725f9" exitCode=7 Jul 15 17:26:20.634444 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:20.634021 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-zwmx8" event={"ID":"f6736155-c8c4-4206-b3dc-a5a56ffce63a","Type":"ContainerDied","Data":"1d0029f4d8f7d5b40b59a3ced99e7ef56434bd6b9eadd5e5ddb163f5ec3725f9"} Jul 15 17:26:20.634444 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:20.634223 2578 scope.go:117] "RemoveContainer" containerID="1d0029f4d8f7d5b40b59a3ced99e7ef56434bd6b9eadd5e5ddb163f5ec3725f9" Jul 15 17:26:21.642586 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:21.642490 2578 generic.go:356] "Generic (PLEG): container finished" podID="f6736155-c8c4-4206-b3dc-a5a56ffce63a" containerID="386b3852c4a01ad765f66d55c14af18cf316a3f2de6cdb2d37eec3c38e740da2" exitCode=7 Jul 15 17:26:21.642586 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:21.642547 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-zwmx8" event={"ID":"f6736155-c8c4-4206-b3dc-a5a56ffce63a","Type":"ContainerDied","Data":"386b3852c4a01ad765f66d55c14af18cf316a3f2de6cdb2d37eec3c38e740da2"} Jul 15 17:26:21.643109 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:21.642617 2578 scope.go:117] "RemoveContainer" containerID="1d0029f4d8f7d5b40b59a3ced99e7ef56434bd6b9eadd5e5ddb163f5ec3725f9" Jul 15 17:26:21.643109 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:21.642816 2578 scope.go:117] "RemoveContainer" containerID="386b3852c4a01ad765f66d55c14af18cf316a3f2de6cdb2d37eec3c38e740da2" Jul 15 17:26:21.643109 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:26:21.643079 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-82b50f4c-zwmx8_odh-ai-gateway-infra(f6736155-c8c4-4206-b3dc-a5a56ffce63a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-zwmx8" podUID="f6736155-c8c4-4206-b3dc-a5a56ffce63a" Jul 15 17:26:22.649635 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:22.649591 2578 scope.go:117] "RemoveContainer" containerID="386b3852c4a01ad765f66d55c14af18cf316a3f2de6cdb2d37eec3c38e740da2" Jul 15 17:26:22.650022 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:26:22.649831 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-82b50f4c-zwmx8_odh-ai-gateway-infra(f6736155-c8c4-4206-b3dc-a5a56ffce63a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-zwmx8" podUID="f6736155-c8c4-4206-b3dc-a5a56ffce63a" Jul 15 17:26:23.627197 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:23.627169 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd" Jul 15 17:26:29.244194 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:29.244077 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29wtx_88f8928c-237e-457f-a032-707f48111f36/ovn-acl-logging/0.log" Jul 15 17:26:29.266678 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:29.255035 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29wtx_88f8928c-237e-457f-a032-707f48111f36/ovn-acl-logging/0.log" Jul 15 17:26:35.984686 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:35.984650 2578 scope.go:117] "RemoveContainer" containerID="386b3852c4a01ad765f66d55c14af18cf316a3f2de6cdb2d37eec3c38e740da2" Jul 15 17:26:36.727080 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:36.727047 2578 generic.go:356] "Generic (PLEG): container finished" podID="f6736155-c8c4-4206-b3dc-a5a56ffce63a" containerID="46d84c9bc09e80b3e811feb6b8d0f248bb4f70e3450f3871f1bb02adae97bb6a" exitCode=0 Jul 15 17:26:36.727225 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:36.727096 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-zwmx8" event={"ID":"f6736155-c8c4-4206-b3dc-a5a56ffce63a","Type":"ContainerDied","Data":"46d84c9bc09e80b3e811feb6b8d0f248bb4f70e3450f3871f1bb02adae97bb6a"} Jul 15 17:26:36.727225 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:36.727142 2578 scope.go:117] "RemoveContainer" containerID="386b3852c4a01ad765f66d55c14af18cf316a3f2de6cdb2d37eec3c38e740da2" Jul 15 17:26:37.761133 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:37.761097 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-zwmx8"] Jul 15 17:26:37.866522 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:37.866494 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-zwmx8" Jul 15 17:26:37.884152 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:37.884124 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6736155-c8c4-4206-b3dc-a5a56ffce63a-maas-api-service-ca\") pod \"f6736155-c8c4-4206-b3dc-a5a56ffce63a\" (UID: \"f6736155-c8c4-4206-b3dc-a5a56ffce63a\") " Jul 15 17:26:37.884513 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:37.884489 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6736155-c8c4-4206-b3dc-a5a56ffce63a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f6736155-c8c4-4206-b3dc-a5a56ffce63a" (UID: "f6736155-c8c4-4206-b3dc-a5a56ffce63a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:26:37.985740 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:37.985636 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6736155-c8c4-4206-b3dc-a5a56ffce63a-maas-api-service-ca\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:26:38.741647 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:38.741614 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-zwmx8" Jul 15 17:26:38.741847 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:38.741611 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-zwmx8" event={"ID":"f6736155-c8c4-4206-b3dc-a5a56ffce63a","Type":"ContainerDied","Data":"ab6d5d748e31096e40b875e08797ae958d4234eec33cebf47e9624780daed02c"} Jul 15 17:26:38.741847 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:38.741744 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ab6d5d748e31096e40b875e08797ae958d4234eec33cebf47e9624780daed02c" Jul 15 17:26:38.773112 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:38.773078 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-zwmx8"] Jul 15 17:26:38.776937 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:38.776910 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-zwmx8"] Jul 15 17:26:38.990731 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:26:38.990699 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6736155-c8c4-4206-b3dc-a5a56ffce63a" path="/var/lib/kubelet/pods/f6736155-c8c4-4206-b3dc-a5a56ffce63a/volumes" Jul 15 17:27:08.977485 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:08.977451 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-987sx"] Jul 15 17:27:08.978093 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:08.978027 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6736155-c8c4-4206-b3dc-a5a56ffce63a" containerName="revoke-keys" Jul 15 17:27:08.978093 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:08.978047 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6736155-c8c4-4206-b3dc-a5a56ffce63a" containerName="revoke-keys" Jul 15 17:27:08.978093 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:08.978073 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6736155-c8c4-4206-b3dc-a5a56ffce63a" containerName="revoke-keys" Jul 15 17:27:08.978093 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:08.978081 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6736155-c8c4-4206-b3dc-a5a56ffce63a" containerName="revoke-keys" Jul 15 17:27:08.978313 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:08.978166 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6736155-c8c4-4206-b3dc-a5a56ffce63a" containerName="revoke-keys" Jul 15 17:27:08.978313 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:08.978178 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6736155-c8c4-4206-b3dc-a5a56ffce63a" containerName="revoke-keys" Jul 15 17:27:08.981332 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:08.981310 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-987sx" Jul 15 17:27:08.983859 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:08.983837 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dlhgl\"" Jul 15 17:27:08.993548 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:08.993524 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-987sx"] Jul 15 17:27:09.059383 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:09.059348 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4040b104-ab36-44de-9196-dbc947a3830a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-82b50f4c-987sx\" (UID: \"4040b104-ab36-44de-9196-dbc947a3830a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-987sx" Jul 15 17:27:09.160675 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:09.160630 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4040b104-ab36-44de-9196-dbc947a3830a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-82b50f4c-987sx\" (UID: \"4040b104-ab36-44de-9196-dbc947a3830a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-987sx" Jul 15 17:27:09.161280 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:09.161259 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4040b104-ab36-44de-9196-dbc947a3830a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-82b50f4c-987sx\" (UID: \"4040b104-ab36-44de-9196-dbc947a3830a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-987sx" Jul 15 17:27:09.294815 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:09.294736 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-987sx" Jul 15 17:27:09.427379 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:09.427354 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-987sx"] Jul 15 17:27:09.428850 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:27:09.428818 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4040b104_ab36_44de_9196_dbc947a3830a.slice/crio-146686248da2429b1609ece9a470f3fe5f6950dddcfec7dfeb65fca3d17426c5 WatchSource:0}: Error finding container 146686248da2429b1609ece9a470f3fe5f6950dddcfec7dfeb65fca3d17426c5: Status 404 returned error can't find the container with id 146686248da2429b1609ece9a470f3fe5f6950dddcfec7dfeb65fca3d17426c5 Jul 15 17:27:09.908224 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:09.908128 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-987sx" event={"ID":"4040b104-ab36-44de-9196-dbc947a3830a","Type":"ContainerStarted","Data":"499255ad7d077904b7e0891f2fc1e54cf0c835eb3cabd9f069f50d76bfa3b86a"} Jul 15 17:27:09.908385 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:09.908222 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-987sx" event={"ID":"4040b104-ab36-44de-9196-dbc947a3830a","Type":"ContainerStarted","Data":"146686248da2429b1609ece9a470f3fe5f6950dddcfec7dfeb65fca3d17426c5"} Jul 15 17:27:09.925623 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:09.925562 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-987sx" podStartSLOduration=1.92554897 podStartE2EDuration="1.92554897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:27:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:27:09.923420334 +0000 UTC m=+1847.476550573" watchObservedRunningTime="2026-07-15 17:27:09.92554897 +0000 UTC m=+1847.478679184" Jul 15 17:27:10.916072 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:10.916038 2578 generic.go:356] "Generic (PLEG): container finished" podID="4040b104-ab36-44de-9196-dbc947a3830a" containerID="499255ad7d077904b7e0891f2fc1e54cf0c835eb3cabd9f069f50d76bfa3b86a" exitCode=0 Jul 15 17:27:10.916437 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:10.916079 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-987sx" event={"ID":"4040b104-ab36-44de-9196-dbc947a3830a","Type":"ContainerDied","Data":"499255ad7d077904b7e0891f2fc1e54cf0c835eb3cabd9f069f50d76bfa3b86a"} Jul 15 17:27:12.047862 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:12.047840 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-987sx" Jul 15 17:27:12.188099 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:12.188011 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4040b104-ab36-44de-9196-dbc947a3830a-maas-api-service-ca\") pod \"4040b104-ab36-44de-9196-dbc947a3830a\" (UID: \"4040b104-ab36-44de-9196-dbc947a3830a\") " Jul 15 17:27:12.188371 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:12.188350 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4040b104-ab36-44de-9196-dbc947a3830a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4040b104-ab36-44de-9196-dbc947a3830a" (UID: "4040b104-ab36-44de-9196-dbc947a3830a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:27:12.289335 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:12.289301 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4040b104-ab36-44de-9196-dbc947a3830a-maas-api-service-ca\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:27:12.927737 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:12.927709 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-987sx" Jul 15 17:27:12.927916 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:12.927753 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82b50f4c-987sx" event={"ID":"4040b104-ab36-44de-9196-dbc947a3830a","Type":"ContainerDied","Data":"146686248da2429b1609ece9a470f3fe5f6950dddcfec7dfeb65fca3d17426c5"} Jul 15 17:27:12.927916 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:12.927804 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="146686248da2429b1609ece9a470f3fe5f6950dddcfec7dfeb65fca3d17426c5" Jul 15 17:27:19.568621 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:19.568572 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd"] Jul 15 17:27:19.569089 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:19.568808 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd" podUID="7a206ebd-a423-4623-8029-097359f73b7c" containerName="maas-api" containerID="cri-o://fe7f6cd5b5c6144769cbbcf651d715972f32c3c5934e65c8e45e1c491069c85f" gracePeriod=30 Jul 15 17:27:19.596889 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:19.596860 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:27:19.809422 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:19.809396 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd" Jul 15 17:27:19.959898 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:19.959860 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a206ebd-a423-4623-8029-097359f73b7c-maas-api-tls\") pod \"7a206ebd-a423-4623-8029-097359f73b7c\" (UID: \"7a206ebd-a423-4623-8029-097359f73b7c\") " Jul 15 17:27:19.960072 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:19.959965 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5nwps\" (UniqueName: \"kubernetes.io/projected/7a206ebd-a423-4623-8029-097359f73b7c-kube-api-access-5nwps\") pod \"7a206ebd-a423-4623-8029-097359f73b7c\" (UID: \"7a206ebd-a423-4623-8029-097359f73b7c\") " Jul 15 17:27:19.960120 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:19.960074 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a206ebd-a423-4623-8029-097359f73b7c-openshift-service-ca\") pod \"7a206ebd-a423-4623-8029-097359f73b7c\" (UID: \"7a206ebd-a423-4623-8029-097359f73b7c\") " Jul 15 17:27:19.960429 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:19.960405 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a206ebd-a423-4623-8029-097359f73b7c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7a206ebd-a423-4623-8029-097359f73b7c" (UID: "7a206ebd-a423-4623-8029-097359f73b7c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:27:19.962226 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:19.962196 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a206ebd-a423-4623-8029-097359f73b7c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7a206ebd-a423-4623-8029-097359f73b7c" (UID: "7a206ebd-a423-4623-8029-097359f73b7c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:27:19.962226 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:19.962205 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a206ebd-a423-4623-8029-097359f73b7c-kube-api-access-5nwps" (OuterVolumeSpecName: "kube-api-access-5nwps") pod "7a206ebd-a423-4623-8029-097359f73b7c" (UID: "7a206ebd-a423-4623-8029-097359f73b7c"). InnerVolumeSpecName "kube-api-access-5nwps". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:27:19.966235 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:19.966206 2578 generic.go:356] "Generic (PLEG): container finished" podID="7a206ebd-a423-4623-8029-097359f73b7c" containerID="fe7f6cd5b5c6144769cbbcf651d715972f32c3c5934e65c8e45e1c491069c85f" exitCode=0 Jul 15 17:27:19.966348 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:19.966249 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd" Jul 15 17:27:19.966348 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:19.966273 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd" event={"ID":"7a206ebd-a423-4623-8029-097359f73b7c","Type":"ContainerDied","Data":"fe7f6cd5b5c6144769cbbcf651d715972f32c3c5934e65c8e45e1c491069c85f"} Jul 15 17:27:19.966348 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:19.966330 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd" event={"ID":"7a206ebd-a423-4623-8029-097359f73b7c","Type":"ContainerDied","Data":"210a6e29eb0aa28e63ac4aea48525d7b1b0d7193250ebc8c7c79c79e06311829"} Jul 15 17:27:19.966348 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:19.966344 2578 scope.go:117] "RemoveContainer" containerID="fe7f6cd5b5c6144769cbbcf651d715972f32c3c5934e65c8e45e1c491069c85f" Jul 15 17:27:19.981253 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:19.981234 2578 scope.go:117] "RemoveContainer" containerID="fe7f6cd5b5c6144769cbbcf651d715972f32c3c5934e65c8e45e1c491069c85f" Jul 15 17:27:19.981470 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:27:19.981451 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe7f6cd5b5c6144769cbbcf651d715972f32c3c5934e65c8e45e1c491069c85f\": container with ID starting with fe7f6cd5b5c6144769cbbcf651d715972f32c3c5934e65c8e45e1c491069c85f not found: ID does not exist" containerID="fe7f6cd5b5c6144769cbbcf651d715972f32c3c5934e65c8e45e1c491069c85f" Jul 15 17:27:19.981533 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:19.981482 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe7f6cd5b5c6144769cbbcf651d715972f32c3c5934e65c8e45e1c491069c85f"} err="failed to get container status \"fe7f6cd5b5c6144769cbbcf651d715972f32c3c5934e65c8e45e1c491069c85f\": rpc error: code = NotFound desc = could not find container \"fe7f6cd5b5c6144769cbbcf651d715972f32c3c5934e65c8e45e1c491069c85f\": container with ID starting with fe7f6cd5b5c6144769cbbcf651d715972f32c3c5934e65c8e45e1c491069c85f not found: ID does not exist" Jul 15 17:27:19.991860 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:19.991837 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd"] Jul 15 17:27:19.995932 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:19.995910 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-82b50f4c-78d444d76c-dqrtd"] Jul 15 17:27:20.060826 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:20.060790 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5nwps\" (UniqueName: \"kubernetes.io/projected/7a206ebd-a423-4623-8029-097359f73b7c-kube-api-access-5nwps\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:27:20.060826 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:20.060821 2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a206ebd-a423-4623-8029-097359f73b7c-openshift-service-ca\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:27:20.061014 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:20.060836 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a206ebd-a423-4623-8029-097359f73b7c-maas-api-tls\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:27:20.995405 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:20.995369 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a206ebd-a423-4623-8029-097359f73b7c" path="/var/lib/kubelet/pods/7a206ebd-a423-4623-8029-097359f73b7c/volumes" Jul 15 17:27:24.069055 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:24.069025 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:27:30.342272 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:30.342228 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:27:30.404935 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:30.404896 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:27:34.256652 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.256591 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:27:34.651295 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.651211 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:27:34.747980 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.747941 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:27:34.834489 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.834457 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b"] Jul 15 17:27:34.835029 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.835007 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a206ebd-a423-4623-8029-097359f73b7c" containerName="maas-api" Jul 15 17:27:34.835096 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.835035 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a206ebd-a423-4623-8029-097359f73b7c" containerName="maas-api" Jul 15 17:27:34.835096 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.835050 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6736155-c8c4-4206-b3dc-a5a56ffce63a" containerName="revoke-keys" Jul 15 17:27:34.835096 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.835057 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6736155-c8c4-4206-b3dc-a5a56ffce63a" containerName="revoke-keys" Jul 15 17:27:34.835096 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.835085 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4040b104-ab36-44de-9196-dbc947a3830a" containerName="revoke-keys" Jul 15 17:27:34.835096 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.835093 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4040b104-ab36-44de-9196-dbc947a3830a" containerName="revoke-keys" Jul 15 17:27:34.835267 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.835194 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4040b104-ab36-44de-9196-dbc947a3830a" containerName="revoke-keys" Jul 15 17:27:34.835267 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.835206 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6736155-c8c4-4206-b3dc-a5a56ffce63a" containerName="revoke-keys" Jul 15 17:27:34.835267 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.835221 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a206ebd-a423-4623-8029-097359f73b7c" containerName="maas-api" Jul 15 17:27:34.838803 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.838777 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:34.841426 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.841397 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-2a15ad60-gw-openshift-default-dockercfg-dkrvs\"" Jul 15 17:27:34.841550 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.841538 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 15 17:27:34.851142 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.851120 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b"] Jul 15 17:27:34.870833 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.870804 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:27:34.877073 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.877052 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e5f5734-0a9f-489b-b687-da6d083bf8ca-istiod-ca-cert\") pod \"e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:34.877173 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.877088 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e5f5734-0a9f-489b-b687-da6d083bf8ca-workload-certs\") pod \"e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:34.877173 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.877110 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e5f5734-0a9f-489b-b687-da6d083bf8ca-istio-podinfo\") pod \"e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:34.877173 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.877137 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e5f5734-0a9f-489b-b687-da6d083bf8ca-credential-socket\") pod \"e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:34.877173 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.877172 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e5f5734-0a9f-489b-b687-da6d083bf8ca-istio-envoy\") pod \"e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:34.877324 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.877191 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e5f5734-0a9f-489b-b687-da6d083bf8ca-istio-token\") pod \"e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:34.877324 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.877243 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e5f5734-0a9f-489b-b687-da6d083bf8ca-istio-data\") pod \"e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:34.877324 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.877292 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e5f5734-0a9f-489b-b687-da6d083bf8ca-workload-socket\") pod \"e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:34.877324 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.877311 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gl24\" (UniqueName: \"kubernetes.io/projected/9e5f5734-0a9f-489b-b687-da6d083bf8ca-kube-api-access-5gl24\") pod \"e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:34.977849 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.977808 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e5f5734-0a9f-489b-b687-da6d083bf8ca-workload-socket\") pod \"e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:34.977849 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.977856 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gl24\" (UniqueName: \"kubernetes.io/projected/9e5f5734-0a9f-489b-b687-da6d083bf8ca-kube-api-access-5gl24\") pod \"e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:34.978087 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.977911 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e5f5734-0a9f-489b-b687-da6d083bf8ca-istiod-ca-cert\") pod \"e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:34.978087 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.977939 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e5f5734-0a9f-489b-b687-da6d083bf8ca-workload-certs\") pod \"e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:34.978087 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.977973 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e5f5734-0a9f-489b-b687-da6d083bf8ca-istio-podinfo\") pod \"e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:34.978087 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.978013 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e5f5734-0a9f-489b-b687-da6d083bf8ca-credential-socket\") pod \"e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:34.978087 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.978064 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e5f5734-0a9f-489b-b687-da6d083bf8ca-istio-envoy\") pod \"e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:34.978370 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.978092 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e5f5734-0a9f-489b-b687-da6d083bf8ca-istio-token\") pod \"e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:34.978370 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.978122 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e5f5734-0a9f-489b-b687-da6d083bf8ca-istio-data\") pod \"e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:34.978370 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.978273 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e5f5734-0a9f-489b-b687-da6d083bf8ca-workload-socket\") pod \"e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:34.978370 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.978312 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e5f5734-0a9f-489b-b687-da6d083bf8ca-workload-certs\") pod \"e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:34.978370 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.978364 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e5f5734-0a9f-489b-b687-da6d083bf8ca-credential-socket\") pod \"e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:34.978653 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.978422 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e5f5734-0a9f-489b-b687-da6d083bf8ca-istio-data\") pod \"e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:34.978975 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.978954 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e5f5734-0a9f-489b-b687-da6d083bf8ca-istiod-ca-cert\") pod \"e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:34.980320 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.980300 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e5f5734-0a9f-489b-b687-da6d083bf8ca-istio-envoy\") pod \"e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:34.980575 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.980557 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e5f5734-0a9f-489b-b687-da6d083bf8ca-istio-podinfo\") pod \"e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:34.987013 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.986984 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gl24\" (UniqueName: \"kubernetes.io/projected/9e5f5734-0a9f-489b-b687-da6d083bf8ca-kube-api-access-5gl24\") pod \"e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:34.987205 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:34.987183 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e5f5734-0a9f-489b-b687-da6d083bf8ca-istio-token\") pod \"e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:35.152686 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:35.152653 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:35.286955 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:35.286930 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b"] Jul 15 17:27:35.288565 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:27:35.288533 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e5f5734_0a9f_489b_b687_da6d083bf8ca.slice/crio-a640857e85b1f14c86fb61f30ecccd833ffaecba74626595665ee06b39b882bd WatchSource:0}: Error finding container a640857e85b1f14c86fb61f30ecccd833ffaecba74626595665ee06b39b882bd: Status 404 returned error can't find the container with id a640857e85b1f14c86fb61f30ecccd833ffaecba74626595665ee06b39b882bd Jul 15 17:27:36.060557 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:36.060519 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" event={"ID":"9e5f5734-0a9f-489b-b687-da6d083bf8ca","Type":"ContainerStarted","Data":"a640857e85b1f14c86fb61f30ecccd833ffaecba74626595665ee06b39b882bd"} Jul 15 17:27:36.296679 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:36.296639 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:27:36.438624 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:36.438574 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:27:36.845959 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:36.845825 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:27:37.010305 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:37.010267 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:27:37.110285 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:37.110196 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:27:38.175286 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:38.175250 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 17:27:38.175579 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:38.175328 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 17:27:38.175579 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:38.175360 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 17:27:39.082539 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:39.082496 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" event={"ID":"9e5f5734-0a9f-489b-b687-da6d083bf8ca","Type":"ContainerStarted","Data":"220c251f2af39b477c2b80cfdec7cb698048631ba988ec7d2171c6ecbeaf0050"} Jul 15 17:27:39.104538 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:39.104487 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" podStartSLOduration=2.22018288 podStartE2EDuration="5.104472823s" totalImagesPullingTime="2.884289943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:27:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:27:39.103147705 +0000 UTC m=+1876.656277918" watchObservedRunningTime="2026-07-15 17:27:39.104472823 +0000 UTC m=+1876.657603084" Jul 15 17:27:39.153371 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:39.153343 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:39.158005 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:39.157979 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:40.087853 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:40.087819 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:40.088753 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:40.088733 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:40.681511 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:40.681474 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:27:40.745287 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:40.745255 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:27:43.980177 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:43.980147 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:27:51.214908 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:51.214871 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:27:52.155490 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:52.155456 2578 generic.go:356] "Generic (PLEG): container finished" podID="9e5f5734-0a9f-489b-b687-da6d083bf8ca" containerID="220c251f2af39b477c2b80cfdec7cb698048631ba988ec7d2171c6ecbeaf0050" exitCode=0 Jul 15 17:27:52.155676 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:52.155534 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" event={"ID":"9e5f5734-0a9f-489b-b687-da6d083bf8ca","Type":"ContainerDied","Data":"220c251f2af39b477c2b80cfdec7cb698048631ba988ec7d2171c6ecbeaf0050"} Jul 15 17:27:52.155943 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:52.155927 2578 scope.go:117] "RemoveContainer" containerID="220c251f2af39b477c2b80cfdec7cb698048631ba988ec7d2171c6ecbeaf0050" Jul 15 17:27:52.157966 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:52.157937 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 17:27:52.158045 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:52.158004 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 17:27:52.158045 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:52.158035 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 17:27:53.170617 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:53.170559 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" event={"ID":"9e5f5734-0a9f-489b-b687-da6d083bf8ca","Type":"ContainerStarted","Data":"0b3fcab5d56ee47b978a626dfb0588b42aa0fcfd81a46660aa138b164300a7a3"} Jul 15 17:27:54.154113 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:54.154070 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:54.154326 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:54.154240 2578 patch_prober.go:28] interesting pod/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 15 17:27:54.154326 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:54.154302 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" podUID="9e5f5734-0a9f-489b-b687-da6d083bf8ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 15 17:27:55.087996 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:55.087961 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:55.154144 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:55.154102 2578 patch_prober.go:28] interesting pod/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 15 17:27:55.154424 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:55.154395 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" podUID="9e5f5734-0a9f-489b-b687-da6d083bf8ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 15 17:27:55.854638 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:55.854590 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:27:56.154003 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:56.153896 2578 patch_prober.go:28] interesting pod/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 15 17:27:56.154003 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:56.153962 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" podUID="9e5f5734-0a9f-489b-b687-da6d083bf8ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 15 17:27:57.157528 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:57.157498 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:27:57.158476 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:27:57.158458 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:28:02.080857 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:02.080818 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:28:02.142088 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:02.142039 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:28:06.040645 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:06.040609 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b"] Jul 15 17:28:06.041114 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:06.040857 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" podUID="9e5f5734-0a9f-489b-b687-da6d083bf8ca" containerName="istio-proxy" containerID="cri-o://0b3fcab5d56ee47b978a626dfb0588b42aa0fcfd81a46660aa138b164300a7a3" gracePeriod=30 Jul 15 17:28:06.057394 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:06.057363 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:28:06.383531 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:06.383499 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:28:07.553138 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:07.553091 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:28:07.626065 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:07.626030 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:28:07.711114 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:07.711075 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:28:07.993350 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:07.993315 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:28:08.359685 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:08.359573 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:28:08.460275 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:08.460214 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:28:11.270128 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:11.270096 2578 generic.go:356] "Generic (PLEG): container finished" podID="9e5f5734-0a9f-489b-b687-da6d083bf8ca" containerID="0b3fcab5d56ee47b978a626dfb0588b42aa0fcfd81a46660aa138b164300a7a3" exitCode=0 Jul 15 17:28:11.270543 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:11.270174 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" event={"ID":"9e5f5734-0a9f-489b-b687-da6d083bf8ca","Type":"ContainerDied","Data":"0b3fcab5d56ee47b978a626dfb0588b42aa0fcfd81a46660aa138b164300a7a3"} Jul 15 17:28:11.270972 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:11.270288 2578 scope.go:117] "RemoveContainer" containerID="220c251f2af39b477c2b80cfdec7cb698048631ba988ec7d2171c6ecbeaf0050" Jul 15 17:28:11.296160 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:11.296143 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:28:11.409388 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:11.409356 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e5f5734-0a9f-489b-b687-da6d083bf8ca-workload-socket\") pod \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " Jul 15 17:28:11.409547 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:11.409408 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e5f5734-0a9f-489b-b687-da6d083bf8ca-workload-certs\") pod \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " Jul 15 17:28:11.409547 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:11.409441 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e5f5734-0a9f-489b-b687-da6d083bf8ca-istio-data\") pod \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " Jul 15 17:28:11.409547 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:11.409466 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e5f5734-0a9f-489b-b687-da6d083bf8ca-istiod-ca-cert\") pod \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " Jul 15 17:28:11.409547 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:11.409482 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e5f5734-0a9f-489b-b687-da6d083bf8ca-istio-podinfo\") pod \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " Jul 15 17:28:11.409547 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:11.409499 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e5f5734-0a9f-489b-b687-da6d083bf8ca-credential-socket\") pod \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " Jul 15 17:28:11.409547 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:11.409521 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e5f5734-0a9f-489b-b687-da6d083bf8ca-istio-token\") pod \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " Jul 15 17:28:11.409547 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:11.409543 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5gl24\" (UniqueName: \"kubernetes.io/projected/9e5f5734-0a9f-489b-b687-da6d083bf8ca-kube-api-access-5gl24\") pod \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " Jul 15 17:28:11.409949 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:11.409582 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e5f5734-0a9f-489b-b687-da6d083bf8ca-istio-envoy\") pod \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\" (UID: \"9e5f5734-0a9f-489b-b687-da6d083bf8ca\") " Jul 15 17:28:11.409949 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:11.409684 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e5f5734-0a9f-489b-b687-da6d083bf8ca-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9e5f5734-0a9f-489b-b687-da6d083bf8ca" (UID: "9e5f5734-0a9f-489b-b687-da6d083bf8ca"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:28:11.409949 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:11.409757 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e5f5734-0a9f-489b-b687-da6d083bf8ca-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9e5f5734-0a9f-489b-b687-da6d083bf8ca" (UID: "9e5f5734-0a9f-489b-b687-da6d083bf8ca"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:28:11.409949 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:11.409907 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e5f5734-0a9f-489b-b687-da6d083bf8ca-workload-certs\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:28:11.409949 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:11.409927 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e5f5734-0a9f-489b-b687-da6d083bf8ca-workload-socket\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:28:11.410208 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:11.410136 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e5f5734-0a9f-489b-b687-da6d083bf8ca-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9e5f5734-0a9f-489b-b687-da6d083bf8ca" (UID: "9e5f5734-0a9f-489b-b687-da6d083bf8ca"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:28:11.410208 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:11.410068 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e5f5734-0a9f-489b-b687-da6d083bf8ca-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9e5f5734-0a9f-489b-b687-da6d083bf8ca" (UID: "9e5f5734-0a9f-489b-b687-da6d083bf8ca"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:28:11.410323 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:11.410297 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e5f5734-0a9f-489b-b687-da6d083bf8ca-istio-data" (OuterVolumeSpecName: "istio-data") pod "9e5f5734-0a9f-489b-b687-da6d083bf8ca" (UID: "9e5f5734-0a9f-489b-b687-da6d083bf8ca"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:28:11.411999 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:11.411971 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9e5f5734-0a9f-489b-b687-da6d083bf8ca-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9e5f5734-0a9f-489b-b687-da6d083bf8ca" (UID: "9e5f5734-0a9f-489b-b687-da6d083bf8ca"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 17:28:11.411999 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:11.411986 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e5f5734-0a9f-489b-b687-da6d083bf8ca-kube-api-access-5gl24" (OuterVolumeSpecName: "kube-api-access-5gl24") pod "9e5f5734-0a9f-489b-b687-da6d083bf8ca" (UID: "9e5f5734-0a9f-489b-b687-da6d083bf8ca"). InnerVolumeSpecName "kube-api-access-5gl24". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:28:11.412316 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:11.412294 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e5f5734-0a9f-489b-b687-da6d083bf8ca-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9e5f5734-0a9f-489b-b687-da6d083bf8ca" (UID: "9e5f5734-0a9f-489b-b687-da6d083bf8ca"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:28:11.412405 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:11.412323 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e5f5734-0a9f-489b-b687-da6d083bf8ca-istio-token" (OuterVolumeSpecName: "istio-token") pod "9e5f5734-0a9f-489b-b687-da6d083bf8ca" (UID: "9e5f5734-0a9f-489b-b687-da6d083bf8ca"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:28:11.510819 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:11.510779 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e5f5734-0a9f-489b-b687-da6d083bf8ca-istio-data\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:28:11.510819 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:11.510807 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e5f5734-0a9f-489b-b687-da6d083bf8ca-istiod-ca-cert\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:28:11.510819 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:11.510817 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e5f5734-0a9f-489b-b687-da6d083bf8ca-istio-podinfo\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:28:11.510819 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:11.510825 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e5f5734-0a9f-489b-b687-da6d083bf8ca-credential-socket\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:28:11.511175 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:11.510835 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e5f5734-0a9f-489b-b687-da6d083bf8ca-istio-token\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:28:11.511175 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:11.510843 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gl24\" (UniqueName: \"kubernetes.io/projected/9e5f5734-0a9f-489b-b687-da6d083bf8ca-kube-api-access-5gl24\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:28:11.511175 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:11.510851 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e5f5734-0a9f-489b-b687-da6d083bf8ca-istio-envoy\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:28:12.277942 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:12.277905 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" event={"ID":"9e5f5734-0a9f-489b-b687-da6d083bf8ca","Type":"ContainerDied","Data":"a640857e85b1f14c86fb61f30ecccd833ffaecba74626595665ee06b39b882bd"} Jul 15 17:28:12.277942 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:12.277924 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b" Jul 15 17:28:12.278456 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:12.277973 2578 scope.go:117] "RemoveContainer" containerID="0b3fcab5d56ee47b978a626dfb0588b42aa0fcfd81a46660aa138b164300a7a3" Jul 15 17:28:12.311419 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:12.311390 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b"] Jul 15 17:28:12.317428 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:12.317405 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-2a15ad60-gw-openshift-default-54c875bfc5-wp58b"] Jul 15 17:28:12.502652 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:12.502622 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:28:12.568731 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:12.568643 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:28:12.991108 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:12.991075 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e5f5734-0a9f-489b-b687-da6d083bf8ca" path="/var/lib/kubelet/pods/9e5f5734-0a9f-489b-b687-da6d083bf8ca/volumes" Jul 15 17:28:15.593872 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:15.593840 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:28:45.801613 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:28:45.801564 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:29:02.607480 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:02.607437 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ts4vc"] Jul 15 17:29:02.608088 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:02.607802 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e5f5734-0a9f-489b-b687-da6d083bf8ca" containerName="istio-proxy" Jul 15 17:29:02.608088 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:02.607814 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e5f5734-0a9f-489b-b687-da6d083bf8ca" containerName="istio-proxy" Jul 15 17:29:02.608088 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:02.607823 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e5f5734-0a9f-489b-b687-da6d083bf8ca" containerName="istio-proxy" Jul 15 17:29:02.608088 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:02.607829 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e5f5734-0a9f-489b-b687-da6d083bf8ca" containerName="istio-proxy" Jul 15 17:29:02.608088 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:02.607902 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e5f5734-0a9f-489b-b687-da6d083bf8ca" containerName="istio-proxy" Jul 15 17:29:02.608088 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:02.607912 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e5f5734-0a9f-489b-b687-da6d083bf8ca" containerName="istio-proxy" Jul 15 17:29:02.610508 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:02.610492 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ts4vc" Jul 15 17:29:02.613199 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:02.613178 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 17:29:02.613321 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:02.613180 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dlhgl\"" Jul 15 17:29:02.618674 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:02.618645 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ts4vc"] Jul 15 17:29:02.704735 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:02.704700 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1aa7614b-e873-4db5-a3cb-6715c1cce017-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e7083808-ts4vc\" (UID: \"1aa7614b-e873-4db5-a3cb-6715c1cce017\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ts4vc" Jul 15 17:29:02.806035 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:02.805999 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1aa7614b-e873-4db5-a3cb-6715c1cce017-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e7083808-ts4vc\" (UID: \"1aa7614b-e873-4db5-a3cb-6715c1cce017\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ts4vc" Jul 15 17:29:02.806644 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:02.806628 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1aa7614b-e873-4db5-a3cb-6715c1cce017-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e7083808-ts4vc\" (UID: \"1aa7614b-e873-4db5-a3cb-6715c1cce017\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ts4vc" Jul 15 17:29:02.923262 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:02.923234 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ts4vc" Jul 15 17:29:03.268859 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:03.268831 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ts4vc"] Jul 15 17:29:03.270565 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:29:03.270540 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aa7614b_e873_4db5_a3cb_6715c1cce017.slice/crio-86c4bf615a79f56966a151922ee63c5a1f6b807cac63dc27b209d7e153da85fe WatchSource:0}: Error finding container 86c4bf615a79f56966a151922ee63c5a1f6b807cac63dc27b209d7e153da85fe: Status 404 returned error can't find the container with id 86c4bf615a79f56966a151922ee63c5a1f6b807cac63dc27b209d7e153da85fe Jul 15 17:29:03.551139 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:03.551043 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ts4vc" event={"ID":"1aa7614b-e873-4db5-a3cb-6715c1cce017","Type":"ContainerStarted","Data":"f43c65538e7804bd08ee8d3ed195df9f6a20d5caf3f92835a897930fb8fa2690"} Jul 15 17:29:03.551139 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:03.551097 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ts4vc" event={"ID":"1aa7614b-e873-4db5-a3cb-6715c1cce017","Type":"ContainerStarted","Data":"86c4bf615a79f56966a151922ee63c5a1f6b807cac63dc27b209d7e153da85fe"} Jul 15 17:29:03.568055 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:03.568012 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ts4vc" podStartSLOduration=1.567997467 podStartE2EDuration="1.567997467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:29:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:29:03.566823098 +0000 UTC m=+1961.119953349" watchObservedRunningTime="2026-07-15 17:29:03.567997467 +0000 UTC m=+1961.121127700" Jul 15 17:29:04.557825 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:04.557790 2578 generic.go:356] "Generic (PLEG): container finished" podID="1aa7614b-e873-4db5-a3cb-6715c1cce017" containerID="f43c65538e7804bd08ee8d3ed195df9f6a20d5caf3f92835a897930fb8fa2690" exitCode=0 Jul 15 17:29:04.558219 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:04.557841 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ts4vc" event={"ID":"1aa7614b-e873-4db5-a3cb-6715c1cce017","Type":"ContainerDied","Data":"f43c65538e7804bd08ee8d3ed195df9f6a20d5caf3f92835a897930fb8fa2690"} Jul 15 17:29:05.689683 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:05.689658 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ts4vc" Jul 15 17:29:05.730321 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:05.730290 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1aa7614b-e873-4db5-a3cb-6715c1cce017-maas-api-service-ca\") pod \"1aa7614b-e873-4db5-a3cb-6715c1cce017\" (UID: \"1aa7614b-e873-4db5-a3cb-6715c1cce017\") " Jul 15 17:29:05.730656 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:05.730634 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1aa7614b-e873-4db5-a3cb-6715c1cce017-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1aa7614b-e873-4db5-a3cb-6715c1cce017" (UID: "1aa7614b-e873-4db5-a3cb-6715c1cce017"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:29:05.831833 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:05.831748 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1aa7614b-e873-4db5-a3cb-6715c1cce017-maas-api-service-ca\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:29:06.570935 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:06.570897 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ts4vc" event={"ID":"1aa7614b-e873-4db5-a3cb-6715c1cce017","Type":"ContainerDied","Data":"86c4bf615a79f56966a151922ee63c5a1f6b807cac63dc27b209d7e153da85fe"} Jul 15 17:29:06.571147 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:06.570953 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="86c4bf615a79f56966a151922ee63c5a1f6b807cac63dc27b209d7e153da85fe" Jul 15 17:29:06.571147 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:06.570908 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ts4vc" Jul 15 17:29:11.195719 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:11.195686 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:29:11.296755 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:11.296724 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:29:13.168506 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:13.168469 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:29:13.266460 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:13.266414 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:29:13.412689 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:13.412658 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:29:17.694660 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:17.694559 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:29:23.690667 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:23.690617 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:29:23.791392 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:23.791354 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:29:28.717981 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:28.717940 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:29:29.001785 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.001758 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj"] Jul 15 17:29:29.002121 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.002107 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1aa7614b-e873-4db5-a3cb-6715c1cce017" containerName="revoke-keys" Jul 15 17:29:29.002121 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.002121 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1aa7614b-e873-4db5-a3cb-6715c1cce017" containerName="revoke-keys" Jul 15 17:29:29.002237 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.002188 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1aa7614b-e873-4db5-a3cb-6715c1cce017" containerName="revoke-keys" Jul 15 17:29:29.005056 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.005040 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:29.008171 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.008147 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-65f681ff-openshift-default-dockercfg-vcz99\"" Jul 15 17:29:29.008706 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.008690 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 15 17:29:29.028918 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.028889 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:29:29.035256 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.035235 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj"] Jul 15 17:29:29.135199 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.135164 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cddf2c6-5652-4784-a080-49ad81bfbf1d-workload-certs\") pod \"e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:29.135199 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.135206 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cddf2c6-5652-4784-a080-49ad81bfbf1d-istio-data\") pod \"e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:29.135427 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.135229 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cddf2c6-5652-4784-a080-49ad81bfbf1d-istio-token\") pod \"e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:29.135427 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.135247 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qblkr\" (UniqueName: \"kubernetes.io/projected/9cddf2c6-5652-4784-a080-49ad81bfbf1d-kube-api-access-qblkr\") pod \"e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:29.135427 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.135359 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cddf2c6-5652-4784-a080-49ad81bfbf1d-istio-envoy\") pod \"e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:29.135427 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.135405 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cddf2c6-5652-4784-a080-49ad81bfbf1d-istio-podinfo\") pod \"e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:29.135578 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.135439 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cddf2c6-5652-4784-a080-49ad81bfbf1d-credential-socket\") pod \"e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:29.135578 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.135504 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cddf2c6-5652-4784-a080-49ad81bfbf1d-istiod-ca-cert\") pod \"e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:29.135578 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.135544 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cddf2c6-5652-4784-a080-49ad81bfbf1d-workload-socket\") pod \"e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:29.236423 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.236388 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cddf2c6-5652-4784-a080-49ad81bfbf1d-istio-envoy\") pod \"e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:29.236614 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.236434 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cddf2c6-5652-4784-a080-49ad81bfbf1d-istio-podinfo\") pod \"e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:29.236614 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.236464 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cddf2c6-5652-4784-a080-49ad81bfbf1d-credential-socket\") pod \"e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:29.236614 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.236510 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cddf2c6-5652-4784-a080-49ad81bfbf1d-istiod-ca-cert\") pod \"e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:29.236857 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.236673 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cddf2c6-5652-4784-a080-49ad81bfbf1d-workload-socket\") pod \"e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:29.237059 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.237039 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cddf2c6-5652-4784-a080-49ad81bfbf1d-workload-certs\") pod \"e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:29.237501 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.236920 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cddf2c6-5652-4784-a080-49ad81bfbf1d-credential-socket\") pod \"e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:29.237501 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.237075 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cddf2c6-5652-4784-a080-49ad81bfbf1d-workload-socket\") pod \"e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:29.237501 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.237214 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cddf2c6-5652-4784-a080-49ad81bfbf1d-istio-data\") pod \"e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:29.237501 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.237252 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cddf2c6-5652-4784-a080-49ad81bfbf1d-istiod-ca-cert\") pod \"e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:29.237501 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.237282 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cddf2c6-5652-4784-a080-49ad81bfbf1d-istio-token\") pod \"e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:29.237501 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.237311 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qblkr\" (UniqueName: \"kubernetes.io/projected/9cddf2c6-5652-4784-a080-49ad81bfbf1d-kube-api-access-qblkr\") pod \"e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:29.237501 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.237410 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cddf2c6-5652-4784-a080-49ad81bfbf1d-workload-certs\") pod \"e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:29.237501 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.237449 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cddf2c6-5652-4784-a080-49ad81bfbf1d-istio-data\") pod \"e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:29.238782 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.238762 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cddf2c6-5652-4784-a080-49ad81bfbf1d-istio-envoy\") pod \"e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:29.239347 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.239317 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cddf2c6-5652-4784-a080-49ad81bfbf1d-istio-podinfo\") pod \"e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:29.245676 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.245621 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cddf2c6-5652-4784-a080-49ad81bfbf1d-istio-token\") pod \"e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:29.245975 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.245958 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qblkr\" (UniqueName: \"kubernetes.io/projected/9cddf2c6-5652-4784-a080-49ad81bfbf1d-kube-api-access-qblkr\") pod \"e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:29.314741 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.314712 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:29.450162 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.450131 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj"] Jul 15 17:29:29.451904 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:29:29.451880 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cddf2c6_5652_4784_a080_49ad81bfbf1d.slice/crio-0d92668f1ceb31f456a509f51a2f8cbaff6dcc0100c2a92be4af4e6880784dfa WatchSource:0}: Error finding container 0d92668f1ceb31f456a509f51a2f8cbaff6dcc0100c2a92be4af4e6880784dfa: Status 404 returned error can't find the container with id 0d92668f1ceb31f456a509f51a2f8cbaff6dcc0100c2a92be4af4e6880784dfa Jul 15 17:29:29.453954 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.453927 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 17:29:29.454022 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.453991 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 17:29:29.454063 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.454020 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 17:29:29.710453 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.710416 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" event={"ID":"9cddf2c6-5652-4784-a080-49ad81bfbf1d","Type":"ContainerStarted","Data":"7480dc964300d92eafb69ab1d06cd666919336fbf31cbf27b376f4426618464f"} Jul 15 17:29:29.710669 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.710467 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" event={"ID":"9cddf2c6-5652-4784-a080-49ad81bfbf1d","Type":"ContainerStarted","Data":"0d92668f1ceb31f456a509f51a2f8cbaff6dcc0100c2a92be4af4e6880784dfa"} Jul 15 17:29:29.732049 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:29.731999 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" podStartSLOduration=1.731982438 podStartE2EDuration="1.731982438s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:29:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:29:29.730496153 +0000 UTC m=+1987.283626401" watchObservedRunningTime="2026-07-15 17:29:29.731982438 +0000 UTC m=+1987.285112706" Jul 15 17:29:30.227968 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:30.227920 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:29:30.314665 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:30.314637 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:29:30.315443 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:30.315422 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:30.320202 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:30.320176 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:30.384974 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:30.384932 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:29:30.716101 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:30.716069 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:30.717093 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:30.717074 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:30.827005 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:30.826972 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:29:30.928577 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:30.928535 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:29:31.025838 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:31.025750 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:29:35.088399 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:35.088351 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:29:35.155581 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:35.155549 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:29:38.441419 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:38.441383 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:29:45.718094 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:45.718062 2578 patch_prober.go:28] interesting pod/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 15 17:29:45.718542 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:45.718119 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" podUID="9cddf2c6-5652-4784-a080-49ad81bfbf1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 15 17:29:45.805150 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:45.805118 2578 generic.go:356] "Generic (PLEG): container finished" podID="9cddf2c6-5652-4784-a080-49ad81bfbf1d" containerID="7480dc964300d92eafb69ab1d06cd666919336fbf31cbf27b376f4426618464f" exitCode=0 Jul 15 17:29:45.805315 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:45.805156 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" event={"ID":"9cddf2c6-5652-4784-a080-49ad81bfbf1d","Type":"ContainerDied","Data":"7480dc964300d92eafb69ab1d06cd666919336fbf31cbf27b376f4426618464f"} Jul 15 17:29:45.805537 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:45.805522 2578 scope.go:117] "RemoveContainer" containerID="7480dc964300d92eafb69ab1d06cd666919336fbf31cbf27b376f4426618464f" Jul 15 17:29:45.806548 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:45.806516 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 17:29:45.806675 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:45.806616 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 17:29:45.806675 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:45.806664 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 17:29:46.815366 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:46.815324 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" event={"ID":"9cddf2c6-5652-4784-a080-49ad81bfbf1d","Type":"ContainerStarted","Data":"b1bdd3f0db8b4850aaa29a99dc2f9efe1e4b2437610c235d8cce52112554f489"} Jul 15 17:29:47.315316 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:47.315276 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:47.315548 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:47.315525 2578 patch_prober.go:28] interesting pod/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 15 17:29:47.315622 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:47.315573 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" podUID="9cddf2c6-5652-4784-a080-49ad81bfbf1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 15 17:29:48.315682 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:48.315640 2578 patch_prober.go:28] interesting pod/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 15 17:29:48.316113 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:48.315699 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" podUID="9cddf2c6-5652-4784-a080-49ad81bfbf1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 15 17:29:49.315316 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:49.315283 2578 patch_prober.go:28] interesting pod/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 15 17:29:49.315513 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:49.315350 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" podUID="9cddf2c6-5652-4784-a080-49ad81bfbf1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 15 17:29:50.315410 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:50.315379 2578 patch_prober.go:28] interesting pod/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 15 17:29:50.315789 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:50.315432 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" podUID="9cddf2c6-5652-4784-a080-49ad81bfbf1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 15 17:29:51.319683 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:51.319651 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:51.320106 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:51.319926 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:29:51.320578 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:29:51.320560 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:30:34.074691 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:30:34.074655 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:30:45.213681 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:30:45.213577 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:30:49.707020 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:30:49.706981 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:30:56.090616 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:30:56.090564 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:30:56.187040 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:30:56.186995 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:30:59.890407 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:30:59.890370 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj"] Jul 15 17:30:59.890840 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:30:59.890611 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" podUID="9cddf2c6-5652-4784-a080-49ad81bfbf1d" containerName="istio-proxy" containerID="cri-o://b1bdd3f0db8b4850aaa29a99dc2f9efe1e4b2437610c235d8cce52112554f489" gracePeriod=30 Jul 15 17:30:59.910020 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:30:59.909987 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:00.515145 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:00.515113 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:00.616780 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:00.616742 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:00.716262 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:00.716227 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:02.188427 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:02.188389 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:05.134672 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.134645 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:31:05.244878 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.244845 2578 generic.go:356] "Generic (PLEG): container finished" podID="9cddf2c6-5652-4784-a080-49ad81bfbf1d" containerID="b1bdd3f0db8b4850aaa29a99dc2f9efe1e4b2437610c235d8cce52112554f489" exitCode=0 Jul 15 17:31:05.245044 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.244903 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" Jul 15 17:31:05.245044 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.244938 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" event={"ID":"9cddf2c6-5652-4784-a080-49ad81bfbf1d","Type":"ContainerDied","Data":"b1bdd3f0db8b4850aaa29a99dc2f9efe1e4b2437610c235d8cce52112554f489"} Jul 15 17:31:05.245044 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.245030 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj" event={"ID":"9cddf2c6-5652-4784-a080-49ad81bfbf1d","Type":"ContainerDied","Data":"0d92668f1ceb31f456a509f51a2f8cbaff6dcc0100c2a92be4af4e6880784dfa"} Jul 15 17:31:05.245156 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.245061 2578 scope.go:117] "RemoveContainer" containerID="b1bdd3f0db8b4850aaa29a99dc2f9efe1e4b2437610c235d8cce52112554f489" Jul 15 17:31:05.253801 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.253785 2578 scope.go:117] "RemoveContainer" containerID="7480dc964300d92eafb69ab1d06cd666919336fbf31cbf27b376f4426618464f" Jul 15 17:31:05.261865 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.261848 2578 scope.go:117] "RemoveContainer" containerID="b1bdd3f0db8b4850aaa29a99dc2f9efe1e4b2437610c235d8cce52112554f489" Jul 15 17:31:05.262105 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:31:05.262087 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1bdd3f0db8b4850aaa29a99dc2f9efe1e4b2437610c235d8cce52112554f489\": container with ID starting with b1bdd3f0db8b4850aaa29a99dc2f9efe1e4b2437610c235d8cce52112554f489 not found: ID does not exist" containerID="b1bdd3f0db8b4850aaa29a99dc2f9efe1e4b2437610c235d8cce52112554f489" Jul 15 17:31:05.262165 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.262112 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1bdd3f0db8b4850aaa29a99dc2f9efe1e4b2437610c235d8cce52112554f489"} err="failed to get container status \"b1bdd3f0db8b4850aaa29a99dc2f9efe1e4b2437610c235d8cce52112554f489\": rpc error: code = NotFound desc = could not find container \"b1bdd3f0db8b4850aaa29a99dc2f9efe1e4b2437610c235d8cce52112554f489\": container with ID starting with b1bdd3f0db8b4850aaa29a99dc2f9efe1e4b2437610c235d8cce52112554f489 not found: ID does not exist" Jul 15 17:31:05.262165 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.262129 2578 scope.go:117] "RemoveContainer" containerID="7480dc964300d92eafb69ab1d06cd666919336fbf31cbf27b376f4426618464f" Jul 15 17:31:05.262350 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:31:05.262334 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7480dc964300d92eafb69ab1d06cd666919336fbf31cbf27b376f4426618464f\": container with ID starting with 7480dc964300d92eafb69ab1d06cd666919336fbf31cbf27b376f4426618464f not found: ID does not exist" containerID="7480dc964300d92eafb69ab1d06cd666919336fbf31cbf27b376f4426618464f" Jul 15 17:31:05.262391 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.262354 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7480dc964300d92eafb69ab1d06cd666919336fbf31cbf27b376f4426618464f"} err="failed to get container status \"7480dc964300d92eafb69ab1d06cd666919336fbf31cbf27b376f4426618464f\": rpc error: code = NotFound desc = could not find container \"7480dc964300d92eafb69ab1d06cd666919336fbf31cbf27b376f4426618464f\": container with ID starting with 7480dc964300d92eafb69ab1d06cd666919336fbf31cbf27b376f4426618464f not found: ID does not exist" Jul 15 17:31:05.274755 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.274734 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qblkr\" (UniqueName: \"kubernetes.io/projected/9cddf2c6-5652-4784-a080-49ad81bfbf1d-kube-api-access-qblkr\") pod \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " Jul 15 17:31:05.274813 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.274785 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cddf2c6-5652-4784-a080-49ad81bfbf1d-istiod-ca-cert\") pod \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " Jul 15 17:31:05.274857 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.274817 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cddf2c6-5652-4784-a080-49ad81bfbf1d-workload-certs\") pod \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " Jul 15 17:31:05.274857 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.274838 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cddf2c6-5652-4784-a080-49ad81bfbf1d-istio-token\") pod \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " Jul 15 17:31:05.274935 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.274861 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cddf2c6-5652-4784-a080-49ad81bfbf1d-workload-socket\") pod \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " Jul 15 17:31:05.274935 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.274899 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cddf2c6-5652-4784-a080-49ad81bfbf1d-istio-envoy\") pod \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " Jul 15 17:31:05.274935 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.274929 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cddf2c6-5652-4784-a080-49ad81bfbf1d-istio-data\") pod \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " Jul 15 17:31:05.275083 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.274955 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cddf2c6-5652-4784-a080-49ad81bfbf1d-credential-socket\") pod \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " Jul 15 17:31:05.275083 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.274986 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cddf2c6-5652-4784-a080-49ad81bfbf1d-istio-podinfo\") pod \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\" (UID: \"9cddf2c6-5652-4784-a080-49ad81bfbf1d\") " Jul 15 17:31:05.275190 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.275098 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cddf2c6-5652-4784-a080-49ad81bfbf1d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9cddf2c6-5652-4784-a080-49ad81bfbf1d" (UID: "9cddf2c6-5652-4784-a080-49ad81bfbf1d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:31:05.275190 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.275162 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cddf2c6-5652-4784-a080-49ad81bfbf1d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9cddf2c6-5652-4784-a080-49ad81bfbf1d" (UID: "9cddf2c6-5652-4784-a080-49ad81bfbf1d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:31:05.275308 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.275253 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cddf2c6-5652-4784-a080-49ad81bfbf1d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9cddf2c6-5652-4784-a080-49ad81bfbf1d" (UID: "9cddf2c6-5652-4784-a080-49ad81bfbf1d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:31:05.275308 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.275291 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cddf2c6-5652-4784-a080-49ad81bfbf1d-workload-certs\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:31:05.275429 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.275309 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cddf2c6-5652-4784-a080-49ad81bfbf1d-workload-socket\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:31:05.275429 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.275343 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cddf2c6-5652-4784-a080-49ad81bfbf1d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9cddf2c6-5652-4784-a080-49ad81bfbf1d" (UID: "9cddf2c6-5652-4784-a080-49ad81bfbf1d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:31:05.275864 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.275835 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cddf2c6-5652-4784-a080-49ad81bfbf1d-istio-data" (OuterVolumeSpecName: "istio-data") pod "9cddf2c6-5652-4784-a080-49ad81bfbf1d" (UID: "9cddf2c6-5652-4784-a080-49ad81bfbf1d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:31:05.276922 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.276900 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cddf2c6-5652-4784-a080-49ad81bfbf1d-kube-api-access-qblkr" (OuterVolumeSpecName: "kube-api-access-qblkr") pod "9cddf2c6-5652-4784-a080-49ad81bfbf1d" (UID: "9cddf2c6-5652-4784-a080-49ad81bfbf1d"). InnerVolumeSpecName "kube-api-access-qblkr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:31:05.277018 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.276981 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cddf2c6-5652-4784-a080-49ad81bfbf1d-istio-token" (OuterVolumeSpecName: "istio-token") pod "9cddf2c6-5652-4784-a080-49ad81bfbf1d" (UID: "9cddf2c6-5652-4784-a080-49ad81bfbf1d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:31:05.277082 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.277066 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9cddf2c6-5652-4784-a080-49ad81bfbf1d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9cddf2c6-5652-4784-a080-49ad81bfbf1d" (UID: "9cddf2c6-5652-4784-a080-49ad81bfbf1d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 17:31:05.277361 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.277345 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cddf2c6-5652-4784-a080-49ad81bfbf1d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9cddf2c6-5652-4784-a080-49ad81bfbf1d" (UID: "9cddf2c6-5652-4784-a080-49ad81bfbf1d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:31:05.376004 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.375969 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cddf2c6-5652-4784-a080-49ad81bfbf1d-istiod-ca-cert\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:31:05.376004 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.376000 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cddf2c6-5652-4784-a080-49ad81bfbf1d-istio-token\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:31:05.376004 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.376009 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cddf2c6-5652-4784-a080-49ad81bfbf1d-istio-envoy\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:31:05.376227 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.376017 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cddf2c6-5652-4784-a080-49ad81bfbf1d-istio-data\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:31:05.376227 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.376024 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cddf2c6-5652-4784-a080-49ad81bfbf1d-credential-socket\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:31:05.376227 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.376033 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cddf2c6-5652-4784-a080-49ad81bfbf1d-istio-podinfo\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:31:05.376227 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.376041 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qblkr\" (UniqueName: \"kubernetes.io/projected/9cddf2c6-5652-4784-a080-49ad81bfbf1d-kube-api-access-qblkr\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:31:05.570479 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.570447 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj"] Jul 15 17:31:05.575294 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:05.575269 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-65f681ff-openshift-default-6bc54c77c5-4v5nj"] Jul 15 17:31:06.990430 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:06.990392 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cddf2c6-5652-4784-a080-49ad81bfbf1d" path="/var/lib/kubelet/pods/9cddf2c6-5652-4784-a080-49ad81bfbf1d/volumes" Jul 15 17:31:08.398399 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:08.398363 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:08.530414 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:08.530380 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:08.623755 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:08.623724 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:08.726404 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:08.726370 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:09.094427 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:09.094335 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:11.262871 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:11.262833 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:29.285572 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:29.285536 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29wtx_88f8928c-237e-457f-a032-707f48111f36/ovn-acl-logging/0.log" Jul 15 17:31:29.291609 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:29.291581 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29wtx_88f8928c-237e-457f-a032-707f48111f36/ovn-acl-logging/0.log" Jul 15 17:31:35.106347 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:35.106313 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:42.770959 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:42.770920 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:42.886664 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:42.886632 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:42.990085 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:42.990049 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:59.019504 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:31:59.019464 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:06.145195 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:06.145162 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:06.257516 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:06.257469 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:06.364164 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:06.364120 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:17.008579 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:17.008493 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:17.848962 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:17.848927 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:24.196142 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:24.196104 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:24.254057 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:24.254022 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:24.351086 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:24.351052 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:24.449906 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:24.449812 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:24.569934 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:24.569898 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:25.420836 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:25.420798 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:25.511867 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:25.511828 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:25.613018 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:25.612984 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:27.347839 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:27.347805 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:33.936726 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:33.936676 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:34.051707 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:34.051657 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:34.693109 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:34.693076 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:41.015472 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:41.015434 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:41.109651 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:41.109614 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:41.284255 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:41.284160 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:41.689619 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:41.689568 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:41.826996 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:41.826959 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:44.550486 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:44.550448 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:44.632418 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:44.632380 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:44.720661 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:44.720621 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:45.060668 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:45.060636 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:45.357054 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:45.356965 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:45.455823 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:45.455794 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:49.191705 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:49.191676 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:49.777008 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:49.776971 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:49.875492 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:49.875461 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:49.969543 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:49.969511 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:50.209006 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:50.208970 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:56.417086 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:56.417051 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:56.522838 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:56.522805 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:56.780701 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:56.780627 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-92183fb9-2s55d"] Jul 15 17:32:56.781030 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:56.781016 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cddf2c6-5652-4784-a080-49ad81bfbf1d" containerName="istio-proxy" Jul 15 17:32:56.781030 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:56.781031 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cddf2c6-5652-4784-a080-49ad81bfbf1d" containerName="istio-proxy" Jul 15 17:32:56.781121 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:56.781109 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cddf2c6-5652-4784-a080-49ad81bfbf1d" containerName="istio-proxy" Jul 15 17:32:56.784177 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:56.784162 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-92183fb9-2s55d" Jul 15 17:32:56.786565 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:56.786541 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 17:32:56.786679 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:56.786582 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dlhgl\"" Jul 15 17:32:56.791489 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:56.791466 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-92183fb9-2s55d"] Jul 15 17:32:56.917504 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:56.917472 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc15b0fe-45c4-41db-aa89-9b48a34f6593-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-92183fb9-2s55d\" (UID: \"dc15b0fe-45c4-41db-aa89-9b48a34f6593\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-92183fb9-2s55d" Jul 15 17:32:57.018406 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:57.018373 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc15b0fe-45c4-41db-aa89-9b48a34f6593-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-92183fb9-2s55d\" (UID: \"dc15b0fe-45c4-41db-aa89-9b48a34f6593\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-92183fb9-2s55d" Jul 15 17:32:57.019110 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:57.019085 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc15b0fe-45c4-41db-aa89-9b48a34f6593-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-92183fb9-2s55d\" (UID: \"dc15b0fe-45c4-41db-aa89-9b48a34f6593\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-92183fb9-2s55d" Jul 15 17:32:57.096944 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:57.096865 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-92183fb9-2s55d" Jul 15 17:32:57.229262 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:57.229236 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-92183fb9-2s55d"] Jul 15 17:32:57.230801 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:32:57.230772 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc15b0fe_45c4_41db_aa89_9b48a34f6593.slice/crio-feda50de63b6ea76a22a28343b9728aedeeb24c42ee9d05855fd772eb1436f59 WatchSource:0}: Error finding container feda50de63b6ea76a22a28343b9728aedeeb24c42ee9d05855fd772eb1436f59: Status 404 returned error can't find the container with id feda50de63b6ea76a22a28343b9728aedeeb24c42ee9d05855fd772eb1436f59 Jul 15 17:32:57.852206 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:57.852118 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-92183fb9-2s55d" event={"ID":"dc15b0fe-45c4-41db-aa89-9b48a34f6593","Type":"ContainerStarted","Data":"5a94518a0e16ce6352b045a58f1d2df0ff784f1495b4a1363c4736e3d564b0d3"} Jul 15 17:32:57.852206 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:57.852173 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-92183fb9-2s55d" event={"ID":"dc15b0fe-45c4-41db-aa89-9b48a34f6593","Type":"ContainerStarted","Data":"feda50de63b6ea76a22a28343b9728aedeeb24c42ee9d05855fd772eb1436f59"} Jul 15 17:32:57.868518 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:57.868470 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-92183fb9-2s55d" podStartSLOduration=1.868458011 podStartE2EDuration="1.868458011s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:32:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:32:57.867369063 +0000 UTC m=+2195.420499276" watchObservedRunningTime="2026-07-15 17:32:57.868458011 +0000 UTC m=+2195.421588248" Jul 15 17:32:58.858960 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:58.858927 2578 generic.go:356] "Generic (PLEG): container finished" podID="dc15b0fe-45c4-41db-aa89-9b48a34f6593" containerID="5a94518a0e16ce6352b045a58f1d2df0ff784f1495b4a1363c4736e3d564b0d3" exitCode=0 Jul 15 17:32:58.859358 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:58.858979 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-92183fb9-2s55d" event={"ID":"dc15b0fe-45c4-41db-aa89-9b48a34f6593","Type":"ContainerDied","Data":"5a94518a0e16ce6352b045a58f1d2df0ff784f1495b4a1363c4736e3d564b0d3"} Jul 15 17:32:59.993551 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:32:59.993528 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-92183fb9-2s55d" Jul 15 17:33:00.146114 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:33:00.146032 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc15b0fe-45c4-41db-aa89-9b48a34f6593-maas-api-service-ca\") pod \"dc15b0fe-45c4-41db-aa89-9b48a34f6593\" (UID: \"dc15b0fe-45c4-41db-aa89-9b48a34f6593\") " Jul 15 17:33:00.146387 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:33:00.146363 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc15b0fe-45c4-41db-aa89-9b48a34f6593-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dc15b0fe-45c4-41db-aa89-9b48a34f6593" (UID: "dc15b0fe-45c4-41db-aa89-9b48a34f6593"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:33:00.247348 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:33:00.247310 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc15b0fe-45c4-41db-aa89-9b48a34f6593-maas-api-service-ca\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:33:00.871073 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:33:00.871046 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-92183fb9-2s55d" Jul 15 17:33:00.871264 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:33:00.871095 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-92183fb9-2s55d" event={"ID":"dc15b0fe-45c4-41db-aa89-9b48a34f6593","Type":"ContainerDied","Data":"feda50de63b6ea76a22a28343b9728aedeeb24c42ee9d05855fd772eb1436f59"} Jul 15 17:33:00.871264 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:33:00.871150 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="feda50de63b6ea76a22a28343b9728aedeeb24c42ee9d05855fd772eb1436f59" Jul 15 17:33:07.344964 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:33:07.344934 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:11.887061 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:33:11.887029 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:18.294966 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:33:18.294929 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:18.452889 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:33:18.452849 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:22.600145 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:33:22.600101 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:22.835835 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:33:22.835800 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:22.953105 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:33:22.953074 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:23.052424 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:33:23.052390 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:23.100709 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:33:23.100681 2578 scope.go:117] "RemoveContainer" containerID="46d84c9bc09e80b3e811feb6b8d0f248bb4f70e3450f3871f1bb02adae97bb6a" Jul 15 17:33:23.258881 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:33:23.258788 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:27.564470 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:33:27.564430 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:27.624544 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:33:27.624507 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:29.146997 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:33:29.146966 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:29.249000 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:33:29.248965 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:29.347645 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:33:29.347582 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:29.518738 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:33:29.518701 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:35.422000 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:33:35.421956 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:35.482161 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:33:35.482121 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:42.390165 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:33:42.390114 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:42.735661 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:33:42.735620 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:42.980017 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:33:42.979984 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:45.458930 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:33:45.458841 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:45.547569 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:33:45.547528 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:45.607487 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:33:45.607456 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:46.118391 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:33:46.118352 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:46.234061 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:33:46.234017 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:46.334239 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:33:46.334199 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:49.965479 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:33:49.965443 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:34:10.502093 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:34:10.502057 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:34:12.096297 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:34:12.096261 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:34:12.196153 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:34:12.196116 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:34:27.149560 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:34:27.149525 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:34:39.252088 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:34:39.252048 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:34:39.354073 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:34:39.354040 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:34:48.969806 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:34:48.969769 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:34:53.543156 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:34:53.543123 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:34:58.572518 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:34:58.572482 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:35.337181 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:35:35.337090 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:35.794148 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:35:35.794110 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:41.608097 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:35:41.608054 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:41.708463 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:35:41.708422 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:45.912794 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:35:45.912759 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:46.159039 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:35:46.159001 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:47.319965 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:35:47.319915 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:47.420544 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:35:47.420509 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:47.529764 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:35:47.529728 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:48.338532 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:35:48.338486 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:48.437886 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:35:48.437853 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:48.540712 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:35:48.540680 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:50.009614 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:35:50.009573 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:55.928395 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:35:55.928359 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:56.029751 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:35:56.029710 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:56.638666 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:35:56.638632 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:02.615313 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:02.615279 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:02.669142 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:02.669099 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:02.761323 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:02.761288 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:02.862639 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:02.862584 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:13.388155 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:13.388121 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:13.742654 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:13.742617 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:28.177918 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:28.177880 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:29.324072 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:29.324035 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29wtx_88f8928c-237e-457f-a032-707f48111f36/ovn-acl-logging/0.log" Jul 15 17:36:29.330972 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:29.330954 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29wtx_88f8928c-237e-457f-a032-707f48111f36/ovn-acl-logging/0.log" Jul 15 17:36:34.737724 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:34.737681 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:40.583164 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:40.583129 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:40.677505 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:40.677466 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:40.963920 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:40.963879 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:41.304871 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:41.304758 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:41.592240 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:41.592149 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:44.418772 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:44.418719 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:44.514261 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:44.514224 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:44.605260 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:44.605186 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42"] Jul 15 17:36:44.605575 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:44.605562 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cddf2c6-5652-4784-a080-49ad81bfbf1d" containerName="istio-proxy" Jul 15 17:36:44.605647 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:44.605577 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cddf2c6-5652-4784-a080-49ad81bfbf1d" containerName="istio-proxy" Jul 15 17:36:44.605647 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:44.605589 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc15b0fe-45c4-41db-aa89-9b48a34f6593" containerName="revoke-keys" Jul 15 17:36:44.605647 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:44.605610 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc15b0fe-45c4-41db-aa89-9b48a34f6593" containerName="revoke-keys" Jul 15 17:36:44.605755 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:44.605684 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cddf2c6-5652-4784-a080-49ad81bfbf1d" containerName="istio-proxy" Jul 15 17:36:44.605755 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:44.605694 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc15b0fe-45c4-41db-aa89-9b48a34f6593" containerName="revoke-keys" Jul 15 17:36:44.608782 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:44.608754 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42" Jul 15 17:36:44.611564 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:44.611541 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 17:36:44.611717 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:44.611576 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-b525cf-serving-cert\"" Jul 15 17:36:44.611717 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:44.611633 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 15 17:36:44.612452 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:44.612434 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-77cxl\"" Jul 15 17:36:44.619501 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:44.619477 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42"] Jul 15 17:36:44.639148 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:44.639119 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:44.664176 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:44.664151 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1612374-0cbc-491c-a83a-cf1728a986d4-openshift-service-ca\") pod \"maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42\" (UID: \"c1612374-0cbc-491c-a83a-cf1728a986d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42" Jul 15 17:36:44.664289 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:44.664185 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4q4b\" (UniqueName: \"kubernetes.io/projected/c1612374-0cbc-491c-a83a-cf1728a986d4-kube-api-access-w4q4b\") pod \"maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42\" (UID: \"c1612374-0cbc-491c-a83a-cf1728a986d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42" Jul 15 17:36:44.664289 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:44.664209 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1612374-0cbc-491c-a83a-cf1728a986d4-maas-api-tls\") pod \"maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42\" (UID: \"c1612374-0cbc-491c-a83a-cf1728a986d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42" Jul 15 17:36:44.764901 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:44.764815 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1612374-0cbc-491c-a83a-cf1728a986d4-openshift-service-ca\") pod \"maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42\" (UID: \"c1612374-0cbc-491c-a83a-cf1728a986d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42" Jul 15 17:36:44.764901 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:44.764866 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4q4b\" (UniqueName: \"kubernetes.io/projected/c1612374-0cbc-491c-a83a-cf1728a986d4-kube-api-access-w4q4b\") pod \"maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42\" (UID: \"c1612374-0cbc-491c-a83a-cf1728a986d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42" Jul 15 17:36:44.765116 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:44.764902 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1612374-0cbc-491c-a83a-cf1728a986d4-maas-api-tls\") pod \"maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42\" (UID: \"c1612374-0cbc-491c-a83a-cf1728a986d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42" Jul 15 17:36:44.765657 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:44.765589 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1612374-0cbc-491c-a83a-cf1728a986d4-openshift-service-ca\") pod \"maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42\" (UID: \"c1612374-0cbc-491c-a83a-cf1728a986d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42" Jul 15 17:36:44.767627 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:44.767583 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1612374-0cbc-491c-a83a-cf1728a986d4-maas-api-tls\") pod \"maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42\" (UID: \"c1612374-0cbc-491c-a83a-cf1728a986d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42" Jul 15 17:36:44.772580 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:44.772560 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4q4b\" (UniqueName: \"kubernetes.io/projected/c1612374-0cbc-491c-a83a-cf1728a986d4-kube-api-access-w4q4b\") pod \"maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42\" (UID: \"c1612374-0cbc-491c-a83a-cf1728a986d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42" Jul 15 17:36:44.802448 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:44.802418 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:44.920494 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:44.920462 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42" Jul 15 17:36:45.055188 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:45.055162 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42"] Jul 15 17:36:45.056738 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:36:45.056711 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1612374_0cbc_491c_a83a_cf1728a986d4.slice/crio-d5af3997a3da73531fa77939713507a41c4e8623fd04aef192819d90ddf1aec1 WatchSource:0}: Error finding container d5af3997a3da73531fa77939713507a41c4e8623fd04aef192819d90ddf1aec1: Status 404 returned error can't find the container with id d5af3997a3da73531fa77939713507a41c4e8623fd04aef192819d90ddf1aec1 Jul 15 17:36:45.058160 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:45.058138 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 17:36:45.099439 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:45.099408 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42" event={"ID":"c1612374-0cbc-491c-a83a-cf1728a986d4","Type":"ContainerStarted","Data":"d5af3997a3da73531fa77939713507a41c4e8623fd04aef192819d90ddf1aec1"} Jul 15 17:36:45.115895 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:45.115866 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:45.214956 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:45.214918 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:46.106758 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:46.106669 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42" event={"ID":"c1612374-0cbc-491c-a83a-cf1728a986d4","Type":"ContainerStarted","Data":"8618487c0a5a3dcd042f80e27a0d9a378e0a348bf606423587297a5eb4f8359a"} Jul 15 17:36:46.107101 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:46.106801 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42" Jul 15 17:36:46.127007 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:46.126952 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42" podStartSLOduration=1.45351248 podStartE2EDuration="2.126936305s" totalImagesPullingTime="673.423825ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:36:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:36:46.124467955 +0000 UTC m=+2423.677598193" watchObservedRunningTime="2026-07-15 17:36:46.126936305 +0000 UTC m=+2423.680066519" Jul 15 17:36:48.981224 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:48.981186 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:49.352137 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:49.352028 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:49.714209 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:49.714180 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6"] Jul 15 17:36:49.717887 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:49.717867 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:36:49.720884 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:49.720863 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-ed384a-openshift-default-dockercfg-74lmh\"" Jul 15 17:36:49.721038 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:49.721019 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 15 17:36:49.743650 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:49.743626 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6"] Jul 15 17:36:49.757474 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:49.757447 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:49.804114 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:49.804089 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-istio-podinfo\") pod \"e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:36:49.804253 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:49.804135 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-istio-envoy\") pod \"e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:36:49.804253 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:49.804159 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-workload-certs\") pod \"e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:36:49.804253 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:49.804195 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwds2\" (UniqueName: \"kubernetes.io/projected/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-kube-api-access-hwds2\") pod \"e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:36:49.804253 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:49.804224 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-istio-data\") pod \"e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:36:49.804253 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:49.804246 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-istiod-ca-cert\") pod \"e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:36:49.804462 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:49.804289 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-istio-token\") pod \"e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:36:49.804462 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:49.804342 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-workload-socket\") pod \"e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:36:49.804462 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:49.804367 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-credential-socket\") pod \"e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:36:49.864372 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:49.864336 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:49.905094 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:49.905059 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-istio-data\") pod \"e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:36:49.905094 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:49.905094 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-istiod-ca-cert\") pod \"e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:36:49.905331 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:49.905118 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-istio-token\") pod \"e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:36:49.905331 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:49.905148 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-workload-socket\") pod \"e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:36:49.905331 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:49.905165 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-credential-socket\") pod \"e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:36:49.905331 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:49.905197 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-istio-podinfo\") pod \"e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:36:49.905331 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:49.905233 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-istio-envoy\") pod \"e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:36:49.905331 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:49.905255 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-workload-certs\") pod \"e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:36:49.905687 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:49.905452 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwds2\" (UniqueName: \"kubernetes.io/projected/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-kube-api-access-hwds2\") pod \"e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:36:49.905687 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:49.905538 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-istio-data\") pod \"e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:36:49.905687 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:49.905649 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-workload-socket\") pod \"e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:36:49.905839 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:49.905700 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-credential-socket\") pod \"e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:36:49.905906 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:49.905850 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-workload-certs\") pod \"e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:36:49.906074 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:49.906057 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-istiod-ca-cert\") pod \"e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:36:49.907671 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:49.907650 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-istio-envoy\") pod \"e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:36:49.907772 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:49.907754 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-istio-podinfo\") pod \"e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:36:49.914406 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:49.914384 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-istio-token\") pod \"e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:36:49.914845 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:49.914825 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwds2\" (UniqueName: \"kubernetes.io/projected/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-kube-api-access-hwds2\") pod \"e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:36:50.028694 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:50.028613 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:36:50.177746 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:50.177722 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6"] Jul 15 17:36:50.179491 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:36:50.179466 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea8b3fb8_f372_45ae_b7dd_4e108da4c7f6.slice/crio-df3b6eda40bb4d2c73297f2a42e5b44a9f9d8d8654e58ad12e17f9a8059a05b4 WatchSource:0}: Error finding container df3b6eda40bb4d2c73297f2a42e5b44a9f9d8d8654e58ad12e17f9a8059a05b4: Status 404 returned error can't find the container with id df3b6eda40bb4d2c73297f2a42e5b44a9f9d8d8654e58ad12e17f9a8059a05b4 Jul 15 17:36:50.182276 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:50.182245 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 17:36:50.182371 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:50.182307 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 17:36:50.182371 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:50.182352 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 17:36:51.145502 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:51.145466 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" event={"ID":"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6","Type":"ContainerStarted","Data":"ddf0d65e189c4198a3ccd8d81cb04085c88ca97a98f207da1d18e53c857506b2"} Jul 15 17:36:51.145916 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:51.145547 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" event={"ID":"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6","Type":"ContainerStarted","Data":"df3b6eda40bb4d2c73297f2a42e5b44a9f9d8d8654e58ad12e17f9a8059a05b4"} Jul 15 17:36:51.171707 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:51.171655 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" podStartSLOduration=2.17163574 podStartE2EDuration="2.17163574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:36:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:36:51.168749895 +0000 UTC m=+2428.721880110" watchObservedRunningTime="2026-07-15 17:36:51.17163574 +0000 UTC m=+2428.724765955" Jul 15 17:36:52.029552 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:52.029518 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:36:52.034913 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:52.034891 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:36:52.117355 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:52.117327 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42" Jul 15 17:36:52.152850 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:52.152819 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:36:52.154059 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:52.154031 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:36:53.101364 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:53.101330 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:53.206905 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:53.206866 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:53.295566 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:53.295530 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:53.498347 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:53.498311 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:54.238468 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:54.238424 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:54.522991 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:54.522899 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:57.611975 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:36:57.611935 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:37:06.285635 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:37:06.285576 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:37:06.385131 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:37:06.385086 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:37:07.154203 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:37:07.154166 2578 patch_prober.go:28] interesting pod/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 15 17:37:07.154450 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:37:07.154265 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" podUID="ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 15 17:37:07.236385 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:37:07.236353 2578 generic.go:356] "Generic (PLEG): container finished" podID="ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6" containerID="ddf0d65e189c4198a3ccd8d81cb04085c88ca97a98f207da1d18e53c857506b2" exitCode=0 Jul 15 17:37:07.236564 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:37:07.236437 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" event={"ID":"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6","Type":"ContainerDied","Data":"ddf0d65e189c4198a3ccd8d81cb04085c88ca97a98f207da1d18e53c857506b2"} Jul 15 17:37:07.236910 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:37:07.236893 2578 scope.go:117] "RemoveContainer" containerID="ddf0d65e189c4198a3ccd8d81cb04085c88ca97a98f207da1d18e53c857506b2" Jul 15 17:37:07.237918 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:37:07.237892 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 17:37:07.238018 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:37:07.237975 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 17:37:07.238079 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:37:07.238023 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 17:37:08.244960 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:37:08.244928 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" event={"ID":"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6","Type":"ContainerStarted","Data":"1a11e785c034f23681c9f20194f00beb4acdfb36bb3aaa9136efe9db375f0b36"} Jul 15 17:37:08.975966 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:37:08.975928 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:37:09.028944 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:37:09.028914 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:37:09.029208 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:37:09.029188 2578 patch_prober.go:28] interesting pod/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 15 17:37:09.029279 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:37:09.029261 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" podUID="ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 15 17:37:10.029811 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:37:10.029767 2578 patch_prober.go:28] interesting pod/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 15 17:37:10.030297 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:37:10.029843 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" podUID="ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 15 17:37:11.030103 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:37:11.030059 2578 patch_prober.go:28] interesting pod/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 15 17:37:11.030570 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:37:11.030124 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" podUID="ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 15 17:37:12.029912 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:37:12.029871 2578 patch_prober.go:28] interesting pod/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 15 17:37:12.030162 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:37:12.029927 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" podUID="ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 15 17:37:13.033148 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:37:13.033117 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:37:13.033549 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:37:13.033498 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:37:13.034064 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:37:13.034046 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:37:16.550883 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:37:16.550848 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:37:16.922501 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:37:16.922475 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:37:32.915139 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:37:32.915099 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:37:51.826123 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:37:51.826084 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:37:52.901000 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:37:52.900963 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:37:54.058663 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:37:54.058623 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:37:54.161382 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:37:54.161336 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:03.901573 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:03.901536 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42"] Jul 15 17:38:03.902071 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:03.901835 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42" podUID="c1612374-0cbc-491c-a83a-cf1728a986d4" containerName="maas-api" containerID="cri-o://8618487c0a5a3dcd042f80e27a0d9a378e0a348bf606423587297a5eb4f8359a" gracePeriod=30 Jul 15 17:38:03.958282 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:03.958249 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:04.155247 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:04.155185 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42" Jul 15 17:38:04.277483 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:04.277448 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w4q4b\" (UniqueName: \"kubernetes.io/projected/c1612374-0cbc-491c-a83a-cf1728a986d4-kube-api-access-w4q4b\") pod \"c1612374-0cbc-491c-a83a-cf1728a986d4\" (UID: \"c1612374-0cbc-491c-a83a-cf1728a986d4\") " Jul 15 17:38:04.277702 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:04.277499 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1612374-0cbc-491c-a83a-cf1728a986d4-openshift-service-ca\") pod \"c1612374-0cbc-491c-a83a-cf1728a986d4\" (UID: \"c1612374-0cbc-491c-a83a-cf1728a986d4\") " Jul 15 17:38:04.277702 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:04.277561 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1612374-0cbc-491c-a83a-cf1728a986d4-maas-api-tls\") pod \"c1612374-0cbc-491c-a83a-cf1728a986d4\" (UID: \"c1612374-0cbc-491c-a83a-cf1728a986d4\") " Jul 15 17:38:04.277982 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:04.277956 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1612374-0cbc-491c-a83a-cf1728a986d4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c1612374-0cbc-491c-a83a-cf1728a986d4" (UID: "c1612374-0cbc-491c-a83a-cf1728a986d4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:38:04.279694 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:04.279672 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1612374-0cbc-491c-a83a-cf1728a986d4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c1612374-0cbc-491c-a83a-cf1728a986d4" (UID: "c1612374-0cbc-491c-a83a-cf1728a986d4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:38:04.279694 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:04.279677 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1612374-0cbc-491c-a83a-cf1728a986d4-kube-api-access-w4q4b" (OuterVolumeSpecName: "kube-api-access-w4q4b") pod "c1612374-0cbc-491c-a83a-cf1728a986d4" (UID: "c1612374-0cbc-491c-a83a-cf1728a986d4"). InnerVolumeSpecName "kube-api-access-w4q4b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:38:04.378757 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:04.378727 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1612374-0cbc-491c-a83a-cf1728a986d4-maas-api-tls\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:38:04.378757 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:04.378753 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4q4b\" (UniqueName: \"kubernetes.io/projected/c1612374-0cbc-491c-a83a-cf1728a986d4-kube-api-access-w4q4b\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:38:04.378757 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:04.378764 2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1612374-0cbc-491c-a83a-cf1728a986d4-openshift-service-ca\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:38:04.572062 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:04.572024 2578 generic.go:356] "Generic (PLEG): container finished" podID="c1612374-0cbc-491c-a83a-cf1728a986d4" containerID="8618487c0a5a3dcd042f80e27a0d9a378e0a348bf606423587297a5eb4f8359a" exitCode=0 Jul 15 17:38:04.572221 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:04.572069 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42" Jul 15 17:38:04.572221 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:04.572112 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42" event={"ID":"c1612374-0cbc-491c-a83a-cf1728a986d4","Type":"ContainerDied","Data":"8618487c0a5a3dcd042f80e27a0d9a378e0a348bf606423587297a5eb4f8359a"} Jul 15 17:38:04.572221 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:04.572176 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42" event={"ID":"c1612374-0cbc-491c-a83a-cf1728a986d4","Type":"ContainerDied","Data":"d5af3997a3da73531fa77939713507a41c4e8623fd04aef192819d90ddf1aec1"} Jul 15 17:38:04.572221 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:04.572208 2578 scope.go:117] "RemoveContainer" containerID="8618487c0a5a3dcd042f80e27a0d9a378e0a348bf606423587297a5eb4f8359a" Jul 15 17:38:04.582350 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:04.582333 2578 scope.go:117] "RemoveContainer" containerID="8618487c0a5a3dcd042f80e27a0d9a378e0a348bf606423587297a5eb4f8359a" Jul 15 17:38:04.582618 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:38:04.582578 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8618487c0a5a3dcd042f80e27a0d9a378e0a348bf606423587297a5eb4f8359a\": container with ID starting with 8618487c0a5a3dcd042f80e27a0d9a378e0a348bf606423587297a5eb4f8359a not found: ID does not exist" containerID="8618487c0a5a3dcd042f80e27a0d9a378e0a348bf606423587297a5eb4f8359a" Jul 15 17:38:04.582694 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:04.582622 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8618487c0a5a3dcd042f80e27a0d9a378e0a348bf606423587297a5eb4f8359a"} err="failed to get container status \"8618487c0a5a3dcd042f80e27a0d9a378e0a348bf606423587297a5eb4f8359a\": rpc error: code = NotFound desc = could not find container \"8618487c0a5a3dcd042f80e27a0d9a378e0a348bf606423587297a5eb4f8359a\": container with ID starting with 8618487c0a5a3dcd042f80e27a0d9a378e0a348bf606423587297a5eb4f8359a not found: ID does not exist" Jul 15 17:38:04.598029 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:04.597997 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42"] Jul 15 17:38:04.605495 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:04.605476 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b525cf-6cb4fc684c-8nv42"] Jul 15 17:38:04.990303 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:04.990272 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1612374-0cbc-491c-a83a-cf1728a986d4" path="/var/lib/kubelet/pods/c1612374-0cbc-491c-a83a-cf1728a986d4/volumes" Jul 15 17:38:08.355833 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:08.355796 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:15.272299 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:15.272205 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:15.370049 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:15.370014 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:19.294845 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:19.294794 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:19.554937 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:19.554844 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:20.902081 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:20.902042 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:21.060509 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:21.060474 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:30.552858 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:30.552818 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:34.862377 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:34.862336 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:42.393408 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:42.393371 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:42.497036 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:42.497001 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:45.663659 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:45.663625 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6"] Jul 15 17:38:45.664057 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:45.663842 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" podUID="ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6" containerName="istio-proxy" containerID="cri-o://1a11e785c034f23681c9f20194f00beb4acdfb36bb3aaa9136efe9db375f0b36" gracePeriod=30 Jul 15 17:38:45.676960 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:45.676928 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:45.930795 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:45.930702 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:46.317952 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:46.317838 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:46.823285 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:46.823253 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:46.967618 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:46.967562 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:50.448864 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:50.448831 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:50.579067 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:50.579036 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:50.832587 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:50.832561 2578 generic.go:356] "Generic (PLEG): container finished" podID="ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6" containerID="1a11e785c034f23681c9f20194f00beb4acdfb36bb3aaa9136efe9db375f0b36" exitCode=0 Jul 15 17:38:50.832786 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:50.832641 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" event={"ID":"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6","Type":"ContainerDied","Data":"1a11e785c034f23681c9f20194f00beb4acdfb36bb3aaa9136efe9db375f0b36"} Jul 15 17:38:50.832786 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:50.832728 2578 scope.go:117] "RemoveContainer" containerID="ddf0d65e189c4198a3ccd8d81cb04085c88ca97a98f207da1d18e53c857506b2" Jul 15 17:38:50.913214 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:50.913191 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:38:50.983995 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:50.983965 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-istio-envoy\") pod \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " Jul 15 17:38:50.983995 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:50.983996 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-workload-certs\") pod \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " Jul 15 17:38:50.984221 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:50.984016 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-workload-socket\") pod \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " Jul 15 17:38:50.984221 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:50.984032 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-credential-socket\") pod \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " Jul 15 17:38:50.984221 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:50.984064 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hwds2\" (UniqueName: \"kubernetes.io/projected/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-kube-api-access-hwds2\") pod \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " Jul 15 17:38:50.984221 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:50.984098 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-istiod-ca-cert\") pod \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " Jul 15 17:38:50.984221 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:50.984128 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-istio-data\") pod \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " Jul 15 17:38:50.984500 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:50.984266 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-istio-token\") pod \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " Jul 15 17:38:50.984500 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:50.984311 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-istio-podinfo\") pod \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\" (UID: \"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6\") " Jul 15 17:38:50.984500 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:50.984311 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6" (UID: "ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:38:50.984500 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:50.984329 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6" (UID: "ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:38:50.984500 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:50.984341 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6" (UID: "ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:38:50.984787 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:50.984560 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-workload-certs\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:38:50.984787 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:50.984581 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-workload-socket\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:38:50.984787 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:50.984622 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-credential-socket\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:38:50.985020 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:50.984969 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6" (UID: "ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:38:50.985144 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:50.985087 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-istio-data" (OuterVolumeSpecName: "istio-data") pod "ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6" (UID: "ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:38:50.986433 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:50.986407 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-kube-api-access-hwds2" (OuterVolumeSpecName: "kube-api-access-hwds2") pod "ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6" (UID: "ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6"). InnerVolumeSpecName "kube-api-access-hwds2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:38:50.986693 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:50.986662 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6" (UID: "ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:38:50.986800 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:50.986712 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-istio-token" (OuterVolumeSpecName: "istio-token") pod "ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6" (UID: "ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:38:50.986977 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:50.986954 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6" (UID: "ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 17:38:51.085503 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:51.085418 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-istio-token\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:38:51.085503 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:51.085443 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-istio-podinfo\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:38:51.085503 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:51.085451 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-istio-envoy\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:38:51.085503 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:51.085462 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwds2\" (UniqueName: \"kubernetes.io/projected/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-kube-api-access-hwds2\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:38:51.085503 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:51.085471 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-istiod-ca-cert\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:38:51.085503 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:51.085479 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6-istio-data\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:38:51.231677 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:51.231640 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:51.428043 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:51.428011 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:51.839902 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:51.839824 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" Jul 15 17:38:51.840356 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:51.839819 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6" event={"ID":"ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6","Type":"ContainerDied","Data":"df3b6eda40bb4d2c73297f2a42e5b44a9f9d8d8654e58ad12e17f9a8059a05b4"} Jul 15 17:38:51.840356 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:51.839957 2578 scope.go:117] "RemoveContainer" containerID="1a11e785c034f23681c9f20194f00beb4acdfb36bb3aaa9136efe9db375f0b36" Jul 15 17:38:51.879991 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:51.879967 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6"] Jul 15 17:38:51.885463 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:51.885443 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-ed384a-openshift-default-5784c57577-mmxh6"] Jul 15 17:38:52.911466 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:52.911434 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:52.990851 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:52.990825 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6" path="/var/lib/kubelet/pods/ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6/volumes" Jul 15 17:38:54.865786 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:54.865752 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:55.751551 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:55.751513 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:56.563110 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:56.563072 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:56.680129 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:38:56.680079 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:00.491932 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:00.491887 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:00.514752 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:00.514726 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z"] Jul 15 17:39:00.515128 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:00.515115 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6" containerName="istio-proxy" Jul 15 17:39:00.515188 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:00.515131 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6" containerName="istio-proxy" Jul 15 17:39:00.515188 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:00.515153 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1612374-0cbc-491c-a83a-cf1728a986d4" containerName="maas-api" Jul 15 17:39:00.515188 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:00.515159 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1612374-0cbc-491c-a83a-cf1728a986d4" containerName="maas-api" Jul 15 17:39:00.515284 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:00.515232 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6" containerName="istio-proxy" Jul 15 17:39:00.515284 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:00.515241 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1612374-0cbc-491c-a83a-cf1728a986d4" containerName="maas-api" Jul 15 17:39:00.518313 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:00.518295 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z" Jul 15 17:39:00.523576 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:00.523556 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-1da9d9-serving-cert\"" Jul 15 17:39:00.524220 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:00.524172 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-77cxl\"" Jul 15 17:39:00.524946 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:00.524922 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 15 17:39:00.531078 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:00.531059 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 17:39:00.554232 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:00.554202 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z"] Jul 15 17:39:00.572080 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:00.572056 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb2fff9e-3c34-442f-85cf-3f90ce560a17-openshift-service-ca\") pod \"maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z\" (UID: \"cb2fff9e-3c34-442f-85cf-3f90ce560a17\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z" Jul 15 17:39:00.572216 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:00.572122 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb2fff9e-3c34-442f-85cf-3f90ce560a17-maas-api-tls\") pod \"maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z\" (UID: \"cb2fff9e-3c34-442f-85cf-3f90ce560a17\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z" Jul 15 17:39:00.572216 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:00.572142 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fx8zg\" (UniqueName: \"kubernetes.io/projected/cb2fff9e-3c34-442f-85cf-3f90ce560a17-kube-api-access-fx8zg\") pod \"maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z\" (UID: \"cb2fff9e-3c34-442f-85cf-3f90ce560a17\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z" Jul 15 17:39:00.673663 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:00.673625 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb2fff9e-3c34-442f-85cf-3f90ce560a17-maas-api-tls\") pod \"maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z\" (UID: \"cb2fff9e-3c34-442f-85cf-3f90ce560a17\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z" Jul 15 17:39:00.673848 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:00.673679 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx8zg\" (UniqueName: \"kubernetes.io/projected/cb2fff9e-3c34-442f-85cf-3f90ce560a17-kube-api-access-fx8zg\") pod \"maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z\" (UID: \"cb2fff9e-3c34-442f-85cf-3f90ce560a17\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z" Jul 15 17:39:00.673848 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:39:00.673700 2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-1da9d9-serving-cert: secret "maas-api-e2e-api-b-1da9d9-serving-cert" not found Jul 15 17:39:00.673848 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:00.673742 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb2fff9e-3c34-442f-85cf-3f90ce560a17-openshift-service-ca\") pod \"maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z\" (UID: \"cb2fff9e-3c34-442f-85cf-3f90ce560a17\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z" Jul 15 17:39:00.673848 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:39:00.673768 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb2fff9e-3c34-442f-85cf-3f90ce560a17-maas-api-tls podName:cb2fff9e-3c34-442f-85cf-3f90ce560a17 nodeName:}" failed. No retries permitted until 2026-07-15 17:39:01.173749205 +0000 UTC m=+2558.726879396 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cb2fff9e-3c34-442f-85cf-3f90ce560a17-maas-api-tls") pod "maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z" (UID: "cb2fff9e-3c34-442f-85cf-3f90ce560a17") : secret "maas-api-e2e-api-b-1da9d9-serving-cert" not found Jul 15 17:39:00.674311 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:00.674294 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb2fff9e-3c34-442f-85cf-3f90ce560a17-openshift-service-ca\") pod \"maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z\" (UID: \"cb2fff9e-3c34-442f-85cf-3f90ce560a17\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z" Jul 15 17:39:00.686031 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:00.686013 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fx8zg\" (UniqueName: \"kubernetes.io/projected/cb2fff9e-3c34-442f-85cf-3f90ce560a17-kube-api-access-fx8zg\") pod \"maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z\" (UID: \"cb2fff9e-3c34-442f-85cf-3f90ce560a17\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z" Jul 15 17:39:00.831633 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:00.831533 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:01.058287 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:01.058242 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:01.179136 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:01.179103 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb2fff9e-3c34-442f-85cf-3f90ce560a17-maas-api-tls\") pod \"maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z\" (UID: \"cb2fff9e-3c34-442f-85cf-3f90ce560a17\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z" Jul 15 17:39:01.181565 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:01.181541 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb2fff9e-3c34-442f-85cf-3f90ce560a17-maas-api-tls\") pod \"maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z\" (UID: \"cb2fff9e-3c34-442f-85cf-3f90ce560a17\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z" Jul 15 17:39:01.429288 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:01.429194 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z" Jul 15 17:39:01.443886 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:01.443849 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:01.566278 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:01.566246 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:01.597908 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:01.597722 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z"] Jul 15 17:39:01.600381 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:39:01.600352 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb2fff9e_3c34_442f_85cf_3f90ce560a17.slice/crio-b495dcaf5e3a711674c42fc76ba1a00877325f9b1cd12eefa7447ba4defb264b WatchSource:0}: Error finding container b495dcaf5e3a711674c42fc76ba1a00877325f9b1cd12eefa7447ba4defb264b: Status 404 returned error can't find the container with id b495dcaf5e3a711674c42fc76ba1a00877325f9b1cd12eefa7447ba4defb264b Jul 15 17:39:01.897100 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:01.897005 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z" event={"ID":"cb2fff9e-3c34-442f-85cf-3f90ce560a17","Type":"ContainerStarted","Data":"b495dcaf5e3a711674c42fc76ba1a00877325f9b1cd12eefa7447ba4defb264b"} Jul 15 17:39:02.904102 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:02.904058 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z" event={"ID":"cb2fff9e-3c34-442f-85cf-3f90ce560a17","Type":"ContainerStarted","Data":"5231a1f4ff2f89be77ea43e25f2faded2d5a5497b2cef8516ea6e5c5ca11d11e"} Jul 15 17:39:02.904487 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:02.904136 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z" Jul 15 17:39:02.927958 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:02.927912 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z" podStartSLOduration=2.205444895 podStartE2EDuration="2.927897578s" totalImagesPullingTime="722.452683ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:39:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:39:02.92575067 +0000 UTC m=+2560.478880884" watchObservedRunningTime="2026-07-15 17:39:02.927897578 +0000 UTC m=+2560.481027791" Jul 15 17:39:04.792485 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:04.792452 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:08.915536 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:08.915509 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z" Jul 15 17:39:13.221831 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:13.221794 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:13.326969 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:13.326936 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:14.237206 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:14.237174 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:14.350259 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:14.350219 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:24.192541 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:24.192506 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:24.292830 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:24.292787 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:24.466611 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:24.466510 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:28.764184 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:28.764144 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:33.302973 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:33.302935 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:35.047110 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:35.047075 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:35.108667 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:35.108631 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:39.469080 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:39.469049 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:39.687947 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:39.687918 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:50.312171 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:50.312087 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z"] Jul 15 17:39:50.312535 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:50.312334 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z" podUID="cb2fff9e-3c34-442f-85cf-3f90ce560a17" containerName="maas-api" containerID="cri-o://5231a1f4ff2f89be77ea43e25f2faded2d5a5497b2cef8516ea6e5c5ca11d11e" gracePeriod=30 Jul 15 17:39:50.354700 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:50.354670 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:50.561128 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:50.561103 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z" Jul 15 17:39:50.689822 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:50.689792 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fx8zg\" (UniqueName: \"kubernetes.io/projected/cb2fff9e-3c34-442f-85cf-3f90ce560a17-kube-api-access-fx8zg\") pod \"cb2fff9e-3c34-442f-85cf-3f90ce560a17\" (UID: \"cb2fff9e-3c34-442f-85cf-3f90ce560a17\") " Jul 15 17:39:50.690010 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:50.689838 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb2fff9e-3c34-442f-85cf-3f90ce560a17-openshift-service-ca\") pod \"cb2fff9e-3c34-442f-85cf-3f90ce560a17\" (UID: \"cb2fff9e-3c34-442f-85cf-3f90ce560a17\") " Jul 15 17:39:50.690010 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:50.689916 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb2fff9e-3c34-442f-85cf-3f90ce560a17-maas-api-tls\") pod \"cb2fff9e-3c34-442f-85cf-3f90ce560a17\" (UID: \"cb2fff9e-3c34-442f-85cf-3f90ce560a17\") " Jul 15 17:39:50.690257 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:50.690232 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb2fff9e-3c34-442f-85cf-3f90ce560a17-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cb2fff9e-3c34-442f-85cf-3f90ce560a17" (UID: "cb2fff9e-3c34-442f-85cf-3f90ce560a17"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:39:50.691968 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:50.691943 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb2fff9e-3c34-442f-85cf-3f90ce560a17-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cb2fff9e-3c34-442f-85cf-3f90ce560a17" (UID: "cb2fff9e-3c34-442f-85cf-3f90ce560a17"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:39:50.692085 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:50.691987 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb2fff9e-3c34-442f-85cf-3f90ce560a17-kube-api-access-fx8zg" (OuterVolumeSpecName: "kube-api-access-fx8zg") pod "cb2fff9e-3c34-442f-85cf-3f90ce560a17" (UID: "cb2fff9e-3c34-442f-85cf-3f90ce560a17"). InnerVolumeSpecName "kube-api-access-fx8zg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:39:50.790488 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:50.790454 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb2fff9e-3c34-442f-85cf-3f90ce560a17-maas-api-tls\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:39:50.790488 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:50.790483 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fx8zg\" (UniqueName: \"kubernetes.io/projected/cb2fff9e-3c34-442f-85cf-3f90ce560a17-kube-api-access-fx8zg\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:39:50.790488 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:50.790493 2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb2fff9e-3c34-442f-85cf-3f90ce560a17-openshift-service-ca\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:39:51.169801 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:51.169761 2578 generic.go:356] "Generic (PLEG): container finished" podID="cb2fff9e-3c34-442f-85cf-3f90ce560a17" containerID="5231a1f4ff2f89be77ea43e25f2faded2d5a5497b2cef8516ea6e5c5ca11d11e" exitCode=0 Jul 15 17:39:51.169971 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:51.169796 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z" event={"ID":"cb2fff9e-3c34-442f-85cf-3f90ce560a17","Type":"ContainerDied","Data":"5231a1f4ff2f89be77ea43e25f2faded2d5a5497b2cef8516ea6e5c5ca11d11e"} Jul 15 17:39:51.169971 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:51.169850 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z" event={"ID":"cb2fff9e-3c34-442f-85cf-3f90ce560a17","Type":"ContainerDied","Data":"b495dcaf5e3a711674c42fc76ba1a00877325f9b1cd12eefa7447ba4defb264b"} Jul 15 17:39:51.169971 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:51.169809 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z" Jul 15 17:39:51.169971 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:51.169872 2578 scope.go:117] "RemoveContainer" containerID="5231a1f4ff2f89be77ea43e25f2faded2d5a5497b2cef8516ea6e5c5ca11d11e" Jul 15 17:39:51.178659 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:51.178638 2578 scope.go:117] "RemoveContainer" containerID="5231a1f4ff2f89be77ea43e25f2faded2d5a5497b2cef8516ea6e5c5ca11d11e" Jul 15 17:39:51.178924 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:39:51.178905 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5231a1f4ff2f89be77ea43e25f2faded2d5a5497b2cef8516ea6e5c5ca11d11e\": container with ID starting with 5231a1f4ff2f89be77ea43e25f2faded2d5a5497b2cef8516ea6e5c5ca11d11e not found: ID does not exist" containerID="5231a1f4ff2f89be77ea43e25f2faded2d5a5497b2cef8516ea6e5c5ca11d11e" Jul 15 17:39:51.178994 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:51.178936 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5231a1f4ff2f89be77ea43e25f2faded2d5a5497b2cef8516ea6e5c5ca11d11e"} err="failed to get container status \"5231a1f4ff2f89be77ea43e25f2faded2d5a5497b2cef8516ea6e5c5ca11d11e\": rpc error: code = NotFound desc = could not find container \"5231a1f4ff2f89be77ea43e25f2faded2d5a5497b2cef8516ea6e5c5ca11d11e\": container with ID starting with 5231a1f4ff2f89be77ea43e25f2faded2d5a5497b2cef8516ea6e5c5ca11d11e not found: ID does not exist" Jul 15 17:39:51.187917 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:51.187896 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z"] Jul 15 17:39:51.194084 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:51.194064 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1da9d9-78dbc84464-qrx7z"] Jul 15 17:39:52.994322 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:52.994290 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb2fff9e-3c34-442f-85cf-3f90ce560a17" path="/var/lib/kubelet/pods/cb2fff9e-3c34-442f-85cf-3f90ce560a17/volumes" Jul 15 17:39:54.818997 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:54.818960 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:54.919989 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:54.919947 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:55.020192 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:39:55.020155 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:40:01.003122 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:01.003085 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:40:01.103404 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:01.103365 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:40:06.243236 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.243080 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:40:06.360655 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.360586 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:40:06.451033 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.450996 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24"] Jul 15 17:40:06.451448 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.451431 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb2fff9e-3c34-442f-85cf-3f90ce560a17" containerName="maas-api" Jul 15 17:40:06.451448 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.451448 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb2fff9e-3c34-442f-85cf-3f90ce560a17" containerName="maas-api" Jul 15 17:40:06.451623 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.451467 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6" containerName="istio-proxy" Jul 15 17:40:06.451623 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.451472 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6" containerName="istio-proxy" Jul 15 17:40:06.451623 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.451535 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb2fff9e-3c34-442f-85cf-3f90ce560a17" containerName="maas-api" Jul 15 17:40:06.451623 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.451544 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea8b3fb8-f372-45ae-b7dd-4e108da4c7f6" containerName="istio-proxy" Jul 15 17:40:06.454657 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.454639 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:06.456935 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.456913 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 15 17:40:06.456935 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.456912 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-3d6c37-openshift-default-dockercfg-vv9m8\"" Jul 15 17:40:06.466209 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.466189 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24"] Jul 15 17:40:06.468366 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.468340 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:40:06.535501 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.535427 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27d54469-f670-4b34-b794-6bc485d670cf-istio-token\") pod \"e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:06.535501 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.535460 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27d54469-f670-4b34-b794-6bc485d670cf-istiod-ca-cert\") pod \"e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:06.535501 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.535485 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27d54469-f670-4b34-b794-6bc485d670cf-istio-podinfo\") pod \"e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:06.535735 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.535510 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27d54469-f670-4b34-b794-6bc485d670cf-istio-envoy\") pod \"e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:06.535735 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.535551 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27d54469-f670-4b34-b794-6bc485d670cf-istio-data\") pod \"e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:06.535735 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.535621 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27d54469-f670-4b34-b794-6bc485d670cf-workload-certs\") pod \"e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:06.535735 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.535653 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27d54469-f670-4b34-b794-6bc485d670cf-workload-socket\") pod \"e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:06.535735 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.535687 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27d54469-f670-4b34-b794-6bc485d670cf-credential-socket\") pod \"e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:06.535735 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.535715 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc5zg\" (UniqueName: \"kubernetes.io/projected/27d54469-f670-4b34-b794-6bc485d670cf-kube-api-access-wc5zg\") pod \"e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:06.636331 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.636299 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27d54469-f670-4b34-b794-6bc485d670cf-istio-token\") pod \"e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:06.636517 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.636345 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27d54469-f670-4b34-b794-6bc485d670cf-istiod-ca-cert\") pod \"e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:06.636645 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.636622 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27d54469-f670-4b34-b794-6bc485d670cf-istio-podinfo\") pod \"e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:06.636740 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.636664 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27d54469-f670-4b34-b794-6bc485d670cf-istio-envoy\") pod \"e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:06.636817 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.636792 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27d54469-f670-4b34-b794-6bc485d670cf-istio-data\") pod \"e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:06.636896 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.636876 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27d54469-f670-4b34-b794-6bc485d670cf-workload-certs\") pod \"e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:06.636960 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.636929 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27d54469-f670-4b34-b794-6bc485d670cf-workload-socket\") pod \"e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:06.637021 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.636992 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27d54469-f670-4b34-b794-6bc485d670cf-credential-socket\") pod \"e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:06.637097 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.637023 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc5zg\" (UniqueName: \"kubernetes.io/projected/27d54469-f670-4b34-b794-6bc485d670cf-kube-api-access-wc5zg\") pod \"e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:06.637229 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.637203 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27d54469-f670-4b34-b794-6bc485d670cf-istio-data\") pod \"e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:06.637303 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.637225 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27d54469-f670-4b34-b794-6bc485d670cf-istiod-ca-cert\") pod \"e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:06.637520 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.637411 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27d54469-f670-4b34-b794-6bc485d670cf-workload-socket\") pod \"e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:06.637520 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.637478 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27d54469-f670-4b34-b794-6bc485d670cf-credential-socket\") pod \"e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:06.637698 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.637519 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27d54469-f670-4b34-b794-6bc485d670cf-workload-certs\") pod \"e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:06.639509 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.639481 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27d54469-f670-4b34-b794-6bc485d670cf-istio-envoy\") pod \"e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:06.639941 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.639921 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27d54469-f670-4b34-b794-6bc485d670cf-istio-podinfo\") pod \"e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:06.645532 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.645509 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27d54469-f670-4b34-b794-6bc485d670cf-istio-token\") pod \"e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:06.652952 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.652926 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc5zg\" (UniqueName: \"kubernetes.io/projected/27d54469-f670-4b34-b794-6bc485d670cf-kube-api-access-wc5zg\") pod \"e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:06.724676 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.724641 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:40:06.765348 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.765314 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:06.907808 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.907770 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24"] Jul 15 17:40:06.908907 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:40:06.908880 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27d54469_f670_4b34_b794_6bc485d670cf.slice/crio-bd5ff309923a62d2f4c169057a0b6da7d7956042dd495eb7e12d48388a9567dd WatchSource:0}: Error finding container bd5ff309923a62d2f4c169057a0b6da7d7956042dd495eb7e12d48388a9567dd: Status 404 returned error can't find the container with id bd5ff309923a62d2f4c169057a0b6da7d7956042dd495eb7e12d48388a9567dd Jul 15 17:40:06.910882 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.910850 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 17:40:06.910990 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.910927 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 17:40:06.910990 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:06.910970 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 17:40:07.260563 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:07.260526 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" event={"ID":"27d54469-f670-4b34-b794-6bc485d670cf","Type":"ContainerStarted","Data":"04ae4256f777af93f4974b4c7348c08b123968a7209dce8adab9937a2df2a521"} Jul 15 17:40:07.260936 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:07.260583 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" event={"ID":"27d54469-f670-4b34-b794-6bc485d670cf","Type":"ContainerStarted","Data":"bd5ff309923a62d2f4c169057a0b6da7d7956042dd495eb7e12d48388a9567dd"} Jul 15 17:40:07.294155 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:07.294108 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" podStartSLOduration=1.294092559 podStartE2EDuration="1.294092559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:40:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:40:07.291295032 +0000 UTC m=+2624.844425264" watchObservedRunningTime="2026-07-15 17:40:07.294092559 +0000 UTC m=+2624.847222772" Jul 15 17:40:07.765618 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:07.765568 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:07.771275 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:07.771250 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:08.266447 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:08.266413 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:08.267512 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:08.267492 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:09.245258 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:09.245223 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:40:09.345326 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:09.345294 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:40:09.631193 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:09.631094 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:40:09.911041 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:09.910955 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:40:10.033634 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:10.033577 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:40:10.565460 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:10.565419 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:40:13.758297 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:13.758260 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:40:14.097436 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:14.097343 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:40:14.897468 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:14.897428 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:40:14.998580 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:14.998547 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:40:17.445289 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:17.445252 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:40:17.526328 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:17.526291 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:40:17.594396 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:17.594366 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:40:18.330914 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:18.330868 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:40:18.389081 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:18.389048 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:40:21.956359 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:21.956255 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:40:22.346791 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:22.346702 2578 generic.go:356] "Generic (PLEG): container finished" podID="27d54469-f670-4b34-b794-6bc485d670cf" containerID="04ae4256f777af93f4974b4c7348c08b123968a7209dce8adab9937a2df2a521" exitCode=0 Jul 15 17:40:22.346791 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:22.346776 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" event={"ID":"27d54469-f670-4b34-b794-6bc485d670cf","Type":"ContainerDied","Data":"04ae4256f777af93f4974b4c7348c08b123968a7209dce8adab9937a2df2a521"} Jul 15 17:40:22.347192 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:22.347177 2578 scope.go:117] "RemoveContainer" containerID="04ae4256f777af93f4974b4c7348c08b123968a7209dce8adab9937a2df2a521" Jul 15 17:40:22.348191 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:22.348161 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 17:40:22.348305 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:22.348235 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 17:40:22.348305 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:22.348267 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 17:40:23.266693 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:23.266650 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:23.359672 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:23.359626 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" event={"ID":"27d54469-f670-4b34-b794-6bc485d670cf","Type":"ContainerStarted","Data":"acf217a82ec10d30e76ea9f833b2787b9e1d3e8ecaa74111adf05be10aaed608"} Jul 15 17:40:23.765914 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:23.765875 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:23.767178 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:23.767151 2578 patch_prober.go:28] interesting pod/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 15 17:40:23.767233 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:23.767206 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" podUID="27d54469-f670-4b34-b794-6bc485d670cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 15 17:40:24.766451 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:24.766412 2578 patch_prober.go:28] interesting pod/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 15 17:40:24.766870 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:24.766488 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" podUID="27d54469-f670-4b34-b794-6bc485d670cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 15 17:40:25.766673 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:25.766633 2578 patch_prober.go:28] interesting pod/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 15 17:40:25.767064 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:25.766692 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" podUID="27d54469-f670-4b34-b794-6bc485d670cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 15 17:40:26.766688 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:26.766649 2578 patch_prober.go:28] interesting pod/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 15 17:40:26.767076 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:26.766719 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" podUID="27d54469-f670-4b34-b794-6bc485d670cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 15 17:40:27.770350 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:27.770318 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:27.770790 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:27.770622 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:27.771330 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:27.771312 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:40:31.939185 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:31.939151 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:40:32.039874 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:32.039834 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:40:38.548730 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:38.548694 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:40:38.650917 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:38.650869 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:40:39.016940 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:39.016898 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:40:39.488686 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:39.488655 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:40:39.589067 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:40:39.589033 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:41:21.198520 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:41:21.198426 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:41:29.362667 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:41:29.362641 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29wtx_88f8928c-237e-457f-a032-707f48111f36/ovn-acl-logging/0.log" Jul 15 17:41:29.370764 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:41:29.370743 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29wtx_88f8928c-237e-457f-a032-707f48111f36/ovn-acl-logging/0.log" Jul 15 17:41:40.663643 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:41:40.663585 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:42:38.044541 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:42:38.044509 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:43:24.134377 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:43:24.134338 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:43:24.917372 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:43:24.917331 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:43:38.837725 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:43:38.837684 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:43:58.310516 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:43:58.310479 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:44:12.626704 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:44:12.626651 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:44:12.680444 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:44:12.680410 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:44:14.043625 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:44:14.043560 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:44:14.146981 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:44:14.146941 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:44:51.192128 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:44:51.192095 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:44:52.109556 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:44:52.109513 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:45:00.300257 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:00.300227 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq"] Jul 15 17:45:00.303800 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:00.303782 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq" Jul 15 17:45:00.307343 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:00.307319 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-3d6c37\"/\"openshift-service-ca.crt\"" Jul 15 17:45:00.307460 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:00.307321 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-3d6c37\"/\"default-dockercfg-xds9s\"" Jul 15 17:45:00.307460 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:00.307360 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-3d6c37\"/\"kube-root-ca.crt\"" Jul 15 17:45:00.307460 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:00.307323 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-3d6c37\"/\"rate-test-model-3d6c37-kserve-self-signed-certs\"" Jul 15 17:45:00.313169 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:00.313147 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq"] Jul 15 17:45:00.326085 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:00.326059 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:45:00.377337 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:00.377313 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58cbb24a-7d01-4b6f-bc06-223dd34abe34-home\") pod \"rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq\" (UID: \"58cbb24a-7d01-4b6f-bc06-223dd34abe34\") " pod="ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq" Jul 15 17:45:00.377482 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:00.377349 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58cbb24a-7d01-4b6f-bc06-223dd34abe34-dshm\") pod \"rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq\" (UID: \"58cbb24a-7d01-4b6f-bc06-223dd34abe34\") " pod="ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq" Jul 15 17:45:00.377482 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:00.377368 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58cbb24a-7d01-4b6f-bc06-223dd34abe34-model-cache\") pod \"rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq\" (UID: \"58cbb24a-7d01-4b6f-bc06-223dd34abe34\") " pod="ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq" Jul 15 17:45:00.377482 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:00.377397 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clfmr\" (UniqueName: \"kubernetes.io/projected/58cbb24a-7d01-4b6f-bc06-223dd34abe34-kube-api-access-clfmr\") pod \"rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq\" (UID: \"58cbb24a-7d01-4b6f-bc06-223dd34abe34\") " pod="ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq" Jul 15 17:45:00.377482 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:00.377451 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58cbb24a-7d01-4b6f-bc06-223dd34abe34-tmp-dir\") pod \"rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq\" (UID: \"58cbb24a-7d01-4b6f-bc06-223dd34abe34\") " pod="ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq" Jul 15 17:45:00.377482 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:00.377473 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58cbb24a-7d01-4b6f-bc06-223dd34abe34-tls-certs\") pod \"rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq\" (UID: \"58cbb24a-7d01-4b6f-bc06-223dd34abe34\") " pod="ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq" Jul 15 17:45:00.477883 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:00.477853 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58cbb24a-7d01-4b6f-bc06-223dd34abe34-home\") pod \"rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq\" (UID: \"58cbb24a-7d01-4b6f-bc06-223dd34abe34\") " pod="ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq" Jul 15 17:45:00.478032 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:00.477888 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58cbb24a-7d01-4b6f-bc06-223dd34abe34-dshm\") pod \"rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq\" (UID: \"58cbb24a-7d01-4b6f-bc06-223dd34abe34\") " pod="ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq" Jul 15 17:45:00.478032 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:00.477931 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58cbb24a-7d01-4b6f-bc06-223dd34abe34-model-cache\") pod \"rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq\" (UID: \"58cbb24a-7d01-4b6f-bc06-223dd34abe34\") " pod="ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq" Jul 15 17:45:00.478032 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:00.477952 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clfmr\" (UniqueName: \"kubernetes.io/projected/58cbb24a-7d01-4b6f-bc06-223dd34abe34-kube-api-access-clfmr\") pod \"rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq\" (UID: \"58cbb24a-7d01-4b6f-bc06-223dd34abe34\") " pod="ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq" Jul 15 17:45:00.478032 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:00.477980 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58cbb24a-7d01-4b6f-bc06-223dd34abe34-tmp-dir\") pod \"rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq\" (UID: \"58cbb24a-7d01-4b6f-bc06-223dd34abe34\") " pod="ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq" Jul 15 17:45:00.478221 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:00.478136 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58cbb24a-7d01-4b6f-bc06-223dd34abe34-tls-certs\") pod \"rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq\" (UID: \"58cbb24a-7d01-4b6f-bc06-223dd34abe34\") " pod="ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq" Jul 15 17:45:00.478298 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:00.478280 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58cbb24a-7d01-4b6f-bc06-223dd34abe34-home\") pod \"rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq\" (UID: \"58cbb24a-7d01-4b6f-bc06-223dd34abe34\") " pod="ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq" Jul 15 17:45:00.478357 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:00.478316 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58cbb24a-7d01-4b6f-bc06-223dd34abe34-model-cache\") pod \"rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq\" (UID: \"58cbb24a-7d01-4b6f-bc06-223dd34abe34\") " pod="ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq" Jul 15 17:45:00.478357 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:00.478346 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58cbb24a-7d01-4b6f-bc06-223dd34abe34-tmp-dir\") pod \"rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq\" (UID: \"58cbb24a-7d01-4b6f-bc06-223dd34abe34\") " pod="ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq" Jul 15 17:45:00.480310 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:00.480279 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58cbb24a-7d01-4b6f-bc06-223dd34abe34-dshm\") pod \"rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq\" (UID: \"58cbb24a-7d01-4b6f-bc06-223dd34abe34\") " pod="ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq" Jul 15 17:45:00.480638 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:00.480620 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58cbb24a-7d01-4b6f-bc06-223dd34abe34-tls-certs\") pod \"rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq\" (UID: \"58cbb24a-7d01-4b6f-bc06-223dd34abe34\") " pod="ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq" Jul 15 17:45:00.486359 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:00.486338 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clfmr\" (UniqueName: \"kubernetes.io/projected/58cbb24a-7d01-4b6f-bc06-223dd34abe34-kube-api-access-clfmr\") pod \"rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq\" (UID: \"58cbb24a-7d01-4b6f-bc06-223dd34abe34\") " pod="ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq" Jul 15 17:45:00.614718 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:00.614588 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq" Jul 15 17:45:00.751717 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:00.751691 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq"] Jul 15 17:45:00.753631 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:45:00.753584 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod58cbb24a_7d01_4b6f_bc06_223dd34abe34.slice/crio-d234cc56a26c6749a9788f652e95f3e58a0216ef4566d2b205aefc0c90e4eb49 WatchSource:0}: Error finding container d234cc56a26c6749a9788f652e95f3e58a0216ef4566d2b205aefc0c90e4eb49: Status 404 returned error can't find the container with id d234cc56a26c6749a9788f652e95f3e58a0216ef4566d2b205aefc0c90e4eb49 Jul 15 17:45:00.907489 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:00.907400 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq" event={"ID":"58cbb24a-7d01-4b6f-bc06-223dd34abe34","Type":"ContainerStarted","Data":"b0c2b6001c44f86a13a63246efe89031db3e9f1a6d153234161665d834d1c199"} Jul 15 17:45:00.907489 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:00.907451 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq" event={"ID":"58cbb24a-7d01-4b6f-bc06-223dd34abe34","Type":"ContainerStarted","Data":"d234cc56a26c6749a9788f652e95f3e58a0216ef4566d2b205aefc0c90e4eb49"} Jul 15 17:45:00.929858 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:00.929807 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq" podStartSLOduration=0.92979233 podStartE2EDuration="929.79233ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:45:00.927529972 +0000 UTC m=+2918.480660186" watchObservedRunningTime="2026-07-15 17:45:00.92979233 +0000 UTC m=+2918.482922543" Jul 15 17:45:10.615111 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:10.615070 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq" Jul 15 17:45:10.615560 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:10.615134 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq" Jul 15 17:45:10.627472 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:10.627447 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq" Jul 15 17:45:10.975496 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:10.975467 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq" Jul 15 17:45:24.203074 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:24.203011 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:45:32.795781 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:32.795747 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:45:32.896314 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:32.896273 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:45:39.991161 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:39.991125 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:45:40.093456 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:45:40.093406 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:46:12.316266 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:12.316188 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq"] Jul 15 17:46:12.329992 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:12.329958 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:46:13.110857 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:13.110830 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:46:14.083652 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:14.083620 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7203-139774453082160"] Jul 15 17:46:14.087056 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:14.087038 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7203-139774453082160" Jul 15 17:46:14.089835 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:14.089808 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-ldrtf\"" Jul 15 17:46:14.094661 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:14.094642 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7203-139774453082160"] Jul 15 17:46:14.216449 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:14.216417 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f67th\" (UniqueName: \"kubernetes.io/projected/3b41b9e4-76d9-4e14-99bb-c661ba3687c4-kube-api-access-f67th\") pod \"test-curl-7203-139774453082160\" (UID: \"3b41b9e4-76d9-4e14-99bb-c661ba3687c4\") " pod="opendatahub/test-curl-7203-139774453082160" Jul 15 17:46:14.317203 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:14.317174 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f67th\" (UniqueName: \"kubernetes.io/projected/3b41b9e4-76d9-4e14-99bb-c661ba3687c4-kube-api-access-f67th\") pod \"test-curl-7203-139774453082160\" (UID: \"3b41b9e4-76d9-4e14-99bb-c661ba3687c4\") " pod="opendatahub/test-curl-7203-139774453082160" Jul 15 17:46:14.325406 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:14.325382 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f67th\" (UniqueName: \"kubernetes.io/projected/3b41b9e4-76d9-4e14-99bb-c661ba3687c4-kube-api-access-f67th\") pod \"test-curl-7203-139774453082160\" (UID: \"3b41b9e4-76d9-4e14-99bb-c661ba3687c4\") " pod="opendatahub/test-curl-7203-139774453082160" Jul 15 17:46:14.397460 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:14.397381 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7203-139774453082160" Jul 15 17:46:14.532634 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:14.532587 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7203-139774453082160"] Jul 15 17:46:14.534360 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:46:14.534332 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3b41b9e4_76d9_4e14_99bb_c661ba3687c4.slice/crio-f248db54a3ea8a548d6a8f168c7cadb89a0369313f9072e0a4fdafd87589cbde WatchSource:0}: Error finding container f248db54a3ea8a548d6a8f168c7cadb89a0369313f9072e0a4fdafd87589cbde: Status 404 returned error can't find the container with id f248db54a3ea8a548d6a8f168c7cadb89a0369313f9072e0a4fdafd87589cbde Jul 15 17:46:14.535769 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:14.535746 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 17:46:15.316783 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:15.316752 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7203-139774453082160" event={"ID":"3b41b9e4-76d9-4e14-99bb-c661ba3687c4","Type":"ContainerStarted","Data":"d1ceb33491555773fc0f24204ade54312119fa8acd169ac52e03d33c2d7823ae"} Jul 15 17:46:15.317080 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:15.316794 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7203-139774453082160" event={"ID":"3b41b9e4-76d9-4e14-99bb-c661ba3687c4","Type":"ContainerStarted","Data":"f248db54a3ea8a548d6a8f168c7cadb89a0369313f9072e0a4fdafd87589cbde"} Jul 15 17:46:16.324815 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:16.324777 2578 generic.go:356] "Generic (PLEG): container finished" podID="3b41b9e4-76d9-4e14-99bb-c661ba3687c4" containerID="d1ceb33491555773fc0f24204ade54312119fa8acd169ac52e03d33c2d7823ae" exitCode=0 Jul 15 17:46:16.325192 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:16.324867 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7203-139774453082160" event={"ID":"3b41b9e4-76d9-4e14-99bb-c661ba3687c4","Type":"ContainerDied","Data":"d1ceb33491555773fc0f24204ade54312119fa8acd169ac52e03d33c2d7823ae"} Jul 15 17:46:17.465959 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:17.465933 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7203-139774453082160" Jul 15 17:46:17.488310 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:17.488290 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7203-139774453082160_3b41b9e4-76d9-4e14-99bb-c661ba3687c4/test-curl-7203-139774453082160/0.log" Jul 15 17:46:17.515413 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:17.515387 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7203-139774453082160"] Jul 15 17:46:17.517914 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:17.517888 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7203-139774453082160"] Jul 15 17:46:17.549535 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:17.549515 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f67th\" (UniqueName: \"kubernetes.io/projected/3b41b9e4-76d9-4e14-99bb-c661ba3687c4-kube-api-access-f67th\") pod \"3b41b9e4-76d9-4e14-99bb-c661ba3687c4\" (UID: \"3b41b9e4-76d9-4e14-99bb-c661ba3687c4\") " Jul 15 17:46:17.551611 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:17.551580 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b41b9e4-76d9-4e14-99bb-c661ba3687c4-kube-api-access-f67th" (OuterVolumeSpecName: "kube-api-access-f67th") pod "3b41b9e4-76d9-4e14-99bb-c661ba3687c4" (UID: "3b41b9e4-76d9-4e14-99bb-c661ba3687c4"). InnerVolumeSpecName "kube-api-access-f67th". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:46:17.642645 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:17.642551 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7203-139774453083952"] Jul 15 17:46:17.642954 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:17.642939 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b41b9e4-76d9-4e14-99bb-c661ba3687c4" containerName="test-curl-7203-139774453082160" Jul 15 17:46:17.643009 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:17.642955 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b41b9e4-76d9-4e14-99bb-c661ba3687c4" containerName="test-curl-7203-139774453082160" Jul 15 17:46:17.643047 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:17.643017 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b41b9e4-76d9-4e14-99bb-c661ba3687c4" containerName="test-curl-7203-139774453082160" Jul 15 17:46:17.646106 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:17.646088 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7203-139774453083952" Jul 15 17:46:17.650711 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:17.650684 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f67th\" (UniqueName: \"kubernetes.io/projected/3b41b9e4-76d9-4e14-99bb-c661ba3687c4-kube-api-access-f67th\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:46:17.653700 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:17.653677 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7203-139774453083952"] Jul 15 17:46:17.751763 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:17.751732 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffdqx\" (UniqueName: \"kubernetes.io/projected/7f6336ed-669a-457c-81e4-a52d3fd61cf8-kube-api-access-ffdqx\") pod \"test-curl-7203-139774453083952\" (UID: \"7f6336ed-669a-457c-81e4-a52d3fd61cf8\") " pod="opendatahub/test-curl-7203-139774453083952" Jul 15 17:46:17.852849 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:17.852811 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffdqx\" (UniqueName: \"kubernetes.io/projected/7f6336ed-669a-457c-81e4-a52d3fd61cf8-kube-api-access-ffdqx\") pod \"test-curl-7203-139774453083952\" (UID: \"7f6336ed-669a-457c-81e4-a52d3fd61cf8\") " pod="opendatahub/test-curl-7203-139774453083952" Jul 15 17:46:17.861453 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:17.861421 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffdqx\" (UniqueName: \"kubernetes.io/projected/7f6336ed-669a-457c-81e4-a52d3fd61cf8-kube-api-access-ffdqx\") pod \"test-curl-7203-139774453083952\" (UID: \"7f6336ed-669a-457c-81e4-a52d3fd61cf8\") " pod="opendatahub/test-curl-7203-139774453083952" Jul 15 17:46:17.956539 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:17.956513 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7203-139774453083952" Jul 15 17:46:18.093399 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:18.093371 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7203-139774453083952"] Jul 15 17:46:18.095000 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:46:18.094968 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7f6336ed_669a_457c_81e4_a52d3fd61cf8.slice/crio-4460d81c412c30f8e8b1c950958dc367edd61095576801e741dc121e9e469dab WatchSource:0}: Error finding container 4460d81c412c30f8e8b1c950958dc367edd61095576801e741dc121e9e469dab: Status 404 returned error can't find the container with id 4460d81c412c30f8e8b1c950958dc367edd61095576801e741dc121e9e469dab Jul 15 17:46:18.340049 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:18.339971 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7203-139774453082160" Jul 15 17:46:18.340049 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:18.340014 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f248db54a3ea8a548d6a8f168c7cadb89a0369313f9072e0a4fdafd87589cbde" Jul 15 17:46:18.341348 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:18.341323 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7203-139774453083952" event={"ID":"7f6336ed-669a-457c-81e4-a52d3fd61cf8","Type":"ContainerStarted","Data":"4460d81c412c30f8e8b1c950958dc367edd61095576801e741dc121e9e469dab"} Jul 15 17:46:18.351121 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:46:18.351097 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7203-139774453082160\" is forbidden: User \"system:node:ip-10-0-140-149.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-149.ec2.internal' and this object" podUID="3b41b9e4-76d9-4e14-99bb-c661ba3687c4" pod="opendatahub/test-curl-7203-139774453082160" Jul 15 17:46:18.990669 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:18.990636 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b41b9e4-76d9-4e14-99bb-c661ba3687c4" path="/var/lib/kubelet/pods/3b41b9e4-76d9-4e14-99bb-c661ba3687c4/volumes" Jul 15 17:46:19.349333 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:19.349239 2578 generic.go:356] "Generic (PLEG): container finished" podID="7f6336ed-669a-457c-81e4-a52d3fd61cf8" containerID="63cb626094c5280df947b241c4d483a07cb266bed559b6ca175db20fcef3beb8" exitCode=0 Jul 15 17:46:19.349333 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:19.349303 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7203-139774453083952" event={"ID":"7f6336ed-669a-457c-81e4-a52d3fd61cf8","Type":"ContainerDied","Data":"63cb626094c5280df947b241c4d483a07cb266bed559b6ca175db20fcef3beb8"} Jul 15 17:46:20.495908 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:20.495883 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7203-139774453083952" Jul 15 17:46:20.518491 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:20.518467 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7203-139774453083952_7f6336ed-669a-457c-81e4-a52d3fd61cf8/test-curl-7203-139774453083952/0.log" Jul 15 17:46:20.546977 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:20.546946 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7203-139774453083952"] Jul 15 17:46:20.553227 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:20.553203 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7203-139774453083952"] Jul 15 17:46:20.578057 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:20.578033 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffdqx\" (UniqueName: \"kubernetes.io/projected/7f6336ed-669a-457c-81e4-a52d3fd61cf8-kube-api-access-ffdqx\") pod \"7f6336ed-669a-457c-81e4-a52d3fd61cf8\" (UID: \"7f6336ed-669a-457c-81e4-a52d3fd61cf8\") " Jul 15 17:46:20.580124 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:20.580098 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f6336ed-669a-457c-81e4-a52d3fd61cf8-kube-api-access-ffdqx" (OuterVolumeSpecName: "kube-api-access-ffdqx") pod "7f6336ed-669a-457c-81e4-a52d3fd61cf8" (UID: "7f6336ed-669a-457c-81e4-a52d3fd61cf8"). InnerVolumeSpecName "kube-api-access-ffdqx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:46:20.679021 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:20.678981 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffdqx\" (UniqueName: \"kubernetes.io/projected/7f6336ed-669a-457c-81e4-a52d3fd61cf8-kube-api-access-ffdqx\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:46:20.716452 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:20.716422 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7203-139774453189200"] Jul 15 17:46:20.716836 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:20.716824 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f6336ed-669a-457c-81e4-a52d3fd61cf8" containerName="test-curl-7203-139774453083952" Jul 15 17:46:20.716880 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:20.716838 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f6336ed-669a-457c-81e4-a52d3fd61cf8" containerName="test-curl-7203-139774453083952" Jul 15 17:46:20.716919 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:20.716911 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f6336ed-669a-457c-81e4-a52d3fd61cf8" containerName="test-curl-7203-139774453083952" Jul 15 17:46:20.719885 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:20.719868 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7203-139774453189200" Jul 15 17:46:20.724578 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:20.724558 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7203-139774453189200"] Jul 15 17:46:20.880498 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:20.880460 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2ctl\" (UniqueName: \"kubernetes.io/projected/82738e23-0305-42ec-b9d0-2300849536dc-kube-api-access-t2ctl\") pod \"test-curl-7203-139774453189200\" (UID: \"82738e23-0305-42ec-b9d0-2300849536dc\") " pod="opendatahub/test-curl-7203-139774453189200" Jul 15 17:46:20.981510 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:20.981421 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2ctl\" (UniqueName: \"kubernetes.io/projected/82738e23-0305-42ec-b9d0-2300849536dc-kube-api-access-t2ctl\") pod \"test-curl-7203-139774453189200\" (UID: \"82738e23-0305-42ec-b9d0-2300849536dc\") " pod="opendatahub/test-curl-7203-139774453189200" Jul 15 17:46:20.989459 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:20.989432 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2ctl\" (UniqueName: \"kubernetes.io/projected/82738e23-0305-42ec-b9d0-2300849536dc-kube-api-access-t2ctl\") pod \"test-curl-7203-139774453189200\" (UID: \"82738e23-0305-42ec-b9d0-2300849536dc\") " pod="opendatahub/test-curl-7203-139774453189200" Jul 15 17:46:20.990618 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:20.990575 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f6336ed-669a-457c-81e4-a52d3fd61cf8" path="/var/lib/kubelet/pods/7f6336ed-669a-457c-81e4-a52d3fd61cf8/volumes" Jul 15 17:46:21.029401 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:21.029378 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7203-139774453189200" Jul 15 17:46:21.162804 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:21.162779 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7203-139774453189200"] Jul 15 17:46:21.164011 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:46:21.163987 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod82738e23_0305_42ec_b9d0_2300849536dc.slice/crio-acc32c9c077b47bfb7a3ee9eb8704827799c7c2d57b1de5d7f9796cedd3077b8 WatchSource:0}: Error finding container acc32c9c077b47bfb7a3ee9eb8704827799c7c2d57b1de5d7f9796cedd3077b8: Status 404 returned error can't find the container with id acc32c9c077b47bfb7a3ee9eb8704827799c7c2d57b1de5d7f9796cedd3077b8 Jul 15 17:46:21.363143 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:21.363065 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7203-139774453083952" Jul 15 17:46:21.363143 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:21.363090 2578 scope.go:117] "RemoveContainer" containerID="63cb626094c5280df947b241c4d483a07cb266bed559b6ca175db20fcef3beb8" Jul 15 17:46:21.364401 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:21.364331 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7203-139774453189200" event={"ID":"82738e23-0305-42ec-b9d0-2300849536dc","Type":"ContainerStarted","Data":"acc32c9c077b47bfb7a3ee9eb8704827799c7c2d57b1de5d7f9796cedd3077b8"} Jul 15 17:46:22.372059 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:22.372023 2578 generic.go:356] "Generic (PLEG): container finished" podID="82738e23-0305-42ec-b9d0-2300849536dc" containerID="7dd92717aaf3f21a41f94caf789ac283bfede3fb29fae9db127b49f8f6020e9d" exitCode=0 Jul 15 17:46:22.372471 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:22.372080 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7203-139774453189200" event={"ID":"82738e23-0305-42ec-b9d0-2300849536dc","Type":"ContainerDied","Data":"7dd92717aaf3f21a41f94caf789ac283bfede3fb29fae9db127b49f8f6020e9d"} Jul 15 17:46:23.526021 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:23.525998 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7203-139774453189200" Jul 15 17:46:23.548706 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:23.548645 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7203-139774453189200_82738e23-0305-42ec-b9d0-2300849536dc/test-curl-7203-139774453189200/0.log" Jul 15 17:46:23.573847 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:23.573824 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7203-139774453189200"] Jul 15 17:46:23.577172 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:23.577150 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7203-139774453189200"] Jul 15 17:46:23.607530 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:23.607509 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t2ctl\" (UniqueName: \"kubernetes.io/projected/82738e23-0305-42ec-b9d0-2300849536dc-kube-api-access-t2ctl\") pod \"82738e23-0305-42ec-b9d0-2300849536dc\" (UID: \"82738e23-0305-42ec-b9d0-2300849536dc\") " Jul 15 17:46:23.609487 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:23.609461 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82738e23-0305-42ec-b9d0-2300849536dc-kube-api-access-t2ctl" (OuterVolumeSpecName: "kube-api-access-t2ctl") pod "82738e23-0305-42ec-b9d0-2300849536dc" (UID: "82738e23-0305-42ec-b9d0-2300849536dc"). InnerVolumeSpecName "kube-api-access-t2ctl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:46:23.691884 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:23.691845 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7203-139774453189200"] Jul 15 17:46:23.692265 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:23.692251 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82738e23-0305-42ec-b9d0-2300849536dc" containerName="test-curl-7203-139774453189200" Jul 15 17:46:23.692310 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:23.692267 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="82738e23-0305-42ec-b9d0-2300849536dc" containerName="test-curl-7203-139774453189200" Jul 15 17:46:23.692352 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:23.692331 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="82738e23-0305-42ec-b9d0-2300849536dc" containerName="test-curl-7203-139774453189200" Jul 15 17:46:23.695374 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:23.695358 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7203-139774453189200" Jul 15 17:46:23.698660 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:23.698635 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7203-139774453189200"] Jul 15 17:46:23.708581 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:23.708559 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t2ctl\" (UniqueName: \"kubernetes.io/projected/82738e23-0305-42ec-b9d0-2300849536dc-kube-api-access-t2ctl\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:46:23.809898 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:23.809828 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5phlm\" (UniqueName: \"kubernetes.io/projected/13021fec-b819-41eb-8c35-ecfdc0fd09c3-kube-api-access-5phlm\") pod \"test-curl-7203-139774453189200\" (UID: \"13021fec-b819-41eb-8c35-ecfdc0fd09c3\") " pod="opendatahub/test-curl-7203-139774453189200" Jul 15 17:46:23.910796 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:23.910751 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5phlm\" (UniqueName: \"kubernetes.io/projected/13021fec-b819-41eb-8c35-ecfdc0fd09c3-kube-api-access-5phlm\") pod \"test-curl-7203-139774453189200\" (UID: \"13021fec-b819-41eb-8c35-ecfdc0fd09c3\") " pod="opendatahub/test-curl-7203-139774453189200" Jul 15 17:46:23.920816 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:23.920783 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5phlm\" (UniqueName: \"kubernetes.io/projected/13021fec-b819-41eb-8c35-ecfdc0fd09c3-kube-api-access-5phlm\") pod \"test-curl-7203-139774453189200\" (UID: \"13021fec-b819-41eb-8c35-ecfdc0fd09c3\") " pod="opendatahub/test-curl-7203-139774453189200" Jul 15 17:46:24.005758 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:24.005728 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7203-139774453189200" Jul 15 17:46:24.348556 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:24.348533 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7203-139774453189200"] Jul 15 17:46:24.349575 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:46:24.349545 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod13021fec_b819_41eb_8c35_ecfdc0fd09c3.slice/crio-7ce4aac6d66fffe66028bd8e022e5fd423c2fab996e154246643eca87477ef8d WatchSource:0}: Error finding container 7ce4aac6d66fffe66028bd8e022e5fd423c2fab996e154246643eca87477ef8d: Status 404 returned error can't find the container with id 7ce4aac6d66fffe66028bd8e022e5fd423c2fab996e154246643eca87477ef8d Jul 15 17:46:24.387937 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:24.387918 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7203-139774453189200" Jul 15 17:46:24.388048 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:24.387953 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="acc32c9c077b47bfb7a3ee9eb8704827799c7c2d57b1de5d7f9796cedd3077b8" Jul 15 17:46:24.389151 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:24.389117 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7203-139774453189200" event={"ID":"13021fec-b819-41eb-8c35-ecfdc0fd09c3","Type":"ContainerStarted","Data":"7ce4aac6d66fffe66028bd8e022e5fd423c2fab996e154246643eca87477ef8d"} Jul 15 17:46:24.400533 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:24.400502 2578 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7203-139774453189200" oldPodUID="82738e23-0305-42ec-b9d0-2300849536dc" podUID="13021fec-b819-41eb-8c35-ecfdc0fd09c3" Jul 15 17:46:24.990322 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:24.990237 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82738e23-0305-42ec-b9d0-2300849536dc" path="/var/lib/kubelet/pods/82738e23-0305-42ec-b9d0-2300849536dc/volumes" Jul 15 17:46:25.396152 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:25.396062 2578 generic.go:356] "Generic (PLEG): container finished" podID="13021fec-b819-41eb-8c35-ecfdc0fd09c3" containerID="1d34ece1916219e66edf57478e8fa7951d37f656219fcb0bdf340ded77844f24" exitCode=0 Jul 15 17:46:25.396294 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:25.396145 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7203-139774453189200" event={"ID":"13021fec-b819-41eb-8c35-ecfdc0fd09c3","Type":"ContainerDied","Data":"1d34ece1916219e66edf57478e8fa7951d37f656219fcb0bdf340ded77844f24"} Jul 15 17:46:26.542716 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:26.542692 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7203-139774453189200" Jul 15 17:46:26.563558 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:26.563522 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7203-139774453189200_13021fec-b819-41eb-8c35-ecfdc0fd09c3/test-curl-7203-139774453189200/0.log" Jul 15 17:46:26.589134 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:26.589106 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7203-139774453189200"] Jul 15 17:46:26.593202 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:26.593180 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7203-139774453189200"] Jul 15 17:46:26.635110 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:26.635084 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5phlm\" (UniqueName: \"kubernetes.io/projected/13021fec-b819-41eb-8c35-ecfdc0fd09c3-kube-api-access-5phlm\") pod \"13021fec-b819-41eb-8c35-ecfdc0fd09c3\" (UID: \"13021fec-b819-41eb-8c35-ecfdc0fd09c3\") " Jul 15 17:46:26.637188 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:26.637168 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13021fec-b819-41eb-8c35-ecfdc0fd09c3-kube-api-access-5phlm" (OuterVolumeSpecName: "kube-api-access-5phlm") pod "13021fec-b819-41eb-8c35-ecfdc0fd09c3" (UID: "13021fec-b819-41eb-8c35-ecfdc0fd09c3"). InnerVolumeSpecName "kube-api-access-5phlm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:46:26.712501 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:26.712470 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7203-139774453190208"] Jul 15 17:46:26.712859 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:26.712845 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13021fec-b819-41eb-8c35-ecfdc0fd09c3" containerName="test-curl-7203-139774453189200" Jul 15 17:46:26.712912 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:26.712861 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="13021fec-b819-41eb-8c35-ecfdc0fd09c3" containerName="test-curl-7203-139774453189200" Jul 15 17:46:26.712949 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:26.712926 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="13021fec-b819-41eb-8c35-ecfdc0fd09c3" containerName="test-curl-7203-139774453189200" Jul 15 17:46:26.715923 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:26.715906 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7203-139774453190208" Jul 15 17:46:26.720931 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:26.720909 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7203-139774453190208"] Jul 15 17:46:26.736948 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:26.736916 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5phlm\" (UniqueName: \"kubernetes.io/projected/13021fec-b819-41eb-8c35-ecfdc0fd09c3-kube-api-access-5phlm\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:46:26.837505 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:26.837469 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdkqk\" (UniqueName: \"kubernetes.io/projected/fa4d4edc-ac37-40ff-bfb1-f7faea4b614e-kube-api-access-sdkqk\") pod \"test-curl-7203-139774453190208\" (UID: \"fa4d4edc-ac37-40ff-bfb1-f7faea4b614e\") " pod="opendatahub/test-curl-7203-139774453190208" Jul 15 17:46:26.938954 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:26.938923 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdkqk\" (UniqueName: \"kubernetes.io/projected/fa4d4edc-ac37-40ff-bfb1-f7faea4b614e-kube-api-access-sdkqk\") pod \"test-curl-7203-139774453190208\" (UID: \"fa4d4edc-ac37-40ff-bfb1-f7faea4b614e\") " pod="opendatahub/test-curl-7203-139774453190208" Jul 15 17:46:26.947826 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:26.947800 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdkqk\" (UniqueName: \"kubernetes.io/projected/fa4d4edc-ac37-40ff-bfb1-f7faea4b614e-kube-api-access-sdkqk\") pod \"test-curl-7203-139774453190208\" (UID: \"fa4d4edc-ac37-40ff-bfb1-f7faea4b614e\") " pod="opendatahub/test-curl-7203-139774453190208" Jul 15 17:46:26.990645 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:26.990572 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13021fec-b819-41eb-8c35-ecfdc0fd09c3" path="/var/lib/kubelet/pods/13021fec-b819-41eb-8c35-ecfdc0fd09c3/volumes" Jul 15 17:46:27.025697 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:27.025663 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7203-139774453190208" Jul 15 17:46:27.159679 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:27.159636 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7203-139774453190208"] Jul 15 17:46:27.337112 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:27.336987 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq" podUID="58cbb24a-7d01-4b6f-bc06-223dd34abe34" containerName="main" containerID="cri-o://b0c2b6001c44f86a13a63246efe89031db3e9f1a6d153234161665d834d1c199" gracePeriod=45 Jul 15 17:46:27.409882 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:27.409843 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7203-139774453190208" event={"ID":"fa4d4edc-ac37-40ff-bfb1-f7faea4b614e","Type":"ContainerStarted","Data":"321d0461d20e6ae3cbd6bd0e19b6a82e3be43c8be89a053587abb04d21130563"} Jul 15 17:46:27.411138 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:27.411117 2578 scope.go:117] "RemoveContainer" containerID="1d34ece1916219e66edf57478e8fa7951d37f656219fcb0bdf340ded77844f24" Jul 15 17:46:27.411194 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:27.411153 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7203-139774453189200" Jul 15 17:46:27.611854 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:27.611832 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq" Jul 15 17:46:27.746173 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:27.746142 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clfmr\" (UniqueName: \"kubernetes.io/projected/58cbb24a-7d01-4b6f-bc06-223dd34abe34-kube-api-access-clfmr\") pod \"58cbb24a-7d01-4b6f-bc06-223dd34abe34\" (UID: \"58cbb24a-7d01-4b6f-bc06-223dd34abe34\") " Jul 15 17:46:27.746347 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:27.746183 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58cbb24a-7d01-4b6f-bc06-223dd34abe34-dshm\") pod \"58cbb24a-7d01-4b6f-bc06-223dd34abe34\" (UID: \"58cbb24a-7d01-4b6f-bc06-223dd34abe34\") " Jul 15 17:46:27.746347 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:27.746206 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58cbb24a-7d01-4b6f-bc06-223dd34abe34-home\") pod \"58cbb24a-7d01-4b6f-bc06-223dd34abe34\" (UID: \"58cbb24a-7d01-4b6f-bc06-223dd34abe34\") " Jul 15 17:46:27.746347 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:27.746329 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58cbb24a-7d01-4b6f-bc06-223dd34abe34-model-cache\") pod \"58cbb24a-7d01-4b6f-bc06-223dd34abe34\" (UID: \"58cbb24a-7d01-4b6f-bc06-223dd34abe34\") " Jul 15 17:46:27.746483 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:27.746385 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58cbb24a-7d01-4b6f-bc06-223dd34abe34-tmp-dir\") pod \"58cbb24a-7d01-4b6f-bc06-223dd34abe34\" (UID: \"58cbb24a-7d01-4b6f-bc06-223dd34abe34\") " Jul 15 17:46:27.746483 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:27.746450 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58cbb24a-7d01-4b6f-bc06-223dd34abe34-tls-certs\") pod \"58cbb24a-7d01-4b6f-bc06-223dd34abe34\" (UID: \"58cbb24a-7d01-4b6f-bc06-223dd34abe34\") " Jul 15 17:46:27.746581 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:27.746489 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58cbb24a-7d01-4b6f-bc06-223dd34abe34-home" (OuterVolumeSpecName: "home") pod "58cbb24a-7d01-4b6f-bc06-223dd34abe34" (UID: "58cbb24a-7d01-4b6f-bc06-223dd34abe34"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:46:27.746669 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:27.746577 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58cbb24a-7d01-4b6f-bc06-223dd34abe34-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "58cbb24a-7d01-4b6f-bc06-223dd34abe34" (UID: "58cbb24a-7d01-4b6f-bc06-223dd34abe34"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:46:27.746669 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:27.746623 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58cbb24a-7d01-4b6f-bc06-223dd34abe34-model-cache" (OuterVolumeSpecName: "model-cache") pod "58cbb24a-7d01-4b6f-bc06-223dd34abe34" (UID: "58cbb24a-7d01-4b6f-bc06-223dd34abe34"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:46:27.746818 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:27.746794 2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58cbb24a-7d01-4b6f-bc06-223dd34abe34-tmp-dir\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:46:27.746818 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:27.746817 2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58cbb24a-7d01-4b6f-bc06-223dd34abe34-home\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:46:27.746974 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:27.746830 2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58cbb24a-7d01-4b6f-bc06-223dd34abe34-model-cache\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:46:27.748515 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:27.748489 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58cbb24a-7d01-4b6f-bc06-223dd34abe34-dshm" (OuterVolumeSpecName: "dshm") pod "58cbb24a-7d01-4b6f-bc06-223dd34abe34" (UID: "58cbb24a-7d01-4b6f-bc06-223dd34abe34"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:46:27.748674 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:27.748512 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58cbb24a-7d01-4b6f-bc06-223dd34abe34-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "58cbb24a-7d01-4b6f-bc06-223dd34abe34" (UID: "58cbb24a-7d01-4b6f-bc06-223dd34abe34"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:46:27.748674 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:27.748559 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58cbb24a-7d01-4b6f-bc06-223dd34abe34-kube-api-access-clfmr" (OuterVolumeSpecName: "kube-api-access-clfmr") pod "58cbb24a-7d01-4b6f-bc06-223dd34abe34" (UID: "58cbb24a-7d01-4b6f-bc06-223dd34abe34"). InnerVolumeSpecName "kube-api-access-clfmr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:46:27.848096 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:27.848024 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clfmr\" (UniqueName: \"kubernetes.io/projected/58cbb24a-7d01-4b6f-bc06-223dd34abe34-kube-api-access-clfmr\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:46:27.848096 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:27.848051 2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58cbb24a-7d01-4b6f-bc06-223dd34abe34-dshm\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:46:27.848096 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:27.848062 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58cbb24a-7d01-4b6f-bc06-223dd34abe34-tls-certs\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:46:28.417283 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:28.417251 2578 generic.go:356] "Generic (PLEG): container finished" podID="58cbb24a-7d01-4b6f-bc06-223dd34abe34" containerID="b0c2b6001c44f86a13a63246efe89031db3e9f1a6d153234161665d834d1c199" exitCode=0 Jul 15 17:46:28.417487 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:28.417304 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq" Jul 15 17:46:28.417487 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:28.417333 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq" event={"ID":"58cbb24a-7d01-4b6f-bc06-223dd34abe34","Type":"ContainerDied","Data":"b0c2b6001c44f86a13a63246efe89031db3e9f1a6d153234161665d834d1c199"} Jul 15 17:46:28.417487 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:28.417373 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq" event={"ID":"58cbb24a-7d01-4b6f-bc06-223dd34abe34","Type":"ContainerDied","Data":"d234cc56a26c6749a9788f652e95f3e58a0216ef4566d2b205aefc0c90e4eb49"} Jul 15 17:46:28.417487 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:28.417392 2578 scope.go:117] "RemoveContainer" containerID="b0c2b6001c44f86a13a63246efe89031db3e9f1a6d153234161665d834d1c199" Jul 15 17:46:28.419019 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:28.418992 2578 generic.go:356] "Generic (PLEG): container finished" podID="fa4d4edc-ac37-40ff-bfb1-f7faea4b614e" containerID="601a879b0770bc9b3721c87dd7c33f724b34c325eaa69de7d6a3178faff767c8" exitCode=0 Jul 15 17:46:28.419147 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:28.419074 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7203-139774453190208" event={"ID":"fa4d4edc-ac37-40ff-bfb1-f7faea4b614e","Type":"ContainerDied","Data":"601a879b0770bc9b3721c87dd7c33f724b34c325eaa69de7d6a3178faff767c8"} Jul 15 17:46:28.427249 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:28.427231 2578 scope.go:117] "RemoveContainer" containerID="b0c2b6001c44f86a13a63246efe89031db3e9f1a6d153234161665d834d1c199" Jul 15 17:46:28.427484 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:46:28.427461 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0c2b6001c44f86a13a63246efe89031db3e9f1a6d153234161665d834d1c199\": container with ID starting with b0c2b6001c44f86a13a63246efe89031db3e9f1a6d153234161665d834d1c199 not found: ID does not exist" containerID="b0c2b6001c44f86a13a63246efe89031db3e9f1a6d153234161665d834d1c199" Jul 15 17:46:28.427561 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:28.427497 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0c2b6001c44f86a13a63246efe89031db3e9f1a6d153234161665d834d1c199"} err="failed to get container status \"b0c2b6001c44f86a13a63246efe89031db3e9f1a6d153234161665d834d1c199\": rpc error: code = NotFound desc = could not find container \"b0c2b6001c44f86a13a63246efe89031db3e9f1a6d153234161665d834d1c199\": container with ID starting with b0c2b6001c44f86a13a63246efe89031db3e9f1a6d153234161665d834d1c199 not found: ID does not exist" Jul 15 17:46:28.447494 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:28.447472 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq"] Jul 15 17:46:28.451375 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:28.451354 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-3d6c37/rate-test-model-3d6c37-kserve-5bc7d8f947-lbfgq"] Jul 15 17:46:28.991360 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:28.991330 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58cbb24a-7d01-4b6f-bc06-223dd34abe34" path="/var/lib/kubelet/pods/58cbb24a-7d01-4b6f-bc06-223dd34abe34/volumes" Jul 15 17:46:29.401424 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:29.401352 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29wtx_88f8928c-237e-457f-a032-707f48111f36/ovn-acl-logging/0.log" Jul 15 17:46:29.408957 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:29.408936 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29wtx_88f8928c-237e-457f-a032-707f48111f36/ovn-acl-logging/0.log" Jul 15 17:46:29.580326 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:29.580300 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7203-139774453190208" Jul 15 17:46:29.602383 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:29.602355 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7203-139774453190208_fa4d4edc-ac37-40ff-bfb1-f7faea4b614e/test-curl-7203-139774453190208/0.log" Jul 15 17:46:29.627860 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:29.627831 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7203-139774453190208"] Jul 15 17:46:29.630867 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:29.630843 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7203-139774453190208"] Jul 15 17:46:29.759429 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:29.759399 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7203-139774453190208"] Jul 15 17:46:29.759785 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:29.759773 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa4d4edc-ac37-40ff-bfb1-f7faea4b614e" containerName="test-curl-7203-139774453190208" Jul 15 17:46:29.759837 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:29.759787 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa4d4edc-ac37-40ff-bfb1-f7faea4b614e" containerName="test-curl-7203-139774453190208" Jul 15 17:46:29.759837 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:29.759800 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58cbb24a-7d01-4b6f-bc06-223dd34abe34" containerName="main" Jul 15 17:46:29.759837 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:29.759805 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="58cbb24a-7d01-4b6f-bc06-223dd34abe34" containerName="main" Jul 15 17:46:29.759927 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:29.759865 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="58cbb24a-7d01-4b6f-bc06-223dd34abe34" containerName="main" Jul 15 17:46:29.759927 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:29.759873 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa4d4edc-ac37-40ff-bfb1-f7faea4b614e" containerName="test-curl-7203-139774453190208" Jul 15 17:46:29.762831 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:29.762815 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7203-139774453190208" Jul 15 17:46:29.765111 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:29.765086 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sdkqk\" (UniqueName: \"kubernetes.io/projected/fa4d4edc-ac37-40ff-bfb1-f7faea4b614e-kube-api-access-sdkqk\") pod \"fa4d4edc-ac37-40ff-bfb1-f7faea4b614e\" (UID: \"fa4d4edc-ac37-40ff-bfb1-f7faea4b614e\") " Jul 15 17:46:29.767077 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:29.767057 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa4d4edc-ac37-40ff-bfb1-f7faea4b614e-kube-api-access-sdkqk" (OuterVolumeSpecName: "kube-api-access-sdkqk") pod "fa4d4edc-ac37-40ff-bfb1-f7faea4b614e" (UID: "fa4d4edc-ac37-40ff-bfb1-f7faea4b614e"). InnerVolumeSpecName "kube-api-access-sdkqk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:46:29.772679 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:29.772657 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7203-139774453190208"] Jul 15 17:46:29.865949 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:29.865915 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lp5kk\" (UniqueName: \"kubernetes.io/projected/e6b61166-1170-4f6b-9349-45f100f1e60b-kube-api-access-lp5kk\") pod \"test-curl-7203-139774453190208\" (UID: \"e6b61166-1170-4f6b-9349-45f100f1e60b\") " pod="opendatahub/test-curl-7203-139774453190208" Jul 15 17:46:29.866105 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:29.866032 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdkqk\" (UniqueName: \"kubernetes.io/projected/fa4d4edc-ac37-40ff-bfb1-f7faea4b614e-kube-api-access-sdkqk\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:46:29.966430 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:29.966397 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lp5kk\" (UniqueName: \"kubernetes.io/projected/e6b61166-1170-4f6b-9349-45f100f1e60b-kube-api-access-lp5kk\") pod \"test-curl-7203-139774453190208\" (UID: \"e6b61166-1170-4f6b-9349-45f100f1e60b\") " pod="opendatahub/test-curl-7203-139774453190208" Jul 15 17:46:29.975672 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:29.975647 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lp5kk\" (UniqueName: \"kubernetes.io/projected/e6b61166-1170-4f6b-9349-45f100f1e60b-kube-api-access-lp5kk\") pod \"test-curl-7203-139774453190208\" (UID: \"e6b61166-1170-4f6b-9349-45f100f1e60b\") " pod="opendatahub/test-curl-7203-139774453190208" Jul 15 17:46:30.079437 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:30.079362 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7203-139774453190208" Jul 15 17:46:30.215905 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:30.215875 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7203-139774453190208"] Jul 15 17:46:30.217544 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:46:30.217519 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode6b61166_1170_4f6b_9349_45f100f1e60b.slice/crio-7a75d4f288e1cc7678ca83d308479b6e677796f98009a60a8e52c9658c9e4d09 WatchSource:0}: Error finding container 7a75d4f288e1cc7678ca83d308479b6e677796f98009a60a8e52c9658c9e4d09: Status 404 returned error can't find the container with id 7a75d4f288e1cc7678ca83d308479b6e677796f98009a60a8e52c9658c9e4d09 Jul 15 17:46:30.432861 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:30.432836 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7203-139774453190208" Jul 15 17:46:30.433011 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:30.432844 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="321d0461d20e6ae3cbd6bd0e19b6a82e3be43c8be89a053587abb04d21130563" Jul 15 17:46:30.433974 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:30.433945 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7203-139774453190208" event={"ID":"e6b61166-1170-4f6b-9349-45f100f1e60b","Type":"ContainerStarted","Data":"7a75d4f288e1cc7678ca83d308479b6e677796f98009a60a8e52c9658c9e4d09"} Jul 15 17:46:30.446400 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:30.446377 2578 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7203-139774453190208" oldPodUID="fa4d4edc-ac37-40ff-bfb1-f7faea4b614e" podUID="e6b61166-1170-4f6b-9349-45f100f1e60b" Jul 15 17:46:30.991037 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:30.991004 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa4d4edc-ac37-40ff-bfb1-f7faea4b614e" path="/var/lib/kubelet/pods/fa4d4edc-ac37-40ff-bfb1-f7faea4b614e/volumes" Jul 15 17:46:31.148565 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:31.148534 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:46:31.441211 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:31.441178 2578 generic.go:356] "Generic (PLEG): container finished" podID="e6b61166-1170-4f6b-9349-45f100f1e60b" containerID="cec2991cc1a681570ff921b6fb17f3f8a5257b12cd08725793ebbcbe726d8bbf" exitCode=0 Jul 15 17:46:31.441377 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:31.441248 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7203-139774453190208" event={"ID":"e6b61166-1170-4f6b-9349-45f100f1e60b","Type":"ContainerDied","Data":"cec2991cc1a681570ff921b6fb17f3f8a5257b12cd08725793ebbcbe726d8bbf"} Jul 15 17:46:32.578849 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:32.578824 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7203-139774453190208" Jul 15 17:46:32.601465 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:32.601443 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7203-139774453190208_e6b61166-1170-4f6b-9349-45f100f1e60b/test-curl-7203-139774453190208/0.log" Jul 15 17:46:32.626570 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:32.626545 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7203-139774453190208"] Jul 15 17:46:32.630078 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:32.630058 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7203-139774453190208"] Jul 15 17:46:32.690750 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:32.690730 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lp5kk\" (UniqueName: \"kubernetes.io/projected/e6b61166-1170-4f6b-9349-45f100f1e60b-kube-api-access-lp5kk\") pod \"e6b61166-1170-4f6b-9349-45f100f1e60b\" (UID: \"e6b61166-1170-4f6b-9349-45f100f1e60b\") " Jul 15 17:46:32.692818 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:32.692790 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6b61166-1170-4f6b-9349-45f100f1e60b-kube-api-access-lp5kk" (OuterVolumeSpecName: "kube-api-access-lp5kk") pod "e6b61166-1170-4f6b-9349-45f100f1e60b" (UID: "e6b61166-1170-4f6b-9349-45f100f1e60b"). InnerVolumeSpecName "kube-api-access-lp5kk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:46:32.791538 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:32.791502 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lp5kk\" (UniqueName: \"kubernetes.io/projected/e6b61166-1170-4f6b-9349-45f100f1e60b-kube-api-access-lp5kk\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:46:32.991017 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:32.990939 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6b61166-1170-4f6b-9349-45f100f1e60b" path="/var/lib/kubelet/pods/e6b61166-1170-4f6b-9349-45f100f1e60b/volumes" Jul 15 17:46:33.453140 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:33.453110 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7203-139774453190208" Jul 15 17:46:33.453341 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:33.453159 2578 scope.go:117] "RemoveContainer" containerID="cec2991cc1a681570ff921b6fb17f3f8a5257b12cd08725793ebbcbe726d8bbf" Jul 15 17:46:33.472877 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:33.472850 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:46:33.570370 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:33.570335 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:46:43.471856 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:43.471816 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:46:47.927386 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:47.927353 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:46:54.495316 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:54.495272 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:46:54.595373 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:54.595337 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:46:58.614361 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:58.614328 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24"] Jul 15 17:46:58.614784 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:58.614539 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" podUID="27d54469-f670-4b34-b794-6bc485d670cf" containerName="istio-proxy" containerID="cri-o://acf217a82ec10d30e76ea9f833b2787b9e1d3e8ecaa74111adf05be10aaed608" gracePeriod=30 Jul 15 17:46:58.627774 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:58.627739 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:46:58.866287 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:58.866189 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:46:59.668999 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:59.668968 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:46:59.753564 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:46:59.753534 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:47:03.870856 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:03.870833 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:47:03.969218 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:03.969187 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27d54469-f670-4b34-b794-6bc485d670cf-credential-socket\") pod \"27d54469-f670-4b34-b794-6bc485d670cf\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " Jul 15 17:47:03.969383 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:03.969237 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27d54469-f670-4b34-b794-6bc485d670cf-istiod-ca-cert\") pod \"27d54469-f670-4b34-b794-6bc485d670cf\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " Jul 15 17:47:03.969383 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:03.969281 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27d54469-f670-4b34-b794-6bc485d670cf-workload-socket\") pod \"27d54469-f670-4b34-b794-6bc485d670cf\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " Jul 15 17:47:03.969383 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:03.969302 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27d54469-f670-4b34-b794-6bc485d670cf-istio-token\") pod \"27d54469-f670-4b34-b794-6bc485d670cf\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " Jul 15 17:47:03.969383 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:03.969327 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27d54469-f670-4b34-b794-6bc485d670cf-istio-data\") pod \"27d54469-f670-4b34-b794-6bc485d670cf\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " Jul 15 17:47:03.969383 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:03.969360 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27d54469-f670-4b34-b794-6bc485d670cf-istio-envoy\") pod \"27d54469-f670-4b34-b794-6bc485d670cf\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " Jul 15 17:47:03.969676 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:03.969422 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27d54469-f670-4b34-b794-6bc485d670cf-istio-podinfo\") pod \"27d54469-f670-4b34-b794-6bc485d670cf\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " Jul 15 17:47:03.969676 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:03.969465 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27d54469-f670-4b34-b794-6bc485d670cf-workload-certs\") pod \"27d54469-f670-4b34-b794-6bc485d670cf\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " Jul 15 17:47:03.969676 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:03.969493 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wc5zg\" (UniqueName: \"kubernetes.io/projected/27d54469-f670-4b34-b794-6bc485d670cf-kube-api-access-wc5zg\") pod \"27d54469-f670-4b34-b794-6bc485d670cf\" (UID: \"27d54469-f670-4b34-b794-6bc485d670cf\") " Jul 15 17:47:03.969676 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:03.969459 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27d54469-f670-4b34-b794-6bc485d670cf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "27d54469-f670-4b34-b794-6bc485d670cf" (UID: "27d54469-f670-4b34-b794-6bc485d670cf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:47:03.969892 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:03.969755 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27d54469-f670-4b34-b794-6bc485d670cf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "27d54469-f670-4b34-b794-6bc485d670cf" (UID: "27d54469-f670-4b34-b794-6bc485d670cf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:47:03.969892 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:03.969660 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27d54469-f670-4b34-b794-6bc485d670cf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "27d54469-f670-4b34-b794-6bc485d670cf" (UID: "27d54469-f670-4b34-b794-6bc485d670cf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:47:03.969892 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:03.969798 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27d54469-f670-4b34-b794-6bc485d670cf-credential-socket\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:47:03.970043 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:03.970003 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27d54469-f670-4b34-b794-6bc485d670cf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "27d54469-f670-4b34-b794-6bc485d670cf" (UID: "27d54469-f670-4b34-b794-6bc485d670cf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:47:03.970389 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:03.970357 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27d54469-f670-4b34-b794-6bc485d670cf-istio-data" (OuterVolumeSpecName: "istio-data") pod "27d54469-f670-4b34-b794-6bc485d670cf" (UID: "27d54469-f670-4b34-b794-6bc485d670cf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:47:03.971694 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:03.971672 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27d54469-f670-4b34-b794-6bc485d670cf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "27d54469-f670-4b34-b794-6bc485d670cf" (UID: "27d54469-f670-4b34-b794-6bc485d670cf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:47:03.971805 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:03.971708 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27d54469-f670-4b34-b794-6bc485d670cf-istio-token" (OuterVolumeSpecName: "istio-token") pod "27d54469-f670-4b34-b794-6bc485d670cf" (UID: "27d54469-f670-4b34-b794-6bc485d670cf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:47:03.971805 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:03.971772 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/27d54469-f670-4b34-b794-6bc485d670cf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "27d54469-f670-4b34-b794-6bc485d670cf" (UID: "27d54469-f670-4b34-b794-6bc485d670cf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 17:47:03.971805 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:03.971787 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27d54469-f670-4b34-b794-6bc485d670cf-kube-api-access-wc5zg" (OuterVolumeSpecName: "kube-api-access-wc5zg") pod "27d54469-f670-4b34-b794-6bc485d670cf" (UID: "27d54469-f670-4b34-b794-6bc485d670cf"). InnerVolumeSpecName "kube-api-access-wc5zg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:47:04.070965 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:04.070930 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27d54469-f670-4b34-b794-6bc485d670cf-istio-podinfo\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:47:04.070965 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:04.070958 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27d54469-f670-4b34-b794-6bc485d670cf-workload-certs\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:47:04.071181 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:04.071044 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wc5zg\" (UniqueName: \"kubernetes.io/projected/27d54469-f670-4b34-b794-6bc485d670cf-kube-api-access-wc5zg\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:47:04.071181 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:04.071057 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27d54469-f670-4b34-b794-6bc485d670cf-istiod-ca-cert\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:47:04.071181 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:04.071070 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27d54469-f670-4b34-b794-6bc485d670cf-workload-socket\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:47:04.071181 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:04.071083 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27d54469-f670-4b34-b794-6bc485d670cf-istio-token\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:47:04.071181 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:04.071094 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27d54469-f670-4b34-b794-6bc485d670cf-istio-data\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:47:04.071181 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:04.071105 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27d54469-f670-4b34-b794-6bc485d670cf-istio-envoy\") on node \"ip-10-0-140-149.ec2.internal\" DevicePath \"\"" Jul 15 17:47:04.634533 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:04.634500 2578 generic.go:356] "Generic (PLEG): container finished" podID="27d54469-f670-4b34-b794-6bc485d670cf" containerID="acf217a82ec10d30e76ea9f833b2787b9e1d3e8ecaa74111adf05be10aaed608" exitCode=0 Jul 15 17:47:04.634743 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:04.634539 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" event={"ID":"27d54469-f670-4b34-b794-6bc485d670cf","Type":"ContainerDied","Data":"acf217a82ec10d30e76ea9f833b2787b9e1d3e8ecaa74111adf05be10aaed608"} Jul 15 17:47:04.634743 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:04.634555 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" Jul 15 17:47:04.634743 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:04.634592 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24" event={"ID":"27d54469-f670-4b34-b794-6bc485d670cf","Type":"ContainerDied","Data":"bd5ff309923a62d2f4c169057a0b6da7d7956042dd495eb7e12d48388a9567dd"} Jul 15 17:47:04.634743 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:04.634620 2578 scope.go:117] "RemoveContainer" containerID="acf217a82ec10d30e76ea9f833b2787b9e1d3e8ecaa74111adf05be10aaed608" Jul 15 17:47:04.644439 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:04.644421 2578 scope.go:117] "RemoveContainer" containerID="04ae4256f777af93f4974b4c7348c08b123968a7209dce8adab9937a2df2a521" Jul 15 17:47:04.652482 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:04.652466 2578 scope.go:117] "RemoveContainer" containerID="acf217a82ec10d30e76ea9f833b2787b9e1d3e8ecaa74111adf05be10aaed608" Jul 15 17:47:04.652752 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:47:04.652734 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acf217a82ec10d30e76ea9f833b2787b9e1d3e8ecaa74111adf05be10aaed608\": container with ID starting with acf217a82ec10d30e76ea9f833b2787b9e1d3e8ecaa74111adf05be10aaed608 not found: ID does not exist" containerID="acf217a82ec10d30e76ea9f833b2787b9e1d3e8ecaa74111adf05be10aaed608" Jul 15 17:47:04.652796 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:04.652761 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acf217a82ec10d30e76ea9f833b2787b9e1d3e8ecaa74111adf05be10aaed608"} err="failed to get container status \"acf217a82ec10d30e76ea9f833b2787b9e1d3e8ecaa74111adf05be10aaed608\": rpc error: code = NotFound desc = could not find container \"acf217a82ec10d30e76ea9f833b2787b9e1d3e8ecaa74111adf05be10aaed608\": container with ID starting with acf217a82ec10d30e76ea9f833b2787b9e1d3e8ecaa74111adf05be10aaed608 not found: ID does not exist" Jul 15 17:47:04.652796 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:04.652777 2578 scope.go:117] "RemoveContainer" containerID="04ae4256f777af93f4974b4c7348c08b123968a7209dce8adab9937a2df2a521" Jul 15 17:47:04.653003 ip-10-0-140-149 kubenswrapper[2578]: E0715 17:47:04.652985 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04ae4256f777af93f4974b4c7348c08b123968a7209dce8adab9937a2df2a521\": container with ID starting with 04ae4256f777af93f4974b4c7348c08b123968a7209dce8adab9937a2df2a521 not found: ID does not exist" containerID="04ae4256f777af93f4974b4c7348c08b123968a7209dce8adab9937a2df2a521" Jul 15 17:47:04.653040 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:04.653011 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04ae4256f777af93f4974b4c7348c08b123968a7209dce8adab9937a2df2a521"} err="failed to get container status \"04ae4256f777af93f4974b4c7348c08b123968a7209dce8adab9937a2df2a521\": rpc error: code = NotFound desc = could not find container \"04ae4256f777af93f4974b4c7348c08b123968a7209dce8adab9937a2df2a521\": container with ID starting with 04ae4256f777af93f4974b4c7348c08b123968a7209dce8adab9937a2df2a521 not found: ID does not exist" Jul 15 17:47:04.658711 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:04.658691 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24"] Jul 15 17:47:04.663199 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:04.663177 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-3d6c37-openshift-default-fbf9cfb99-wdx24"] Jul 15 17:47:04.990814 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:04.990774 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27d54469-f670-4b34-b794-6bc485d670cf" path="/var/lib/kubelet/pods/27d54469-f670-4b34-b794-6bc485d670cf/volumes" Jul 15 17:47:09.716765 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:09.716735 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:47:14.081381 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:14.081345 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:47:19.952186 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:19.952107 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:47:20.320706 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:20.320613 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:47:20.421080 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:20.421049 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:47:24.787700 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:24.787657 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:47:25.042221 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:25.042134 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:47:25.497049 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:25.497010 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7bcb78779c-85hpg_2fbbec08-4030-4af6-8f8e-4ccc6c783a49/authorino/0.log" Jul 15 17:47:31.758208 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:31.758174 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b767d9f5c-pf7tz_0c424b8d-1396-4915-a075-a379ca4dfad5/copy-manifests/0.log" Jul 15 17:47:31.779119 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:31.779084 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b767d9f5c-pf7tz_0c424b8d-1396-4915-a075-a379ca4dfad5/manager/0.log" Jul 15 17:47:32.417414 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:32.417381 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735595-29zc9_c033c931-9fd1-4689-a00b-002d7379af84/cleanup/0.log" Jul 15 17:47:32.764723 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:32.764643 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-82b50f4c-987sx_4040b104-ab36-44de-9196-dbc947a3830a/revoke-keys/0.log" Jul 15 17:47:32.875297 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:32.875268 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-e7083808-ts4vc_1aa7614b-e873-4db5-a3cb-6715c1cce017/revoke-keys/0.log" Jul 15 17:47:33.802296 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:33.802269 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-92183fb9-2s55d_dc15b0fe-45c4-41db-aa89-9b48a34f6593/revoke-keys/0.log" Jul 15 17:47:34.975503 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:34.975471 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd_b454fd31-3cf9-43f3-80d8-4f38886d03ef/util/0.log" Jul 15 17:47:34.982775 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:34.982748 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd_b454fd31-3cf9-43f3-80d8-4f38886d03ef/pull/0.log" Jul 15 17:47:34.989276 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:34.989254 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd_b454fd31-3cf9-43f3-80d8-4f38886d03ef/extract/0.log" Jul 15 17:47:35.101698 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:35.101670 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh_720f778f-b686-46d0-81b9-c85c5f3db32c/util/0.log" Jul 15 17:47:35.108945 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:35.108920 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh_720f778f-b686-46d0-81b9-c85c5f3db32c/pull/0.log" Jul 15 17:47:35.116830 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:35.116810 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh_720f778f-b686-46d0-81b9-c85c5f3db32c/extract/0.log" Jul 15 17:47:35.235995 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:35.235912 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z_039839f4-8025-477b-b8d7-8137daafc9d9/util/0.log" Jul 15 17:47:35.243459 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:35.243425 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z_039839f4-8025-477b-b8d7-8137daafc9d9/pull/0.log" Jul 15 17:47:35.250914 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:35.250898 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z_039839f4-8025-477b-b8d7-8137daafc9d9/extract/0.log" Jul 15 17:47:35.363523 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:35.363497 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn_2795c4a6-2feb-470e-b29f-8ac8c291b4cc/util/0.log" Jul 15 17:47:35.369763 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:35.369741 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn_2795c4a6-2feb-470e-b29f-8ac8c291b4cc/pull/0.log" Jul 15 17:47:35.375987 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:35.375967 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn_2795c4a6-2feb-470e-b29f-8ac8c291b4cc/extract/0.log" Jul 15 17:47:35.507054 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:35.506970 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7bcb78779c-85hpg_2fbbec08-4030-4af6-8f8e-4ccc6c783a49/authorino/0.log" Jul 15 17:47:36.107663 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:36.107632 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-rk7bm_23a87bb5-b7ed-4ccf-941e-27c800dedc92/manager/0.log" Jul 15 17:47:36.945925 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:36.945897 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b44776b5f-k6nnt_ea760cc3-88ea-467e-94c8-6a790eb96d71/kube-auth-proxy/0.log" Jul 15 17:47:38.244158 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:38.244125 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnmc64_b541d6ad-49d6-4731-8a19-3f3ca904ac51/main/0.log" Jul 15 17:47:44.015501 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:44.015447 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8cbcq/must-gather-8xrbz"] Jul 15 17:47:44.016012 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:44.015935 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27d54469-f670-4b34-b794-6bc485d670cf" containerName="istio-proxy" Jul 15 17:47:44.016012 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:44.015953 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="27d54469-f670-4b34-b794-6bc485d670cf" containerName="istio-proxy" Jul 15 17:47:44.016012 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:44.015965 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6b61166-1170-4f6b-9349-45f100f1e60b" containerName="test-curl-7203-139774453190208" Jul 15 17:47:44.016012 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:44.015971 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6b61166-1170-4f6b-9349-45f100f1e60b" containerName="test-curl-7203-139774453190208" Jul 15 17:47:44.016187 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:44.016063 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="27d54469-f670-4b34-b794-6bc485d670cf" containerName="istio-proxy" Jul 15 17:47:44.016187 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:44.016070 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6b61166-1170-4f6b-9349-45f100f1e60b" containerName="test-curl-7203-139774453190208" Jul 15 17:47:44.016187 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:44.016078 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="27d54469-f670-4b34-b794-6bc485d670cf" containerName="istio-proxy" Jul 15 17:47:44.016187 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:44.016146 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27d54469-f670-4b34-b794-6bc485d670cf" containerName="istio-proxy" Jul 15 17:47:44.016187 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:44.016152 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="27d54469-f670-4b34-b794-6bc485d670cf" containerName="istio-proxy" Jul 15 17:47:44.019227 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:44.019202 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8cbcq/must-gather-8xrbz" Jul 15 17:47:44.021573 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:44.021550 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8cbcq\"/\"kube-root-ca.crt\"" Jul 15 17:47:44.021787 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:44.021754 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8cbcq\"/\"default-dockercfg-zrwf4\"" Jul 15 17:47:44.022643 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:44.022628 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8cbcq\"/\"openshift-service-ca.crt\"" Jul 15 17:47:44.027108 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:44.027085 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8cbcq/must-gather-8xrbz"] Jul 15 17:47:44.031944 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:44.031918 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:47:44.093713 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:44.093680 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" limit=1024 Jul 15 17:47:44.106643 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:44.106620 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkr6f\" (UniqueName: \"kubernetes.io/projected/df8e533b-d8f0-4959-826e-c9c1cf07bddc-kube-api-access-jkr6f\") pod \"must-gather-8xrbz\" (UID: \"df8e533b-d8f0-4959-826e-c9c1cf07bddc\") " pod="openshift-must-gather-8cbcq/must-gather-8xrbz" Jul 15 17:47:44.106813 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:44.106787 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/df8e533b-d8f0-4959-826e-c9c1cf07bddc-must-gather-output\") pod \"must-gather-8xrbz\" (UID: \"df8e533b-d8f0-4959-826e-c9c1cf07bddc\") " pod="openshift-must-gather-8cbcq/must-gather-8xrbz" Jul 15 17:47:44.207248 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:44.207214 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkr6f\" (UniqueName: \"kubernetes.io/projected/df8e533b-d8f0-4959-826e-c9c1cf07bddc-kube-api-access-jkr6f\") pod \"must-gather-8xrbz\" (UID: \"df8e533b-d8f0-4959-826e-c9c1cf07bddc\") " pod="openshift-must-gather-8cbcq/must-gather-8xrbz" Jul 15 17:47:44.207413 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:44.207285 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/df8e533b-d8f0-4959-826e-c9c1cf07bddc-must-gather-output\") pod \"must-gather-8xrbz\" (UID: \"df8e533b-d8f0-4959-826e-c9c1cf07bddc\") " pod="openshift-must-gather-8cbcq/must-gather-8xrbz" Jul 15 17:47:44.207564 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:44.207547 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/df8e533b-d8f0-4959-826e-c9c1cf07bddc-must-gather-output\") pod \"must-gather-8xrbz\" (UID: \"df8e533b-d8f0-4959-826e-c9c1cf07bddc\") " pod="openshift-must-gather-8cbcq/must-gather-8xrbz" Jul 15 17:47:44.214935 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:44.214911 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkr6f\" (UniqueName: \"kubernetes.io/projected/df8e533b-d8f0-4959-826e-c9c1cf07bddc-kube-api-access-jkr6f\") pod \"must-gather-8xrbz\" (UID: \"df8e533b-d8f0-4959-826e-c9c1cf07bddc\") " pod="openshift-must-gather-8cbcq/must-gather-8xrbz" Jul 15 17:47:44.329165 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:44.329077 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8cbcq/must-gather-8xrbz" Jul 15 17:47:44.482841 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:44.482811 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8cbcq/must-gather-8xrbz"] Jul 15 17:47:44.484832 ip-10-0-140-149 kubenswrapper[2578]: W0715 17:47:44.484806 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf8e533b_d8f0_4959_826e_c9c1cf07bddc.slice/crio-224071f9bdb88cd6b02f5022342e36d3b689fb97f09c191bf8ba7ba0bd58c648 WatchSource:0}: Error finding container 224071f9bdb88cd6b02f5022342e36d3b689fb97f09c191bf8ba7ba0bd58c648: Status 404 returned error can't find the container with id 224071f9bdb88cd6b02f5022342e36d3b689fb97f09c191bf8ba7ba0bd58c648 Jul 15 17:47:44.854191 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:44.854156 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8cbcq/must-gather-8xrbz" event={"ID":"df8e533b-d8f0-4959-826e-c9c1cf07bddc","Type":"ContainerStarted","Data":"224071f9bdb88cd6b02f5022342e36d3b689fb97f09c191bf8ba7ba0bd58c648"} Jul 15 17:47:45.863099 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:45.863060 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8cbcq/must-gather-8xrbz" event={"ID":"df8e533b-d8f0-4959-826e-c9c1cf07bddc","Type":"ContainerStarted","Data":"42e51dfbf1ae9e3e825983194c6c457d966bbfb983bda154c46e47310b4d9917"} Jul 15 17:47:45.864541 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:45.864514 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8cbcq/must-gather-8xrbz" event={"ID":"df8e533b-d8f0-4959-826e-c9c1cf07bddc","Type":"ContainerStarted","Data":"5d983fb70bbfcaea3bbbba34100690546c159b2fa1a877bcef6292983f2fb1f6"} Jul 15 17:47:45.882154 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:45.882092 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8cbcq/must-gather-8xrbz" podStartSLOduration=2.124678799 podStartE2EDuration="2.882076844s" totalImagesPullingTime="757.398045ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:47:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:47:45.880371413 +0000 UTC m=+3083.433501628" watchObservedRunningTime="2026-07-15 17:47:45.882076844 +0000 UTC m=+3083.435207058" Jul 15 17:47:46.925460 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:46.925424 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-nzk7p_b25e2334-b15b-4c03-9754-cd9a14f177cc/global-pull-secret-syncer/0.log" Jul 15 17:47:46.985204 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:46.985157 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9whs6_55622740-75bd-4532-bda0-e3fb5aa35185/konnectivity-agent/0.log" Jul 15 17:47:47.096903 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:47.096850 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-149.ec2.internal_3ab212e854df387dd5d9339a36377d05/haproxy/0.log" Jul 15 17:47:51.846813 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:51.846780 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd_b454fd31-3cf9-43f3-80d8-4f38886d03ef/extract/0.log" Jul 15 17:47:51.871219 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:51.871190 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd_b454fd31-3cf9-43f3-80d8-4f38886d03ef/util/0.log" Jul 15 17:47:51.894316 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:51.894284 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jh2xd_b454fd31-3cf9-43f3-80d8-4f38886d03ef/pull/0.log" Jul 15 17:47:51.927836 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:51.927805 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh_720f778f-b686-46d0-81b9-c85c5f3db32c/extract/0.log" Jul 15 17:47:51.949447 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:51.949418 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh_720f778f-b686-46d0-81b9-c85c5f3db32c/util/0.log" Jul 15 17:47:51.972237 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:51.972210 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tnfhh_720f778f-b686-46d0-81b9-c85c5f3db32c/pull/0.log" Jul 15 17:47:52.005539 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:52.005512 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z_039839f4-8025-477b-b8d7-8137daafc9d9/extract/0.log" Jul 15 17:47:52.028303 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:52.028271 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z_039839f4-8025-477b-b8d7-8137daafc9d9/util/0.log" Jul 15 17:47:52.052738 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:52.052701 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ddt4z_039839f4-8025-477b-b8d7-8137daafc9d9/pull/0.log" Jul 15 17:47:52.082411 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:52.082380 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn_2795c4a6-2feb-470e-b29f-8ac8c291b4cc/extract/0.log" Jul 15 17:47:52.105787 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:52.105708 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn_2795c4a6-2feb-470e-b29f-8ac8c291b4cc/util/0.log" Jul 15 17:47:52.131532 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:52.131483 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h6cbn_2795c4a6-2feb-470e-b29f-8ac8c291b4cc/pull/0.log" Jul 15 17:47:52.784824 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:52.784790 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7bcb78779c-85hpg_2fbbec08-4030-4af6-8f8e-4ccc6c783a49/authorino/0.log" Jul 15 17:47:53.141215 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:53.140645 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-rk7bm_23a87bb5-b7ed-4ccf-941e-27c800dedc92/manager/0.log" Jul 15 17:47:54.809864 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:54.809832 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3018848b-f926-43e9-a530-bcb25f059f95/alertmanager/0.log" Jul 15 17:47:54.847909 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:54.847876 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3018848b-f926-43e9-a530-bcb25f059f95/config-reloader/0.log" Jul 15 17:47:54.872329 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:54.872295 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3018848b-f926-43e9-a530-bcb25f059f95/kube-rbac-proxy-web/0.log" Jul 15 17:47:54.898324 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:54.898292 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3018848b-f926-43e9-a530-bcb25f059f95/kube-rbac-proxy/0.log" Jul 15 17:47:54.922441 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:54.922400 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3018848b-f926-43e9-a530-bcb25f059f95/kube-rbac-proxy-metric/0.log" Jul 15 17:47:54.950752 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:54.950723 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3018848b-f926-43e9-a530-bcb25f059f95/prom-label-proxy/0.log" Jul 15 17:47:54.978167 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:54.978141 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3018848b-f926-43e9-a530-bcb25f059f95/init-config-reloader/0.log" Jul 15 17:47:55.026003 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:55.025973 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-qtxm9_68d72523-54cd-412a-b697-0f04e891d984/cluster-monitoring-operator/0.log" Jul 15 17:47:55.183335 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:55.183298 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-6w89l_56d144de-0dbf-4f44-ad3e-3277889a317c/monitoring-plugin/0.log" Jul 15 17:47:55.214113 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:55.214082 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7bfnv_6e05b600-bd45-4d94-8988-48895eb43d8f/node-exporter/0.log" Jul 15 17:47:55.234152 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:55.234126 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7bfnv_6e05b600-bd45-4d94-8988-48895eb43d8f/kube-rbac-proxy/0.log" Jul 15 17:47:55.257367 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:55.257335 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7bfnv_6e05b600-bd45-4d94-8988-48895eb43d8f/init-textfile/0.log" Jul 15 17:47:55.733880 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:55.733847 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-rf7pz_367ca386-e717-4704-94a6-0ef7e9d5a6bf/prometheus-operator/0.log" Jul 15 17:47:55.755565 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:55.755529 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-rf7pz_367ca386-e717-4704-94a6-0ef7e9d5a6bf/kube-rbac-proxy/0.log" Jul 15 17:47:55.824013 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:55.823984 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6dbdd47cdc-gc2g6_22c130e6-6358-403d-bdd5-8a722f066d87/telemeter-client/0.log" Jul 15 17:47:55.850112 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:55.850067 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6dbdd47cdc-gc2g6_22c130e6-6358-403d-bdd5-8a722f066d87/reload/0.log" Jul 15 17:47:55.872613 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:55.872567 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6dbdd47cdc-gc2g6_22c130e6-6358-403d-bdd5-8a722f066d87/kube-rbac-proxy/0.log" Jul 15 17:47:56.601800 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:56.601761 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8cbcq/perf-node-gather-daemonset-hkgv2"] Jul 15 17:47:56.608060 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:56.608030 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-hkgv2" Jul 15 17:47:56.633146 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:56.633115 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8cbcq/perf-node-gather-daemonset-hkgv2"] Jul 15 17:47:56.748201 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:56.748167 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/62814fd3-2717-4546-8f28-8418c0868e98-podres\") pod \"perf-node-gather-daemonset-hkgv2\" (UID: \"62814fd3-2717-4546-8f28-8418c0868e98\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-hkgv2" Jul 15 17:47:56.748397 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:56.748240 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/62814fd3-2717-4546-8f28-8418c0868e98-lib-modules\") pod \"perf-node-gather-daemonset-hkgv2\" (UID: \"62814fd3-2717-4546-8f28-8418c0868e98\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-hkgv2" Jul 15 17:47:56.748397 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:56.748363 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5dnj\" (UniqueName: \"kubernetes.io/projected/62814fd3-2717-4546-8f28-8418c0868e98-kube-api-access-m5dnj\") pod \"perf-node-gather-daemonset-hkgv2\" (UID: \"62814fd3-2717-4546-8f28-8418c0868e98\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-hkgv2" Jul 15 17:47:56.748520 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:56.748471 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/62814fd3-2717-4546-8f28-8418c0868e98-proc\") pod \"perf-node-gather-daemonset-hkgv2\" (UID: \"62814fd3-2717-4546-8f28-8418c0868e98\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-hkgv2" Jul 15 17:47:56.748520 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:56.748502 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/62814fd3-2717-4546-8f28-8418c0868e98-sys\") pod \"perf-node-gather-daemonset-hkgv2\" (UID: \"62814fd3-2717-4546-8f28-8418c0868e98\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-hkgv2" Jul 15 17:47:56.849121 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:56.849079 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/62814fd3-2717-4546-8f28-8418c0868e98-proc\") pod \"perf-node-gather-daemonset-hkgv2\" (UID: \"62814fd3-2717-4546-8f28-8418c0868e98\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-hkgv2" Jul 15 17:47:56.849573 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:56.849134 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/62814fd3-2717-4546-8f28-8418c0868e98-sys\") pod \"perf-node-gather-daemonset-hkgv2\" (UID: \"62814fd3-2717-4546-8f28-8418c0868e98\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-hkgv2" Jul 15 17:47:56.849573 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:56.849182 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/62814fd3-2717-4546-8f28-8418c0868e98-podres\") pod \"perf-node-gather-daemonset-hkgv2\" (UID: \"62814fd3-2717-4546-8f28-8418c0868e98\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-hkgv2" Jul 15 17:47:56.849573 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:56.849202 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/62814fd3-2717-4546-8f28-8418c0868e98-proc\") pod \"perf-node-gather-daemonset-hkgv2\" (UID: \"62814fd3-2717-4546-8f28-8418c0868e98\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-hkgv2" Jul 15 17:47:56.849573 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:56.849264 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/62814fd3-2717-4546-8f28-8418c0868e98-sys\") pod \"perf-node-gather-daemonset-hkgv2\" (UID: \"62814fd3-2717-4546-8f28-8418c0868e98\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-hkgv2" Jul 15 17:47:56.849573 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:56.849269 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/62814fd3-2717-4546-8f28-8418c0868e98-lib-modules\") pod \"perf-node-gather-daemonset-hkgv2\" (UID: \"62814fd3-2717-4546-8f28-8418c0868e98\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-hkgv2" Jul 15 17:47:56.849573 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:56.849327 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/62814fd3-2717-4546-8f28-8418c0868e98-podres\") pod \"perf-node-gather-daemonset-hkgv2\" (UID: \"62814fd3-2717-4546-8f28-8418c0868e98\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-hkgv2" Jul 15 17:47:56.849573 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:56.849338 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5dnj\" (UniqueName: \"kubernetes.io/projected/62814fd3-2717-4546-8f28-8418c0868e98-kube-api-access-m5dnj\") pod \"perf-node-gather-daemonset-hkgv2\" (UID: \"62814fd3-2717-4546-8f28-8418c0868e98\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-hkgv2" Jul 15 17:47:56.849573 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:56.849348 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/62814fd3-2717-4546-8f28-8418c0868e98-lib-modules\") pod \"perf-node-gather-daemonset-hkgv2\" (UID: \"62814fd3-2717-4546-8f28-8418c0868e98\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-hkgv2" Jul 15 17:47:56.859959 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:56.859894 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5dnj\" (UniqueName: \"kubernetes.io/projected/62814fd3-2717-4546-8f28-8418c0868e98-kube-api-access-m5dnj\") pod \"perf-node-gather-daemonset-hkgv2\" (UID: \"62814fd3-2717-4546-8f28-8418c0868e98\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-hkgv2" Jul 15 17:47:56.921564 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:56.921527 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-hkgv2" Jul 15 17:47:57.121323 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:57.121116 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8cbcq/perf-node-gather-daemonset-hkgv2"] Jul 15 17:47:57.167956 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:57.167927 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-fzc59_bbfcb238-e568-4881-9f87-6cbad5fe3674/networking-console-plugin/0.log" Jul 15 17:47:57.965425 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:57.965385 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-hkgv2" event={"ID":"62814fd3-2717-4546-8f28-8418c0868e98","Type":"ContainerStarted","Data":"ce961978316f7e822ee40bc85e898965e9954a9e08791555d938bc2ff6d33e68"} Jul 15 17:47:57.965902 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:57.965484 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-hkgv2" event={"ID":"62814fd3-2717-4546-8f28-8418c0868e98","Type":"ContainerStarted","Data":"9940bf0159b9b3d67b26a27d8a1008d58da772252ed40b50630360786fda3b9a"} Jul 15 17:47:57.966294 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:57.966269 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-hkgv2" Jul 15 17:47:57.985997 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:57.985936 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-hkgv2" podStartSLOduration=1.985917363 podStartE2EDuration="1.985917363s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 17:47:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:47:57.982231119 +0000 UTC m=+3095.535361353" watchObservedRunningTime="2026-07-15 17:47:57.985917363 +0000 UTC m=+3095.539047579" Jul 15 17:47:58.240859 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:58.240789 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-fvqjs_9446a817-b692-4190-89c1-a8ae61d15bb7/download-server/0.log" Jul 15 17:47:58.804887 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:58.804859 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-5d55j_7a492e65-62e9-48e5-ae6d-f04e48964b37/volume-data-source-validator/0.log" Jul 15 17:47:59.940395 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:59.940366 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jvr75_191e57d2-b61f-443a-a7ca-a8a7ad699155/dns/0.log" Jul 15 17:47:59.961839 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:59.961808 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jvr75_191e57d2-b61f-443a-a7ca-a8a7ad699155/kube-rbac-proxy/0.log" Jul 15 17:47:59.986334 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:47:59.986309 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cvlbn_c9c2ba93-cd7a-4aa9-81f4-b5afeee112db/dns-node-resolver/0.log" Jul 15 17:48:00.664298 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:48:00.664265 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-whrn6_d0867616-074a-4b55-94ce-df0e85196692/node-ca/0.log" Jul 15 17:48:01.761398 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:48:01.761369 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b44776b5f-k6nnt_ea760cc3-88ea-467e-94c8-6a790eb96d71/kube-auth-proxy/0.log" Jul 15 17:48:02.728138 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:48:02.728108 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qsxzw_f7f08403-7f03-4072-993c-9b02881fd48f/serve-healthcheck-canary/0.log" Jul 15 17:48:03.203096 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:48:03.203056 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-hthbq_f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5/insights-operator/0.log" Jul 15 17:48:03.204154 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:48:03.204130 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-hthbq_f4bf7f61-aa9c-4a55-a300-a5ba3ffc48e5/insights-operator/1.log" Jul 15 17:48:03.291451 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:48:03.291420 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vs662_eb9d6e60-0095-487e-9c95-b26cfed57365/kube-rbac-proxy/0.log" Jul 15 17:48:03.313021 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:48:03.312993 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vs662_eb9d6e60-0095-487e-9c95-b26cfed57365/exporter/0.log" Jul 15 17:48:03.335133 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:48:03.335108 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vs662_eb9d6e60-0095-487e-9c95-b26cfed57365/extractor/0.log" Jul 15 17:48:04.991876 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:48:04.991851 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-hkgv2" Jul 15 17:48:06.056493 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:48:06.056449 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b767d9f5c-pf7tz_0c424b8d-1396-4915-a075-a379ca4dfad5/manager/0.log" Jul 15 17:48:06.077346 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:48:06.077319 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b767d9f5c-pf7tz_0c424b8d-1396-4915-a075-a379ca4dfad5/copy-manifests/0.log" Jul 15 17:48:07.452570 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:48:07.452532 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b44776b5f-k6nnt_ea760cc3-88ea-467e-94c8-6a790eb96d71/kube-auth-proxy/0.log" Jul 15 17:48:07.568261 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:48:07.568226 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5c485d79f6-4szdj_6aa5939a-5bc2-4b1c-9294-f955d142ede2/manager/0.log" Jul 15 17:48:08.845933 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:48:08.845903 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b44776b5f-k6nnt_ea760cc3-88ea-467e-94c8-6a790eb96d71/kube-auth-proxy/0.log" Jul 15 17:48:12.674924 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:48:12.674883 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-plnkx_ab04f5d8-2173-4722-8495-32636d4d91f0/migrator/0.log" Jul 15 17:48:12.694379 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:48:12.694342 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-plnkx_ab04f5d8-2173-4722-8495-32636d4d91f0/graceful-termination/0.log" Jul 15 17:48:14.023471 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:48:14.023439 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-7c67r_16c5ca09-925c-4046-8ade-ae582e290339/kube-multus/0.log" Jul 15 17:48:14.345168 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:48:14.345101 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tv5fg_a488304f-ae1d-4dd5-9af5-a1906ce229a9/kube-multus-additional-cni-plugins/0.log" Jul 15 17:48:14.368321 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:48:14.368293 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tv5fg_a488304f-ae1d-4dd5-9af5-a1906ce229a9/egress-router-binary-copy/0.log" Jul 15 17:48:14.392979 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:48:14.392954 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tv5fg_a488304f-ae1d-4dd5-9af5-a1906ce229a9/cni-plugins/0.log" Jul 15 17:48:14.415835 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:48:14.415815 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tv5fg_a488304f-ae1d-4dd5-9af5-a1906ce229a9/bond-cni-plugin/0.log" Jul 15 17:48:14.433881 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:48:14.433861 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tv5fg_a488304f-ae1d-4dd5-9af5-a1906ce229a9/routeoverride-cni/0.log" Jul 15 17:48:14.453986 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:48:14.453963 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tv5fg_a488304f-ae1d-4dd5-9af5-a1906ce229a9/whereabouts-cni-bincopy/0.log" Jul 15 17:48:14.472965 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:48:14.472938 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tv5fg_a488304f-ae1d-4dd5-9af5-a1906ce229a9/whereabouts-cni/0.log" Jul 15 17:48:14.525682 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:48:14.525647 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-62hbp_df1447df-1b9a-4dc3-ba2c-ecbab1514d09/network-metrics-daemon/0.log" Jul 15 17:48:14.541549 ip-10-0-140-149 kubenswrapper[2578]: I0715 17:48:14.541515 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-62hbp_df1447df-1b9a-4dc3-ba2c-ecbab1514d09/kube-rbac-proxy/0.log"