Jul 12 08:21:16.223815 ip-10-0-136-138 systemd[1]: Starting Kubernetes Kubelet... Jul 12 08:21:16.644760 ip-10-0-136-138 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 08:21:16.644760 ip-10-0-136-138 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 12 08:21:16.644760 ip-10-0-136-138 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 08:21:16.644760 ip-10-0-136-138 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 12 08:21:16.644760 ip-10-0-136-138 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 08:21:16.644760 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.630737 2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 12 08:21:16.644760 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640202 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 08:21:16.644760 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640214 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 08:21:16.644760 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640217 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 08:21:16.644760 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640222 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 08:21:16.644760 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640225 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 08:21:16.644760 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640228 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 08:21:16.644760 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640231 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 08:21:16.644760 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640233 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 08:21:16.656826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640236 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 08:21:16.656826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640239 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 08:21:16.656826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640241 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 08:21:16.656826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640244 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 08:21:16.656826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640247 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 08:21:16.656826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640249 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 08:21:16.656826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640252 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 08:21:16.656826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640255 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 08:21:16.656826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640257 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 08:21:16.656826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640261 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 08:21:16.656826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640265 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 08:21:16.656826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640268 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 08:21:16.656826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640271 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 08:21:16.656826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640275 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 08:21:16.656826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640279 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 08:21:16.656826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640283 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 08:21:16.656826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640286 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 08:21:16.656826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640288 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 08:21:16.656826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640291 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 08:21:16.656826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640294 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 08:21:16.660197 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640296 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 08:21:16.660197 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640299 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 08:21:16.660197 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640302 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 08:21:16.660197 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640305 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 08:21:16.660197 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640309 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 08:21:16.660197 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640312 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 08:21:16.660197 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640315 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 08:21:16.660197 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640317 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 08:21:16.660197 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640320 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 08:21:16.660197 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640322 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 08:21:16.660197 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640325 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 08:21:16.660197 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640328 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 08:21:16.660197 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640330 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 08:21:16.660197 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640335 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 08:21:16.660197 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640339 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 08:21:16.660197 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640342 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 08:21:16.660197 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640345 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 08:21:16.660197 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640348 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 08:21:16.660197 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640351 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 08:21:16.673391 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640355 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 08:21:16.673391 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640358 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 08:21:16.673391 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640361 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 08:21:16.673391 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640363 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 08:21:16.673391 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640366 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 08:21:16.673391 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640369 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 08:21:16.673391 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640371 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 08:21:16.673391 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640374 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 08:21:16.673391 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640377 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 08:21:16.673391 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640379 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 08:21:16.673391 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640382 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 08:21:16.673391 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640384 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 08:21:16.673391 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640387 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 08:21:16.673391 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640389 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 08:21:16.673391 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640392 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 08:21:16.673391 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640395 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 08:21:16.673391 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640398 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 08:21:16.673391 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640401 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 08:21:16.673391 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640403 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 08:21:16.677703 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640406 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 08:21:16.677703 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640408 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 08:21:16.677703 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640411 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 08:21:16.677703 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640413 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 08:21:16.677703 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640416 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 08:21:16.677703 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640418 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 08:21:16.677703 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640421 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 08:21:16.677703 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640423 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 08:21:16.677703 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640425 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 08:21:16.677703 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640428 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 08:21:16.677703 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640430 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 08:21:16.677703 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640434 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 08:21:16.677703 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640436 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 12 08:21:16.677703 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640439 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 08:21:16.677703 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640442 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 08:21:16.677703 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640444 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 08:21:16.677703 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640448 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 08:21:16.677703 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640452 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 08:21:16.677703 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640455 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 08:21:16.677703 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640458 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 08:21:16.695621 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640461 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 08:21:16.695621 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640463 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 08:21:16.695621 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640466 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 08:21:16.695621 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640468 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 08:21:16.695621 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640471 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 08:21:16.695621 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640473 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 08:21:16.695621 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640476 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 08:21:16.695621 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640478 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 08:21:16.695621 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640482 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 08:21:16.695621 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640485 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 08:21:16.695621 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640487 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 08:21:16.695621 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640490 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 08:21:16.695621 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.640492 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 08:21:16.695621 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641197 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 08:21:16.695621 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641204 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 08:21:16.695621 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641209 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 08:21:16.695621 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641212 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 08:21:16.695621 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641214 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 08:21:16.695621 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641217 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 08:21:16.695621 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641220 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 08:21:16.697765 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641223 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 08:21:16.697765 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641226 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 08:21:16.697765 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641229 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 08:21:16.697765 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641232 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 08:21:16.697765 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641235 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 08:21:16.697765 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641237 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 08:21:16.697765 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641240 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 08:21:16.697765 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641243 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 08:21:16.697765 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641245 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 08:21:16.697765 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641248 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 08:21:16.697765 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641251 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 08:21:16.697765 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641253 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 08:21:16.697765 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641256 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 08:21:16.697765 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641259 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 08:21:16.697765 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641261 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 08:21:16.697765 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641264 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 08:21:16.697765 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641266 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 08:21:16.697765 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641280 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 08:21:16.697765 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641284 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 08:21:16.697765 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641288 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 08:21:16.729835 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641292 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 08:21:16.729835 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641295 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 08:21:16.729835 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641298 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 08:21:16.729835 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641300 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 08:21:16.729835 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641303 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 08:21:16.729835 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641308 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 08:21:16.729835 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641311 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 08:21:16.729835 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641314 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 08:21:16.729835 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641316 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 08:21:16.729835 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641319 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 08:21:16.729835 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641321 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 08:21:16.729835 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641325 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 08:21:16.729835 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641328 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 08:21:16.729835 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641331 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 08:21:16.729835 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641333 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 08:21:16.729835 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641336 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 08:21:16.729835 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641339 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 08:21:16.729835 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641341 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 08:21:16.729835 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641344 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 08:21:16.704323 ip-10-0-136-138 systemd[1]: Started Kubernetes Kubelet. Jul 12 08:21:16.732505 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641347 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 08:21:16.732505 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641350 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 08:21:16.732505 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641352 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 08:21:16.732505 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641355 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 08:21:16.732505 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641358 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 08:21:16.732505 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641360 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 08:21:16.732505 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641363 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 08:21:16.732505 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641365 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 08:21:16.732505 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641368 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 08:21:16.732505 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641372 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 08:21:16.732505 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641376 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 08:21:16.732505 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641379 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 08:21:16.732505 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641382 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 08:21:16.732505 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641385 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 08:21:16.732505 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641388 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 08:21:16.732505 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641391 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 08:21:16.732505 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641393 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 08:21:16.732505 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641396 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 08:21:16.732505 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641399 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 08:21:16.736787 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641401 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 08:21:16.736787 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641404 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 08:21:16.736787 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641406 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 08:21:16.736787 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641409 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 08:21:16.736787 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641411 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 08:21:16.736787 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641415 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 08:21:16.736787 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641418 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 08:21:16.736787 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641421 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 08:21:16.736787 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641424 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 08:21:16.736787 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641427 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 08:21:16.736787 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641429 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 08:21:16.736787 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641432 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 08:21:16.736787 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641434 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 08:21:16.736787 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641437 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 08:21:16.736787 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641439 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 08:21:16.736787 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641442 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 08:21:16.736787 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641444 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 08:21:16.736787 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641447 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 08:21:16.736787 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641449 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 08:21:16.760622 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641452 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 08:21:16.760622 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641454 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 08:21:16.760622 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641457 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 08:21:16.760622 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641459 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 08:21:16.760622 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641461 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 12 08:21:16.760622 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641464 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 08:21:16.760622 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641466 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 08:21:16.760622 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641469 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 08:21:16.760622 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641472 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 08:21:16.760622 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641475 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 08:21:16.760622 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641487 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 08:21:16.760622 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641490 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 08:21:16.760622 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641493 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 08:21:16.760622 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641495 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 08:21:16.760622 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641498 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 08:21:16.760622 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641862 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 08:21:16.760622 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641867 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 08:21:16.760622 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641869 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 08:21:16.760622 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641872 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 08:21:16.760622 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641874 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 12 08:21:16.760622 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641877 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 08:21:16.763234 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641880 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 08:21:16.763234 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641882 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 08:21:16.763234 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641887 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 08:21:16.763234 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641891 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 08:21:16.763234 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641894 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 08:21:16.763234 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641896 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 08:21:16.763234 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641899 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 08:21:16.763234 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641903 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 08:21:16.763234 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641906 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 08:21:16.763234 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641909 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 08:21:16.763234 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641913 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 08:21:16.763234 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641915 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 08:21:16.763234 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641918 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 08:21:16.763234 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641921 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 08:21:16.763234 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641923 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 08:21:16.763234 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641926 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 08:21:16.763234 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641929 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 08:21:16.763234 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641931 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 08:21:16.763234 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641934 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 08:21:16.766710 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641937 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 08:21:16.766710 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641939 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 08:21:16.766710 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641942 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 08:21:16.766710 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641945 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 08:21:16.766710 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641947 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 08:21:16.766710 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641950 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 08:21:16.766710 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641952 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 08:21:16.766710 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641957 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 08:21:16.766710 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641959 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 08:21:16.766710 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641962 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 08:21:16.766710 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641965 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 08:21:16.766710 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641968 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 08:21:16.766710 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641970 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 08:21:16.766710 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641974 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 08:21:16.766710 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641977 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 08:21:16.766710 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641979 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 08:21:16.766710 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641982 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 08:21:16.766710 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641984 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 08:21:16.766710 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641987 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 08:21:16.766710 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641989 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 08:21:16.934814 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641992 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 08:21:16.934814 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641994 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 08:21:16.934814 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641996 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 08:21:16.934814 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.641999 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 08:21:16.934814 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642002 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 08:21:16.934814 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642004 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 08:21:16.934814 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642006 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 08:21:16.934814 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642010 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 08:21:16.934814 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642012 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 08:21:16.934814 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642014 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 08:21:16.934814 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642018 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 08:21:16.934814 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642020 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 08:21:16.934814 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642023 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 08:21:16.934814 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642025 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 08:21:16.934814 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642028 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 08:21:16.934814 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642030 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 08:21:16.934814 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642032 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 08:21:16.934814 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642035 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 08:21:16.934814 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642038 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 08:21:16.934814 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642042 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 08:21:16.951398 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642045 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 08:21:16.951398 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642047 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 08:21:16.951398 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642050 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 08:21:16.951398 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642052 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 08:21:16.951398 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642055 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 08:21:16.951398 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642057 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 08:21:16.951398 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642060 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 08:21:16.951398 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642062 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 08:21:16.951398 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642065 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 08:21:16.951398 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642068 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 08:21:16.951398 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642070 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 08:21:16.951398 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642073 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 08:21:16.951398 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642076 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 08:21:16.951398 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642078 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 08:21:16.951398 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642081 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 08:21:16.951398 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642084 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 08:21:16.951398 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642086 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 08:21:16.951398 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642089 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 08:21:16.951398 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642091 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 08:21:16.962034 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642094 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 08:21:16.962034 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642096 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 08:21:16.962034 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642099 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 08:21:16.962034 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642101 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 08:21:16.962034 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642104 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 08:21:16.962034 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642106 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 08:21:16.962034 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642109 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 08:21:16.962034 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642111 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 08:21:16.962034 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642113 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 08:21:16.962034 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642116 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 08:21:16.962034 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642118 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 08:21:16.962034 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642121 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 08:21:16.962034 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642124 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 08:21:16.962034 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642127 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 08:21:16.962034 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642129 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 08:21:16.962034 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642197 2566 flags.go:64] FLAG: --address="0.0.0.0" Jul 12 08:21:16.962034 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642203 2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 12 08:21:16.962034 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642210 2566 flags.go:64] FLAG: --anonymous-auth="true" Jul 12 08:21:16.962034 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642214 2566 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 12 08:21:16.962034 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642218 2566 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 12 08:21:16.962034 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642221 2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 12 08:21:16.963362 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642225 2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 12 08:21:16.963362 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642229 2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 12 08:21:16.963362 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642232 2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 12 08:21:16.963362 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642235 2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 12 08:21:16.963362 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642238 2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 12 08:21:16.963362 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642241 2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 12 08:21:16.963362 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642244 2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 12 08:21:16.963362 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642247 2566 flags.go:64] FLAG: --cgroup-root="" Jul 12 08:21:16.963362 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642250 2566 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 12 08:21:16.963362 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642253 2566 flags.go:64] FLAG: --client-ca-file="" Jul 12 08:21:16.963362 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642257 2566 flags.go:64] FLAG: --cloud-provider="external" Jul 12 08:21:16.963362 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642260 2566 flags.go:64] FLAG: --cluster-dns="[]" Jul 12 08:21:16.963362 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642266 2566 flags.go:64] FLAG: --cluster-domain="" Jul 12 08:21:16.963362 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642268 2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 12 08:21:16.963362 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642271 2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 12 08:21:16.963362 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642275 2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 12 08:21:16.963362 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642278 2566 flags.go:64] FLAG: --container-log-max-files="5" Jul 12 08:21:16.963362 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642282 2566 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 12 08:21:16.963362 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642285 2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 12 08:21:16.963362 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642287 2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 12 08:21:16.963362 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642291 2566 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 12 08:21:16.963362 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642293 2566 flags.go:64] FLAG: --contention-profiling="false" Jul 12 08:21:16.963362 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642297 2566 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 12 08:21:16.963362 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642301 2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 12 08:21:16.966538 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642304 2566 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 12 08:21:16.966538 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642306 2566 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 12 08:21:16.966538 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642310 2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 12 08:21:16.966538 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642313 2566 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 12 08:21:16.966538 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642316 2566 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 12 08:21:16.966538 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642319 2566 flags.go:64] FLAG: --enable-load-reader="false" Jul 12 08:21:16.966538 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642321 2566 flags.go:64] FLAG: --enable-server="true" Jul 12 08:21:16.966538 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642324 2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 12 08:21:16.966538 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642328 2566 flags.go:64] FLAG: --event-burst="100" Jul 12 08:21:16.966538 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642331 2566 flags.go:64] FLAG: --event-qps="50" Jul 12 08:21:16.966538 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642334 2566 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 12 08:21:16.966538 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642337 2566 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 12 08:21:16.966538 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642340 2566 flags.go:64] FLAG: --eviction-hard="" Jul 12 08:21:16.966538 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642343 2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 12 08:21:16.966538 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642346 2566 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 12 08:21:16.966538 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642349 2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 12 08:21:16.966538 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642351 2566 flags.go:64] FLAG: --eviction-soft="" Jul 12 08:21:16.966538 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642354 2566 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 12 08:21:16.966538 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642357 2566 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 12 08:21:16.966538 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642360 2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 12 08:21:16.966538 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642364 2566 flags.go:64] FLAG: --experimental-mounter-path="" Jul 12 08:21:16.966538 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642367 2566 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 12 08:21:16.966538 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642370 2566 flags.go:64] FLAG: --fail-swap-on="true" Jul 12 08:21:16.966538 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642372 2566 flags.go:64] FLAG: --feature-gates="" Jul 12 08:21:16.966538 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642376 2566 flags.go:64] FLAG: --file-check-frequency="20s" Jul 12 08:21:16.968063 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642379 2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 12 08:21:16.968063 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642382 2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 12 08:21:16.968063 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642385 2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 12 08:21:16.968063 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642388 2566 flags.go:64] FLAG: --healthz-port="10248" Jul 12 08:21:16.968063 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642390 2566 flags.go:64] FLAG: --help="false" Jul 12 08:21:16.968063 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642393 2566 flags.go:64] FLAG: --hostname-override="ip-10-0-136-138.ec2.internal" Jul 12 08:21:16.968063 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642400 2566 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 12 08:21:16.968063 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642403 2566 flags.go:64] FLAG: --http-check-frequency="20s" Jul 12 08:21:16.968063 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642405 2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 12 08:21:16.968063 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642409 2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 12 08:21:16.968063 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642412 2566 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 12 08:21:16.968063 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642415 2566 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 12 08:21:16.968063 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642418 2566 flags.go:64] FLAG: --image-service-endpoint="" Jul 12 08:21:16.968063 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642420 2566 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 12 08:21:16.968063 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642423 2566 flags.go:64] FLAG: --kube-api-burst="100" Jul 12 08:21:16.968063 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642426 2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 12 08:21:16.968063 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642428 2566 flags.go:64] FLAG: --kube-api-qps="50" Jul 12 08:21:16.968063 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642431 2566 flags.go:64] FLAG: --kube-reserved="" Jul 12 08:21:16.968063 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642434 2566 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 12 08:21:16.968063 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642437 2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 12 08:21:16.968063 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642439 2566 flags.go:64] FLAG: --kubelet-cgroups="" Jul 12 08:21:16.968063 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642442 2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 12 08:21:16.968063 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642445 2566 flags.go:64] FLAG: --lock-file="" Jul 12 08:21:16.968063 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642447 2566 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 12 08:21:17.072159 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642450 2566 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 12 08:21:17.072159 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642453 2566 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 12 08:21:17.072159 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642458 2566 flags.go:64] FLAG: --log-json-split-stream="false" Jul 12 08:21:17.072159 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642463 2566 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 12 08:21:17.072159 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642466 2566 flags.go:64] FLAG: --log-text-split-stream="false" Jul 12 08:21:17.072159 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642468 2566 flags.go:64] FLAG: --logging-format="text" Jul 12 08:21:17.072159 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642471 2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 12 08:21:17.072159 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642474 2566 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 12 08:21:17.072159 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642477 2566 flags.go:64] FLAG: --manifest-url="" Jul 12 08:21:17.072159 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642480 2566 flags.go:64] FLAG: --manifest-url-header="" Jul 12 08:21:17.072159 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642484 2566 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 12 08:21:17.072159 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642487 2566 flags.go:64] FLAG: --max-open-files="1000000" Jul 12 08:21:17.072159 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642490 2566 flags.go:64] FLAG: --max-pods="110" Jul 12 08:21:17.072159 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642493 2566 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 12 08:21:17.072159 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642497 2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 12 08:21:17.072159 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642500 2566 flags.go:64] FLAG: --memory-manager-policy="None" Jul 12 08:21:17.072159 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642503 2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 12 08:21:17.072159 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642505 2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 12 08:21:17.072159 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642508 2566 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 12 08:21:17.072159 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642511 2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 12 08:21:17.072159 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642518 2566 flags.go:64] FLAG: --node-status-max-images="50" Jul 12 08:21:17.072159 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642521 2566 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 12 08:21:17.072159 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642524 2566 flags.go:64] FLAG: --oom-score-adj="-999" Jul 12 08:21:17.072159 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642527 2566 flags.go:64] FLAG: --pod-cidr="" Jul 12 08:21:17.073889 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642529 2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 12 08:21:17.073889 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642534 2566 flags.go:64] FLAG: --pod-manifest-path="" Jul 12 08:21:17.073889 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642537 2566 flags.go:64] FLAG: --pod-max-pids="-1" Jul 12 08:21:17.073889 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642540 2566 flags.go:64] FLAG: --pods-per-core="0" Jul 12 08:21:17.073889 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642543 2566 flags.go:64] FLAG: --port="10250" Jul 12 08:21:17.073889 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642560 2566 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 12 08:21:17.073889 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642564 2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-099aa02ad7eaa0210" Jul 12 08:21:17.073889 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642568 2566 flags.go:64] FLAG: --qos-reserved="" Jul 12 08:21:17.073889 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642570 2566 flags.go:64] FLAG: --read-only-port="10255" Jul 12 08:21:17.073889 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642573 2566 flags.go:64] FLAG: --register-node="true" Jul 12 08:21:17.073889 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642577 2566 flags.go:64] FLAG: --register-with-taints="" Jul 12 08:21:17.073889 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642582 2566 flags.go:64] FLAG: --registry-burst="10" Jul 12 08:21:17.073889 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642586 2566 flags.go:64] FLAG: --registry-qps="5" Jul 12 08:21:17.073889 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642589 2566 flags.go:64] FLAG: --reserved-cpus="" Jul 12 08:21:17.073889 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642592 2566 flags.go:64] FLAG: --reserved-memory="" Jul 12 08:21:17.073889 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642596 2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 12 08:21:17.073889 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642599 2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 12 08:21:17.073889 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642602 2566 flags.go:64] FLAG: --rotate-certificates="false" Jul 12 08:21:17.073889 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642604 2566 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 12 08:21:17.073889 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642607 2566 flags.go:64] FLAG: --runonce="false" Jul 12 08:21:17.073889 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642610 2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 12 08:21:17.073889 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642613 2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 12 08:21:17.073889 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642616 2566 flags.go:64] FLAG: --seccomp-default="false" Jul 12 08:21:17.073889 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642620 2566 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 12 08:21:17.073889 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642623 2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 12 08:21:17.075211 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642625 2566 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 12 08:21:17.075211 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642628 2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 12 08:21:17.075211 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642631 2566 flags.go:64] FLAG: --storage-driver-password="root" Jul 12 08:21:17.075211 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642634 2566 flags.go:64] FLAG: --storage-driver-secure="false" Jul 12 08:21:17.075211 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642637 2566 flags.go:64] FLAG: --storage-driver-table="stats" Jul 12 08:21:17.075211 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642639 2566 flags.go:64] FLAG: --storage-driver-user="root" Jul 12 08:21:17.075211 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642642 2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 12 08:21:17.075211 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642645 2566 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 12 08:21:17.075211 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642648 2566 flags.go:64] FLAG: --system-cgroups="" Jul 12 08:21:17.075211 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642650 2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 12 08:21:17.075211 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642656 2566 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 12 08:21:17.075211 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642658 2566 flags.go:64] FLAG: --tls-cert-file="" Jul 12 08:21:17.075211 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642661 2566 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 12 08:21:17.075211 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642665 2566 flags.go:64] FLAG: --tls-min-version="" Jul 12 08:21:17.075211 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642667 2566 flags.go:64] FLAG: --tls-private-key-file="" Jul 12 08:21:17.075211 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642670 2566 flags.go:64] FLAG: --topology-manager-policy="none" Jul 12 08:21:17.075211 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642673 2566 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 12 08:21:17.075211 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642676 2566 flags.go:64] FLAG: --topology-manager-scope="container" Jul 12 08:21:17.075211 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642680 2566 flags.go:64] FLAG: --v="2" Jul 12 08:21:17.075211 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642684 2566 flags.go:64] FLAG: --version="false" Jul 12 08:21:17.075211 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642691 2566 flags.go:64] FLAG: --vmodule="" Jul 12 08:21:17.075211 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642694 2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 12 08:21:17.075211 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.642697 2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 12 08:21:17.075211 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642799 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 08:21:17.075211 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642803 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 08:21:17.077347 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642806 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 08:21:17.077347 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642808 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 08:21:17.077347 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642811 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 08:21:17.077347 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642813 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 08:21:17.077347 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642816 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 08:21:17.077347 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642819 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 08:21:17.077347 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642822 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 08:21:17.077347 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642825 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 08:21:17.077347 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642827 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 08:21:17.077347 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642830 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 08:21:17.077347 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642833 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 08:21:17.077347 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642836 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 08:21:17.077347 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642839 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 08:21:17.077347 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642842 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 08:21:17.077347 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642845 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 08:21:17.077347 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642847 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 08:21:17.077347 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642850 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 08:21:17.077347 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642852 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 08:21:17.077347 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642855 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 08:21:17.077347 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642857 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 08:21:17.078446 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642860 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 08:21:17.078446 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642862 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 08:21:17.078446 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642865 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 08:21:17.078446 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642868 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 08:21:17.078446 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642870 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 08:21:17.078446 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642874 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 08:21:17.078446 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642876 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 08:21:17.078446 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642878 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 08:21:17.078446 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642882 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 08:21:17.078446 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642884 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 08:21:17.078446 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642887 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 08:21:17.078446 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642889 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 08:21:17.078446 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642892 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 08:21:17.078446 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642894 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 08:21:17.078446 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642897 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 08:21:17.078446 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642899 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 08:21:17.078446 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642902 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 08:21:17.078446 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642906 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 08:21:17.078446 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642908 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 08:21:17.079937 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642911 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 08:21:17.079937 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642913 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 08:21:17.079937 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642916 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 08:21:17.079937 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642918 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 08:21:17.079937 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642921 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 08:21:17.079937 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642923 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 08:21:17.079937 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642926 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 08:21:17.079937 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642928 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 08:21:17.079937 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642931 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 08:21:17.079937 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642933 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 08:21:17.079937 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642936 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 08:21:17.079937 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642938 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 08:21:17.079937 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642940 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 08:21:17.079937 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642943 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 08:21:17.079937 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642945 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 08:21:17.079937 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642948 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 08:21:17.079937 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642950 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 08:21:17.079937 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642953 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 08:21:17.079937 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642957 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 08:21:17.079937 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642959 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 08:21:17.081021 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642961 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 08:21:17.081021 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642964 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 08:21:17.081021 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642967 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 08:21:17.081021 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642970 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 08:21:17.081021 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642972 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 08:21:17.081021 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642975 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 12 08:21:17.081021 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642977 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 08:21:17.081021 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642981 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 08:21:17.081021 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642985 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 08:21:17.081021 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.642988 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 08:21:17.081021 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.643008 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 08:21:17.081021 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.643013 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 08:21:17.081021 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.643016 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 08:21:17.081021 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.643020 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 08:21:17.081021 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.643025 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 08:21:17.081021 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.643028 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 08:21:17.081021 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.643031 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 08:21:17.081021 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.643034 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 08:21:17.084318 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.643037 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 08:21:17.084318 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.643040 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 08:21:17.084318 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.643042 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 08:21:17.084318 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.643045 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 08:21:17.084318 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.643048 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 08:21:17.084318 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.643051 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 08:21:17.084318 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.643054 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 08:21:17.084318 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.643056 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 08:21:17.084318 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.643059 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 08:21:17.084318 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.643061 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 08:21:17.084318 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.643064 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 08:21:17.084318 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.643066 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 08:21:17.084318 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.643071 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 08:21:17.084318 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.643074 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 08:21:17.084318 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.643077 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 08:21:17.084318 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.643079 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 08:21:17.084318 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.643082 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 08:21:17.084318 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.643085 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 08:21:17.084318 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.643088 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 08:21:17.084318 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.643090 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 08:21:17.086266 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.643747 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 08:21:17.086266 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.650219 2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 12 08:21:17.086266 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.650234 2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 12 08:21:17.086266 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.651088 2566 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 12 08:21:17.086266 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.651095 2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 12 08:21:17.086266 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651147 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 08:21:17.086266 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651152 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 08:21:17.086266 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651155 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 08:21:17.086266 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651158 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 08:21:17.086266 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651161 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 08:21:17.086266 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651164 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 08:21:17.086266 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651167 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 08:21:17.086266 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651170 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 08:21:17.086266 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651172 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 08:21:17.086943 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651175 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 08:21:17.086943 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651177 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 08:21:17.086943 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651180 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 08:21:17.086943 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651182 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 08:21:17.086943 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651185 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 08:21:17.086943 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651188 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 08:21:17.086943 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651191 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 08:21:17.086943 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651193 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 08:21:17.086943 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651196 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 08:21:17.086943 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651199 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 08:21:17.086943 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651202 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 08:21:17.086943 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651205 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 08:21:17.086943 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651208 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 08:21:17.086943 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651210 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 08:21:17.086943 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651212 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 08:21:17.086943 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651215 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 12 08:21:17.086943 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651218 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 08:21:17.086943 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651221 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 08:21:17.086943 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651223 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 08:21:17.086943 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651227 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 08:21:17.087826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651230 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 08:21:17.087826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651233 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 08:21:17.087826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651236 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 08:21:17.087826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651238 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 08:21:17.087826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651241 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 08:21:17.087826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651243 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 08:21:17.087826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651246 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 08:21:17.087826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651248 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 08:21:17.087826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651251 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 08:21:17.087826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651253 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 08:21:17.087826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651256 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 08:21:17.087826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651258 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 08:21:17.087826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651261 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 08:21:17.087826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651263 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 08:21:17.087826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651266 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 08:21:17.087826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651268 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 08:21:17.087826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651270 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 08:21:17.087826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651273 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 08:21:17.087826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651276 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 08:21:17.087826 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651278 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 08:21:17.088690 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651281 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 08:21:17.088690 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651283 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 08:21:17.088690 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651286 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 08:21:17.088690 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651288 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 08:21:17.088690 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651291 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 08:21:17.088690 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651293 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 08:21:17.088690 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651296 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 08:21:17.088690 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651298 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 08:21:17.088690 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651300 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 08:21:17.088690 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651303 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 08:21:17.088690 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651305 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 08:21:17.088690 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651311 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 08:21:17.088690 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651313 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 08:21:17.088690 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651316 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 08:21:17.088690 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651319 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 08:21:17.088690 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651322 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 08:21:17.088690 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651324 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 08:21:17.088690 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651327 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 08:21:17.088690 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651329 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 08:21:17.088690 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651332 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 08:21:17.089569 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651334 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 08:21:17.089569 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651337 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 08:21:17.089569 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651339 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 08:21:17.089569 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651342 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 08:21:17.089569 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651344 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 08:21:17.089569 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651347 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 08:21:17.089569 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651349 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 08:21:17.089569 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651352 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 08:21:17.089569 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651355 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 08:21:17.089569 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651357 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 08:21:17.089569 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651362 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 08:21:17.089569 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651365 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 08:21:17.089569 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651368 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 08:21:17.089569 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651371 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 08:21:17.089569 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651374 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 08:21:17.089569 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651376 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 08:21:17.089569 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651379 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 08:21:17.089569 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651383 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 08:21:17.089569 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651386 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 08:21:17.090390 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651389 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 08:21:17.090390 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651392 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 08:21:17.090390 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651395 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 08:21:17.090390 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651397 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 08:21:17.090390 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651402 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 08:21:17.090390 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651407 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 08:21:17.090390 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651411 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 08:21:17.090390 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651415 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 08:21:17.090390 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651418 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 08:21:17.090390 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651420 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 08:21:17.090390 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651423 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 08:21:17.090390 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.651428 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 08:21:17.090390 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651530 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 08:21:17.090390 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651534 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 08:21:17.090390 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651537 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 08:21:17.091041 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651540 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 08:21:17.091041 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651543 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 08:21:17.091041 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651560 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 08:21:17.091041 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651563 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 08:21:17.091041 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651566 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 08:21:17.091041 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651568 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 08:21:17.091041 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651571 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 08:21:17.091041 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651574 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 08:21:17.091041 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651577 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 08:21:17.091041 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651579 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 08:21:17.091041 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651582 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 08:21:17.091041 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651585 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 08:21:17.091041 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651587 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 08:21:17.091041 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651590 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 08:21:17.091041 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651592 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 08:21:17.091041 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651595 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 08:21:17.091041 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651597 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 08:21:17.091041 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651600 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 08:21:17.091041 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651603 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 08:21:17.091041 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651605 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 08:21:17.091927 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651608 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 08:21:17.091927 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651612 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 08:21:17.091927 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651616 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 08:21:17.091927 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651619 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 08:21:17.091927 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651621 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 08:21:17.091927 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651624 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 08:21:17.091927 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651627 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 08:21:17.091927 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651629 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 08:21:17.091927 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651632 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 08:21:17.091927 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651635 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 08:21:17.091927 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651637 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 08:21:17.091927 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651640 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 08:21:17.091927 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651642 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 12 08:21:17.091927 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651645 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 08:21:17.091927 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651647 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 08:21:17.091927 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651651 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 08:21:17.091927 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651655 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 08:21:17.091927 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651657 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 08:21:17.091927 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651660 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 08:21:17.091927 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651663 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 08:21:17.092793 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651666 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 08:21:17.092793 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651668 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 08:21:17.092793 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651671 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 08:21:17.092793 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651674 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 08:21:17.092793 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651676 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 08:21:17.092793 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651679 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 08:21:17.092793 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651683 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 08:21:17.092793 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651686 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 08:21:17.092793 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651689 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 08:21:17.092793 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651692 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 08:21:17.092793 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651695 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 08:21:17.092793 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651698 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 08:21:17.092793 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651700 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 08:21:17.092793 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651704 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 08:21:17.092793 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651708 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 08:21:17.092793 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651712 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 08:21:17.092793 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651714 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 08:21:17.092793 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651717 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 08:21:17.092793 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651720 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 08:21:17.093638 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651723 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 08:21:17.093638 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651725 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 08:21:17.093638 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651728 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 08:21:17.093638 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651730 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 08:21:17.093638 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651733 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 08:21:17.093638 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651735 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 08:21:17.093638 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651738 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 08:21:17.093638 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651741 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 08:21:17.093638 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651743 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 08:21:17.093638 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651746 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 08:21:17.093638 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651749 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 08:21:17.093638 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651751 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 08:21:17.093638 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651754 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 08:21:17.093638 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651756 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 08:21:17.093638 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651759 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 08:21:17.093638 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651761 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 08:21:17.093638 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651764 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 08:21:17.093638 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651766 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 08:21:17.093638 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651769 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 08:21:17.093638 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651772 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 08:21:17.094491 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651774 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 08:21:17.094491 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651776 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 08:21:17.094491 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651780 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 08:21:17.094491 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651783 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 08:21:17.094491 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651785 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 08:21:17.094491 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651788 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 08:21:17.094491 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651792 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 08:21:17.094491 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651796 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 08:21:17.094491 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651799 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 08:21:17.094491 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651801 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 08:21:17.094491 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651803 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 08:21:17.094491 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651806 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 08:21:17.094491 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651810 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 08:21:17.094491 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651813 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 08:21:17.094491 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651815 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 08:21:17.094491 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651818 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 08:21:17.094491 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:16.651820 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 08:21:17.095233 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.651825 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 08:21:17.095233 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.651917 2566 server.go:962] "Client rotation is on, will bootstrap in background" Jul 12 08:21:17.095233 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.654141 2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 12 08:21:17.095233 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.655609 2566 server.go:1019] "Starting client certificate rotation" Jul 12 08:21:17.095233 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.655709 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 12 08:21:17.095233 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.655746 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 12 08:21:17.095233 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.658121 2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 12 08:21:17.095233 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.659139 2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 12 08:21:17.095233 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.669349 2566 log.go:25] "Validated CRI v1 runtime API" Jul 12 08:21:17.095233 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.671301 2566 log.go:25] "Validated CRI v1 image API" Jul 12 08:21:17.095233 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.672872 2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 12 08:21:17.095233 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.677134 2566 fs.go:135] Filesystem UUIDs: map[70ce1df9-bb32-46d1-a99f-d32c102f0874:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a3d15ae2-0b3f-4a9e-b165-ea8b2363204d:/dev/nvme0n1p4] Jul 12 08:21:17.095781 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.677148 2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 12 08:21:17.095835 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.682879 2566 manager.go:216] Machine: {Timestamp:2026-07-12 08:21:16.680689929 +0000 UTC m=+0.341485582 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3201549 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2cab7ebe2a42b1413c5fb9a6f68936 SystemUUID:ec2cab7e-be2a-42b1-413c-5fb9a6f68936 BootID:cc95156b-fd46-4081-ba5d-434f7e50c551 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:89:a3:0d:78:c5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:89:a3:0d:78:c5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:4e:ba:04:dd:b9:d0 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 12 08:21:17.095835 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.682970 2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 12 08:21:17.095835 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.683040 2566 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 12 08:21:17.095835 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.685977 2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 12 08:21:17.095835 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.686006 2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-138.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 12 08:21:17.095835 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.686819 2566 topology_manager.go:138] "Creating topology manager with none policy" Jul 12 08:21:17.095835 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.686829 2566 container_manager_linux.go:309] "Creating device plugin manager" Jul 12 08:21:17.095835 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.686840 2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.687519 2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.688728 2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.690499 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.691753 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.691864 2566 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.691893 2566 kubelet.go:486] "Attempting to sync node with API server" Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.691906 2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.691919 2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.691928 2566 kubelet.go:398] "Adding apiserver pod source" Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.691937 2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.692254 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.692270 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.696184 2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.698193 2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.698249 2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.698500 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.698513 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.698519 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.698524 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.698532 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.698537 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.699083 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.699089 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.699099 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.699105 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.699113 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.699119 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.700052 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.700059 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.703539 2566 server.go:1268] "Started kubelet" Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.703630 2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.703747 2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.704001 2566 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.706081 2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:16.712692 2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.712746 2566 server.go:310] "Adding debug handlers to kubelet server" Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.713759 2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-138.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:16.713772 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-138.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 12 08:21:17.096232 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:16.713793 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 12 08:21:17.099891 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:16.713789 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-138.ec2.internal.18c17d19b0749bd9 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-138.ec2.internal,UID:ip-10-0-136-138.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-138.ec2.internal,},FirstTimestamp:2026-07-12 08:21:16.703513561 +0000 UTC m=+0.364309212,LastTimestamp:2026-07-12 08:21:16.703513561 +0000 UTC m=+0.364309212,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-138.ec2.internal,}" Jul 12 08:21:17.099891 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.714733 2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 12 08:21:17.099891 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.714754 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 12 08:21:17.099891 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:16.714811 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-138.ec2.internal\" not found" Jul 12 08:21:17.099891 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.715298 2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 12 08:21:17.099891 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.715303 2566 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 12 08:21:17.099891 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.715326 2566 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 12 08:21:17.099891 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.715392 2566 reconstruct.go:97] "Volume reconstruction finished" Jul 12 08:21:17.099891 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.715401 2566 reconciler.go:29] "Reconciler: start to sync state" Jul 12 08:21:17.099891 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.715510 2566 factory.go:55] Registering systemd factory Jul 12 08:21:17.099891 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.715524 2566 factory.go:223] Registration of the systemd container factory successfully Jul 12 08:21:17.099891 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.715767 2566 factory.go:153] Registering CRI-O factory Jul 12 08:21:17.099891 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.715781 2566 factory.go:223] Registration of the crio container factory successfully Jul 12 08:21:17.099891 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.715855 2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 12 08:21:17.099891 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.715886 2566 factory.go:103] Registering Raw factory Jul 12 08:21:17.099891 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.715898 2566 manager.go:1196] Started watching for new ooms in manager Jul 12 08:21:17.099891 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.716562 2566 manager.go:349] Starting recovery of all containers Jul 12 08:21:17.099891 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:16.717094 2566 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-138.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 12 08:21:17.099891 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:16.717265 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 12 08:21:17.099891 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.726928 2566 manager.go:354] Recovery completed Jul 12 08:21:17.099891 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.730830 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 08:21:17.099891 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.733413 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-138.ec2.internal" event="NodeHasSufficientMemory" Jul 12 08:21:17.099891 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.733450 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-138.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 08:21:17.099891 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.733466 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-138.ec2.internal" event="NodeHasSufficientPID" Jul 12 08:21:17.099891 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.736892 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wn5qd" Jul 12 08:21:17.099891 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.737087 2566 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 12 08:21:17.099891 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.737098 2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 12 08:21:17.099891 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.737113 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 12 08:21:17.101169 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:16.737066 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-138.ec2.internal.18c17d19b23d1a4e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-138.ec2.internal,UID:ip-10-0-136-138.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-138.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-138.ec2.internal,},FirstTimestamp:2026-07-12 08:21:16.73343035 +0000 UTC m=+0.394226007,LastTimestamp:2026-07-12 08:21:16.73343035 +0000 UTC m=+0.394226007,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-138.ec2.internal,}" Jul 12 08:21:17.101169 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.737088 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 08:21:17.101169 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.737939 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-138.ec2.internal" event="NodeHasSufficientMemory" Jul 12 08:21:17.101169 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.737963 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-138.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 08:21:17.101169 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.737976 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-138.ec2.internal" event="NodeHasSufficientPID" Jul 12 08:21:17.101169 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.740884 2566 policy_none.go:49] "None policy: Start" Jul 12 08:21:17.101169 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.740900 2566 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 12 08:21:17.101169 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.740912 2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 12 08:21:17.101169 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.742110 2566 policy_none.go:47] "Start" Jul 12 08:21:17.101169 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.745037 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wn5qd" Jul 12 08:21:17.101169 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.805055 2566 manager.go:337] "Starting Device Plugin manager" Jul 12 08:21:17.101169 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:16.805205 2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 12 08:21:17.101169 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.805220 2566 server.go:87] "Starting device plugin registration server" Jul 12 08:21:17.101169 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.805496 2566 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 12 08:21:17.101169 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.805509 2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 12 08:21:17.101169 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.805596 2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 12 08:21:17.101169 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.805693 2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 12 08:21:17.101169 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.805702 2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 12 08:21:17.101169 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:16.806179 2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 12 08:21:17.101169 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:16.806223 2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-138.ec2.internal\" not found" Jul 12 08:21:17.101169 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.818924 2566 container_manager_linux.go:676] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2600 into the system container \"/system.slice\": " Jul 12 08:21:17.101169 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.906352 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 08:21:17.101169 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.907772 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-138.ec2.internal" event="NodeHasSufficientMemory" Jul 12 08:21:17.101169 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.907806 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-138.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 08:21:17.101169 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.907819 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-138.ec2.internal" event="NodeHasSufficientPID" Jul 12 08:21:17.101169 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.907851 2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-138.ec2.internal" Jul 12 08:21:17.101169 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:16.916179 2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-138.ec2.internal" Jul 12 08:21:17.101169 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:16.916196 2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-138.ec2.internal\": node \"ip-10-0-136-138.ec2.internal\" not found" Jul 12 08:21:17.101169 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:16.940277 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-138.ec2.internal\" not found" Jul 12 08:21:17.101169 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.018928 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 12 08:21:17.101169 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.019980 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 12 08:21:17.101169 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.020019 2566 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 12 08:21:17.101169 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.020039 2566 kubelet.go:2448] "Starting kubelet main sync loop" Jul 12 08:21:17.102618 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:17.020071 2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 12 08:21:17.102618 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.022225 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 08:21:17.102618 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:17.041171 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-138.ec2.internal\" not found" Jul 12 08:21:17.120612 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.120594 2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-138.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-138.ec2.internal"] Jul 12 08:21:17.120678 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.120660 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 08:21:17.121518 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.121503 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-138.ec2.internal" event="NodeHasSufficientMemory" Jul 12 08:21:17.121602 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.121533 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-138.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 08:21:17.121602 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.121543 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-138.ec2.internal" event="NodeHasSufficientPID" Jul 12 08:21:17.122748 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.122736 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 08:21:17.122854 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.122841 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-138.ec2.internal" Jul 12 08:21:17.122891 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.122871 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 08:21:17.123435 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.123420 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-138.ec2.internal" event="NodeHasSufficientMemory" Jul 12 08:21:17.123513 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.123423 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-138.ec2.internal" event="NodeHasSufficientMemory" Jul 12 08:21:17.123513 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.123465 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-138.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 08:21:17.123513 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.123476 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-138.ec2.internal" event="NodeHasSufficientPID" Jul 12 08:21:17.123513 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.123448 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-138.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 08:21:17.123660 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.123522 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-138.ec2.internal" event="NodeHasSufficientPID" Jul 12 08:21:17.124766 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.124753 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-138.ec2.internal" Jul 12 08:21:17.124800 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.124777 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 08:21:17.125680 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.125667 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-138.ec2.internal" event="NodeHasSufficientMemory" Jul 12 08:21:17.125753 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.125695 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-138.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 08:21:17.125753 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.125709 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-138.ec2.internal" event="NodeHasSufficientPID" Jul 12 08:21:17.141766 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:17.141748 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-138.ec2.internal\" not found" Jul 12 08:21:17.149102 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:17.149089 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-138.ec2.internal\" not found" node="ip-10-0-136-138.ec2.internal" Jul 12 08:21:17.153384 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:17.153369 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-138.ec2.internal\" not found" node="ip-10-0-136-138.ec2.internal" Jul 12 08:21:17.217988 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.217942 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4f7c17a66495e6fef40567703d162ecc-config\") pod \"kube-apiserver-proxy-ip-10-0-136-138.ec2.internal\" (UID: \"4f7c17a66495e6fef40567703d162ecc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-138.ec2.internal" Jul 12 08:21:17.217988 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.217966 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/73eb6c7cf5825b2f7fddf7b8ec918121-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-138.ec2.internal\" (UID: \"73eb6c7cf5825b2f7fddf7b8ec918121\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-138.ec2.internal" Jul 12 08:21:17.217988 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.217981 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/73eb6c7cf5825b2f7fddf7b8ec918121-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-138.ec2.internal\" (UID: \"73eb6c7cf5825b2f7fddf7b8ec918121\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-138.ec2.internal" Jul 12 08:21:17.242038 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:17.242019 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-138.ec2.internal\" not found" Jul 12 08:21:17.318432 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.318410 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4f7c17a66495e6fef40567703d162ecc-config\") pod \"kube-apiserver-proxy-ip-10-0-136-138.ec2.internal\" (UID: \"4f7c17a66495e6fef40567703d162ecc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-138.ec2.internal" Jul 12 08:21:17.318493 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.318435 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/73eb6c7cf5825b2f7fddf7b8ec918121-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-138.ec2.internal\" (UID: \"73eb6c7cf5825b2f7fddf7b8ec918121\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-138.ec2.internal" Jul 12 08:21:17.318493 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.318452 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/73eb6c7cf5825b2f7fddf7b8ec918121-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-138.ec2.internal\" (UID: \"73eb6c7cf5825b2f7fddf7b8ec918121\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-138.ec2.internal" Jul 12 08:21:17.318582 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.318499 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4f7c17a66495e6fef40567703d162ecc-config\") pod \"kube-apiserver-proxy-ip-10-0-136-138.ec2.internal\" (UID: \"4f7c17a66495e6fef40567703d162ecc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-138.ec2.internal" Jul 12 08:21:17.318582 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.318496 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/73eb6c7cf5825b2f7fddf7b8ec918121-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-138.ec2.internal\" (UID: \"73eb6c7cf5825b2f7fddf7b8ec918121\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-138.ec2.internal" Jul 12 08:21:17.318582 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.318517 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/73eb6c7cf5825b2f7fddf7b8ec918121-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-138.ec2.internal\" (UID: \"73eb6c7cf5825b2f7fddf7b8ec918121\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-138.ec2.internal" Jul 12 08:21:17.342528 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:17.342514 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-138.ec2.internal\" not found" Jul 12 08:21:17.442993 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:17.442965 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-138.ec2.internal\" not found" Jul 12 08:21:17.453126 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.453112 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-138.ec2.internal" Jul 12 08:21:17.456807 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.456715 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-138.ec2.internal" Jul 12 08:21:17.543655 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:17.543593 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-138.ec2.internal\" not found" Jul 12 08:21:17.549574 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.549543 2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 08:21:17.606100 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.606079 2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 08:21:17.645302 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:17.645276 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-138.ec2.internal\" not found" Jul 12 08:21:17.656003 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.655981 2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 12 08:21:17.656166 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.656145 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 12 08:21:17.656166 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.656153 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 12 08:21:17.656252 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.656161 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 12 08:21:17.745419 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:17.745389 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-138.ec2.internal\" not found" Jul 12 08:21:17.755319 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.755285 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-11 08:16:16 +0000 UTC" deadline="2028-04-14 18:38:40.739568634 +0000 UTC" Jul 12 08:21:17.755319 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.755317 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15418h17m22.984254645s" Jul 12 08:21:17.846532 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:17.846466 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-138.ec2.internal\" not found" Jul 12 08:21:17.947438 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:17.947403 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-138.ec2.internal\" not found" Jul 12 08:21:17.975393 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:17.975369 2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 08:21:18.015657 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.015625 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-138.ec2.internal" Jul 12 08:21:18.049254 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:18.049223 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73eb6c7cf5825b2f7fddf7b8ec918121.slice/crio-f07831ef3b4d1c73e5a1dc8697905df39f7a2a45493357a16d91a7940c5596bf WatchSource:0}: Error finding container f07831ef3b4d1c73e5a1dc8697905df39f7a2a45493357a16d91a7940c5596bf: Status 404 returned error can't find the container with id f07831ef3b4d1c73e5a1dc8697905df39f7a2a45493357a16d91a7940c5596bf Jul 12 08:21:18.053712 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.053697 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 08:21:18.058620 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:18.058599 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f7c17a66495e6fef40567703d162ecc.slice/crio-97906fee2d083db1aa06b88e1daa8626ba063c93e01c993d8caf6c6cfd73fa00 WatchSource:0}: Error finding container 97906fee2d083db1aa06b88e1daa8626ba063c93e01c993d8caf6c6cfd73fa00: Status 404 returned error can't find the container with id 97906fee2d083db1aa06b88e1daa8626ba063c93e01c993d8caf6c6cfd73fa00 Jul 12 08:21:18.100283 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.100233 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 12 08:21:18.101297 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.101284 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-138.ec2.internal" Jul 12 08:21:18.107715 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.107700 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 12 08:21:18.696555 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.696526 2566 apiserver.go:52] "Watching apiserver" Jul 12 08:21:18.706361 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.706328 2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 12 08:21:18.706850 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.706792 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-lwln2","openshift-image-registry/node-ca-pzgtb","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-138.ec2.internal","openshift-multus/multus-pfjqv","openshift-network-diagnostics/network-check-target-hr58s","openshift-network-operator/iptables-alerter-kk4zw","kube-system/global-pull-secret-syncer-w4cfw","kube-system/kube-apiserver-proxy-ip-10-0-136-138.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c8ncb","openshift-multus/multus-additional-cni-plugins-xvhw4","openshift-multus/network-metrics-daemon-9ms85","openshift-ovn-kubernetes/ovnkube-node-lfbsq","kube-system/konnectivity-agent-8mwdg","openshift-cluster-node-tuning-operator/tuned-j8z49"] Jul 12 08:21:18.709869 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.709848 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:21:18.709979 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:18.709927 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4cfw" podUID="0f993375-bf43-40a2-b8c6-558a97dcba75" Jul 12 08:21:18.712215 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.712183 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8mwdg" Jul 12 08:21:18.716348 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.714742 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.716348 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.714950 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 12 08:21:18.716348 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.715412 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 12 08:21:18.716348 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.715555 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-npj98\"" Jul 12 08:21:18.716348 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.715717 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 12 08:21:18.717267 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.716891 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 12 08:21:18.717378 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.717327 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.717971 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.717583 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 12 08:21:18.717971 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.717821 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ktflc\"" Jul 12 08:21:18.719893 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.719871 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hr58s" Jul 12 08:21:18.719988 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:18.719932 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hr58s" podUID="f66d4032-1ab2-4d52-874b-a7d0296525ee" Jul 12 08:21:18.723829 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.722885 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kk4zw" Jul 12 08:21:18.723829 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.723498 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 12 08:21:18.723829 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.723599 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 12 08:21:18.723829 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.723629 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 12 08:21:18.723829 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.723721 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-mxc8n\"" Jul 12 08:21:18.723829 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.723755 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 12 08:21:18.726323 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.724439 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3dcfbd82-b903-426b-9573-7e68317111b0-cni-binary-copy\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.726323 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.724620 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c8ncb" Jul 12 08:21:18.726323 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.724782 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-multus-socket-dir-parent\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.726323 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.724854 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-host-run-netns\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.726323 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.724879 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3dcfbd82-b903-426b-9573-7e68317111b0-multus-daemon-config\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.726323 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.724904 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/37e6726a-f4aa-431c-8d02-961f53c45038-agent-certs\") pod \"konnectivity-agent-8mwdg\" (UID: \"37e6726a-f4aa-431c-8d02-961f53c45038\") " pod="kube-system/konnectivity-agent-8mwdg" Jul 12 08:21:18.726323 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.724928 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/869193b6-0a98-421f-be76-cd6ca5de6d75-etc-sysconfig\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.726323 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.724952 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/869193b6-0a98-421f-be76-cd6ca5de6d75-etc-systemd\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.726323 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.724976 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/869193b6-0a98-421f-be76-cd6ca5de6d75-tmp\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.726323 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.725000 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dnqm\" (UniqueName: \"kubernetes.io/projected/3dcfbd82-b903-426b-9573-7e68317111b0-kube-api-access-2dnqm\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.726323 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.725023 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/869193b6-0a98-421f-be76-cd6ca5de6d75-run\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.726323 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.725046 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-host-run-multus-certs\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.726323 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.725070 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/869193b6-0a98-421f-be76-cd6ca5de6d75-etc-sysctl-conf\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.726323 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.725092 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/869193b6-0a98-421f-be76-cd6ca5de6d75-etc-tuned\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.726323 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.725117 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-host-run-k8s-cni-cncf-io\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.727306 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.725143 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-host-var-lib-cni-bin\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.727306 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.725166 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-host-var-lib-kubelet\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.727306 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.725188 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-hostroot\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.727306 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.725231 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-system-cni-dir\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.727306 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.725257 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-cnibin\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.727306 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.725281 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/869193b6-0a98-421f-be76-cd6ca5de6d75-etc-modprobe-d\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.727306 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.725313 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/869193b6-0a98-421f-be76-cd6ca5de6d75-host\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.727306 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.725341 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f993375-bf43-40a2-b8c6-558a97dcba75-original-pull-secret\") pod \"global-pull-secret-syncer-w4cfw\" (UID: \"0f993375-bf43-40a2-b8c6-558a97dcba75\") " pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:21:18.727306 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.725365 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/869193b6-0a98-421f-be76-cd6ca5de6d75-etc-sysctl-d\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.727306 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.725400 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/869193b6-0a98-421f-be76-cd6ca5de6d75-sys\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.727306 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.725424 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-os-release\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.727306 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.725448 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-host-var-lib-cni-multus\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.727306 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.725475 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0f993375-bf43-40a2-b8c6-558a97dcba75-kubelet-config\") pod \"global-pull-secret-syncer-w4cfw\" (UID: \"0f993375-bf43-40a2-b8c6-558a97dcba75\") " pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:21:18.727306 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.725509 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0f993375-bf43-40a2-b8c6-558a97dcba75-dbus\") pod \"global-pull-secret-syncer-w4cfw\" (UID: \"0f993375-bf43-40a2-b8c6-558a97dcba75\") " pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:21:18.728107 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.725542 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/37e6726a-f4aa-431c-8d02-961f53c45038-konnectivity-ca\") pod \"konnectivity-agent-8mwdg\" (UID: \"37e6726a-f4aa-431c-8d02-961f53c45038\") " pod="kube-system/konnectivity-agent-8mwdg" Jul 12 08:21:18.728107 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.725567 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/869193b6-0a98-421f-be76-cd6ca5de6d75-etc-kubernetes\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.728107 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.725591 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/869193b6-0a98-421f-be76-cd6ca5de6d75-var-lib-kubelet\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.728107 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.725615 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-multus-cni-dir\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.728107 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.725676 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-multus-conf-dir\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.728107 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.725829 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-etc-kubernetes\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.728107 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.725857 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/869193b6-0a98-421f-be76-cd6ca5de6d75-lib-modules\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.728107 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.725878 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-x62k4\"" Jul 12 08:21:18.728107 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.725901 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fh2b2\" (UniqueName: \"kubernetes.io/projected/869193b6-0a98-421f-be76-cd6ca5de6d75-kube-api-access-fh2b2\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.728107 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.726038 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 12 08:21:18.728107 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.726136 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 12 08:21:18.728107 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.726218 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 12 08:21:18.728107 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.727905 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lwln2" Jul 12 08:21:18.729062 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.729022 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 12 08:21:18.729684 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.729397 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sdhbf\"" Jul 12 08:21:18.729684 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.729646 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 12 08:21:18.729865 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.729847 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 12 08:21:18.730022 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.730000 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 12 08:21:18.730495 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.729851 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 12 08:21:18.730495 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.730274 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wvnp5\"" Jul 12 08:21:18.730495 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.730361 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 12 08:21:18.730495 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.730453 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pzgtb" Jul 12 08:21:18.732766 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.732036 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xvhw4" Jul 12 08:21:18.733819 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.733518 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 12 08:21:18.733819 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.733551 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 12 08:21:18.733819 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.733585 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 12 08:21:18.733819 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.733583 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5vz84\"" Jul 12 08:21:18.734465 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.734445 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 12 08:21:18.734646 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.734603 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 12 08:21:18.734727 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.734670 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6dqx5\"" Jul 12 08:21:18.735410 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.735381 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ms85" Jul 12 08:21:18.735499 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:18.735447 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ms85" podUID="508b3df5-2cbe-4cd6-a8fd-e6cb29188184" Jul 12 08:21:18.738901 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.738829 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.741012 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.740995 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 12 08:21:18.741291 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.741274 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 12 08:21:18.741453 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.741433 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 12 08:21:18.741536 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.741453 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mxl4z\"" Jul 12 08:21:18.741733 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.741707 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 12 08:21:18.742094 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.741991 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 12 08:21:18.742094 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.742026 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 12 08:21:18.754400 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.754372 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-cnjrd" Jul 12 08:21:18.762035 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.762009 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cnjrd" Jul 12 08:21:18.817018 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.816990 2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 12 08:21:18.826963 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.826691 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/869193b6-0a98-421f-be76-cd6ca5de6d75-etc-tuned\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.826963 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.826741 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6b580b23-6e3f-47b0-bcf9-2aca08bc5578-socket-dir\") pod \"aws-ebs-csi-driver-node-c8ncb\" (UID: \"6b580b23-6e3f-47b0-bcf9-2aca08bc5578\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c8ncb" Jul 12 08:21:18.826963 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.826771 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6b580b23-6e3f-47b0-bcf9-2aca08bc5578-etc-selinux\") pod \"aws-ebs-csi-driver-node-c8ncb\" (UID: \"6b580b23-6e3f-47b0-bcf9-2aca08bc5578\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c8ncb" Jul 12 08:21:18.826963 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.826796 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-host-kubelet\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.826963 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.826820 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-var-lib-openvswitch\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.826963 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.826848 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-host-run-k8s-cni-cncf-io\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.826963 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.826872 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-host-var-lib-cni-bin\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.826963 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.826895 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-hostroot\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.826963 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.826920 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ccef3090-87ec-4e72-a24c-5dd65f19c55b-hosts-file\") pod \"node-resolver-lwln2\" (UID: \"ccef3090-87ec-4e72-a24c-5dd65f19c55b\") " pod="openshift-dns/node-resolver-lwln2" Jul 12 08:21:18.826963 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.826944 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6cd77bb0-5bbe-4d49-9e22-05dad8c650af-host\") pod \"node-ca-pzgtb\" (UID: \"6cd77bb0-5bbe-4d49-9e22-05dad8c650af\") " pod="openshift-image-registry/node-ca-pzgtb" Jul 12 08:21:18.826963 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.826970 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a69a78ba-31f4-488d-a7a5-0d2a197ef775-cnibin\") pod \"multus-additional-cni-plugins-xvhw4\" (UID: \"a69a78ba-31f4-488d-a7a5-0d2a197ef775\") " pod="openshift-multus/multus-additional-cni-plugins-xvhw4" Jul 12 08:21:18.827541 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.826995 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a69a78ba-31f4-488d-a7a5-0d2a197ef775-cni-binary-copy\") pod \"multus-additional-cni-plugins-xvhw4\" (UID: \"a69a78ba-31f4-488d-a7a5-0d2a197ef775\") " pod="openshift-multus/multus-additional-cni-plugins-xvhw4" Jul 12 08:21:18.827541 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.827023 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a69a78ba-31f4-488d-a7a5-0d2a197ef775-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xvhw4\" (UID: \"a69a78ba-31f4-488d-a7a5-0d2a197ef775\") " pod="openshift-multus/multus-additional-cni-plugins-xvhw4" Jul 12 08:21:18.827541 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.827047 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-system-cni-dir\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.827541 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.827071 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/869193b6-0a98-421f-be76-cd6ca5de6d75-etc-modprobe-d\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.827541 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.827094 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/869193b6-0a98-421f-be76-cd6ca5de6d75-host\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.827541 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.827119 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c9625c83-f82b-4eec-9e5d-1529523bc741-iptables-alerter-script\") pod \"iptables-alerter-kk4zw\" (UID: \"c9625c83-f82b-4eec-9e5d-1529523bc741\") " pod="openshift-network-operator/iptables-alerter-kk4zw" Jul 12 08:21:18.827541 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.827146 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ccef3090-87ec-4e72-a24c-5dd65f19c55b-tmp-dir\") pod \"node-resolver-lwln2\" (UID: \"ccef3090-87ec-4e72-a24c-5dd65f19c55b\") " pod="openshift-dns/node-resolver-lwln2" Jul 12 08:21:18.827541 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.827168 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-host-slash\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.827541 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.827211 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-host-run-ovn-kubernetes\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.827541 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.827277 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-host-var-lib-cni-bin\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.827541 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.827302 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-hostroot\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.827541 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.827310 2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 12 08:21:18.827541 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.827350 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/869193b6-0a98-421f-be76-cd6ca5de6d75-host\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.827541 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.827350 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/869193b6-0a98-421f-be76-cd6ca5de6d75-etc-modprobe-d\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.827541 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.827366 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-system-cni-dir\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.828262 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.827384 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-host-run-k8s-cni-cncf-io\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.828262 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.827386 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-ovn-node-metrics-cert\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.828262 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.827431 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f6lb\" (UniqueName: \"kubernetes.io/projected/6b580b23-6e3f-47b0-bcf9-2aca08bc5578-kube-api-access-4f6lb\") pod \"aws-ebs-csi-driver-node-c8ncb\" (UID: \"6b580b23-6e3f-47b0-bcf9-2aca08bc5578\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c8ncb" Jul 12 08:21:18.828262 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.827461 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-log-socket\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.828262 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.827522 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0f993375-bf43-40a2-b8c6-558a97dcba75-kubelet-config\") pod \"global-pull-secret-syncer-w4cfw\" (UID: \"0f993375-bf43-40a2-b8c6-558a97dcba75\") " pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:21:18.828262 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.827558 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/869193b6-0a98-421f-be76-cd6ca5de6d75-etc-kubernetes\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.828262 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.827607 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0f993375-bf43-40a2-b8c6-558a97dcba75-kubelet-config\") pod \"global-pull-secret-syncer-w4cfw\" (UID: \"0f993375-bf43-40a2-b8c6-558a97dcba75\") " pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:21:18.828262 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.827653 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a69a78ba-31f4-488d-a7a5-0d2a197ef775-system-cni-dir\") pod \"multus-additional-cni-plugins-xvhw4\" (UID: \"a69a78ba-31f4-488d-a7a5-0d2a197ef775\") " pod="openshift-multus/multus-additional-cni-plugins-xvhw4" Jul 12 08:21:18.828262 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.827682 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a69a78ba-31f4-488d-a7a5-0d2a197ef775-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xvhw4\" (UID: \"a69a78ba-31f4-488d-a7a5-0d2a197ef775\") " pod="openshift-multus/multus-additional-cni-plugins-xvhw4" Jul 12 08:21:18.828262 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.827709 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/869193b6-0a98-421f-be76-cd6ca5de6d75-etc-kubernetes\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.828262 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.827753 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-host-run-netns\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.828262 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.827786 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-etc-openvswitch\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.828262 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.827819 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-node-log\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.828262 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.827849 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-multus-cni-dir\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.828971 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.827866 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-etc-kubernetes\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.828971 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.827903 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fh2b2\" (UniqueName: \"kubernetes.io/projected/869193b6-0a98-421f-be76-cd6ca5de6d75-kube-api-access-fh2b2\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.828971 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.827922 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-etc-kubernetes\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.828971 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.827959 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/508b3df5-2cbe-4cd6-a8fd-e6cb29188184-metrics-certs\") pod \"network-metrics-daemon-9ms85\" (UID: \"508b3df5-2cbe-4cd6-a8fd-e6cb29188184\") " pod="openshift-multus/network-metrics-daemon-9ms85" Jul 12 08:21:18.828971 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.827989 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-env-overrides\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.828971 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828016 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/37e6726a-f4aa-431c-8d02-961f53c45038-agent-certs\") pod \"konnectivity-agent-8mwdg\" (UID: \"37e6726a-f4aa-431c-8d02-961f53c45038\") " pod="kube-system/konnectivity-agent-8mwdg" Jul 12 08:21:18.828971 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828041 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/869193b6-0a98-421f-be76-cd6ca5de6d75-etc-sysconfig\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.828971 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828068 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/869193b6-0a98-421f-be76-cd6ca5de6d75-etc-systemd\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.828971 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828071 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-multus-cni-dir\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.828971 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828092 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a69a78ba-31f4-488d-a7a5-0d2a197ef775-os-release\") pod \"multus-additional-cni-plugins-xvhw4\" (UID: \"a69a78ba-31f4-488d-a7a5-0d2a197ef775\") " pod="openshift-multus/multus-additional-cni-plugins-xvhw4" Jul 12 08:21:18.828971 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828121 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5tt9\" (UniqueName: \"kubernetes.io/projected/508b3df5-2cbe-4cd6-a8fd-e6cb29188184-kube-api-access-p5tt9\") pod \"network-metrics-daemon-9ms85\" (UID: \"508b3df5-2cbe-4cd6-a8fd-e6cb29188184\") " pod="openshift-multus/network-metrics-daemon-9ms85" Jul 12 08:21:18.828971 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828136 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/869193b6-0a98-421f-be76-cd6ca5de6d75-etc-sysconfig\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.828971 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828146 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dnqm\" (UniqueName: \"kubernetes.io/projected/3dcfbd82-b903-426b-9573-7e68317111b0-kube-api-access-2dnqm\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.828971 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828172 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/869193b6-0a98-421f-be76-cd6ca5de6d75-run\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.829683 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828179 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/869193b6-0a98-421f-be76-cd6ca5de6d75-etc-systemd\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.829683 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828196 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-run-ovn\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.829683 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828221 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-host-run-multus-certs\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.829683 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828246 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6b580b23-6e3f-47b0-bcf9-2aca08bc5578-device-dir\") pod \"aws-ebs-csi-driver-node-c8ncb\" (UID: \"6b580b23-6e3f-47b0-bcf9-2aca08bc5578\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c8ncb" Jul 12 08:21:18.829683 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828250 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/869193b6-0a98-421f-be76-cd6ca5de6d75-run\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.829683 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828272 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6b580b23-6e3f-47b0-bcf9-2aca08bc5578-sys-fs\") pod \"aws-ebs-csi-driver-node-c8ncb\" (UID: \"6b580b23-6e3f-47b0-bcf9-2aca08bc5578\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c8ncb" Jul 12 08:21:18.829683 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828299 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-host-run-multus-certs\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.829683 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828298 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6cd77bb0-5bbe-4d49-9e22-05dad8c650af-serviceca\") pod \"node-ca-pzgtb\" (UID: \"6cd77bb0-5bbe-4d49-9e22-05dad8c650af\") " pod="openshift-image-registry/node-ca-pzgtb" Jul 12 08:21:18.829683 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828338 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwzjr\" (UniqueName: \"kubernetes.io/projected/6cd77bb0-5bbe-4d49-9e22-05dad8c650af-kube-api-access-lwzjr\") pod \"node-ca-pzgtb\" (UID: \"6cd77bb0-5bbe-4d49-9e22-05dad8c650af\") " pod="openshift-image-registry/node-ca-pzgtb" Jul 12 08:21:18.829683 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828367 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-host-var-lib-kubelet\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.829683 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828393 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2klr5\" (UniqueName: \"kubernetes.io/projected/a69a78ba-31f4-488d-a7a5-0d2a197ef775-kube-api-access-2klr5\") pod \"multus-additional-cni-plugins-xvhw4\" (UID: \"a69a78ba-31f4-488d-a7a5-0d2a197ef775\") " pod="openshift-multus/multus-additional-cni-plugins-xvhw4" Jul 12 08:21:18.829683 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828446 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-host-var-lib-kubelet\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.829683 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828448 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5k8nw\" (UniqueName: \"kubernetes.io/projected/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-kube-api-access-5k8nw\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.829683 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828493 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-cnibin\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.830405 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828519 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6b580b23-6e3f-47b0-bcf9-2aca08bc5578-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c8ncb\" (UID: \"6b580b23-6e3f-47b0-bcf9-2aca08bc5578\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c8ncb" Jul 12 08:21:18.830405 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828546 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-run-openvswitch\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.830405 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828592 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f993375-bf43-40a2-b8c6-558a97dcba75-original-pull-secret\") pod \"global-pull-secret-syncer-w4cfw\" (UID: \"0f993375-bf43-40a2-b8c6-558a97dcba75\") " pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:21:18.830405 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828634 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/869193b6-0a98-421f-be76-cd6ca5de6d75-etc-sysctl-d\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.830405 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828657 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/869193b6-0a98-421f-be76-cd6ca5de6d75-sys\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.830405 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828607 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-cnibin\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.830405 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:18.828694 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 08:21:18.830405 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828713 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sr7dk\" (UniqueName: \"kubernetes.io/projected/f66d4032-1ab2-4d52-874b-a7d0296525ee-kube-api-access-sr7dk\") pod \"network-check-target-hr58s\" (UID: \"f66d4032-1ab2-4d52-874b-a7d0296525ee\") " pod="openshift-network-diagnostics/network-check-target-hr58s" Jul 12 08:21:18.830405 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828739 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-host-cni-bin\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.830405 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:18.828784 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f993375-bf43-40a2-b8c6-558a97dcba75-original-pull-secret podName:0f993375-bf43-40a2-b8c6-558a97dcba75 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:19.32874821 +0000 UTC m=+2.989543869 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0f993375-bf43-40a2-b8c6-558a97dcba75-original-pull-secret") pod "global-pull-secret-syncer-w4cfw" (UID: "0f993375-bf43-40a2-b8c6-558a97dcba75") : object "kube-system"/"original-pull-secret" not registered Jul 12 08:21:18.830405 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828796 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/869193b6-0a98-421f-be76-cd6ca5de6d75-etc-sysctl-d\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.830405 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828805 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-ovnkube-script-lib\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.830405 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828819 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/869193b6-0a98-421f-be76-cd6ca5de6d75-sys\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.830405 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828836 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-os-release\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.831113 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828877 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-host-var-lib-cni-multus\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.831113 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828899 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0f993375-bf43-40a2-b8c6-558a97dcba75-dbus\") pod \"global-pull-secret-syncer-w4cfw\" (UID: \"0f993375-bf43-40a2-b8c6-558a97dcba75\") " pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:21:18.831113 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828927 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/37e6726a-f4aa-431c-8d02-961f53c45038-konnectivity-ca\") pod \"konnectivity-agent-8mwdg\" (UID: \"37e6726a-f4aa-431c-8d02-961f53c45038\") " pod="kube-system/konnectivity-agent-8mwdg" Jul 12 08:21:18.831113 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828949 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/869193b6-0a98-421f-be76-cd6ca5de6d75-var-lib-kubelet\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.831113 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828973 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-os-release\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.831113 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.828976 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6b580b23-6e3f-47b0-bcf9-2aca08bc5578-registration-dir\") pod \"aws-ebs-csi-driver-node-c8ncb\" (UID: \"6b580b23-6e3f-47b0-bcf9-2aca08bc5578\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c8ncb" Jul 12 08:21:18.831113 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.829016 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a69a78ba-31f4-488d-a7a5-0d2a197ef775-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xvhw4\" (UID: \"a69a78ba-31f4-488d-a7a5-0d2a197ef775\") " pod="openshift-multus/multus-additional-cni-plugins-xvhw4" Jul 12 08:21:18.831113 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.829033 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0f993375-bf43-40a2-b8c6-558a97dcba75-dbus\") pod \"global-pull-secret-syncer-w4cfw\" (UID: \"0f993375-bf43-40a2-b8c6-558a97dcba75\") " pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:21:18.831113 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.829039 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-systemd-units\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.831113 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.829075 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-host-var-lib-cni-multus\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.831113 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.829089 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-multus-conf-dir\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.831113 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.829137 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/869193b6-0a98-421f-be76-cd6ca5de6d75-var-lib-kubelet\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.831113 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.829150 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/869193b6-0a98-421f-be76-cd6ca5de6d75-lib-modules\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.831113 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.829185 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9npxj\" (UniqueName: \"kubernetes.io/projected/ccef3090-87ec-4e72-a24c-5dd65f19c55b-kube-api-access-9npxj\") pod \"node-resolver-lwln2\" (UID: \"ccef3090-87ec-4e72-a24c-5dd65f19c55b\") " pod="openshift-dns/node-resolver-lwln2" Jul 12 08:21:18.831805 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.829214 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-multus-conf-dir\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.831805 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.829222 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-host-cni-netd\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.831805 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.829248 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3dcfbd82-b903-426b-9573-7e68317111b0-cni-binary-copy\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.831805 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.829276 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-multus-socket-dir-parent\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.831805 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.829324 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/869193b6-0a98-421f-be76-cd6ca5de6d75-lib-modules\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.831805 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.829332 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-host-run-netns\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.831805 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.829360 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3dcfbd82-b903-426b-9573-7e68317111b0-multus-daemon-config\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.831805 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.829384 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/869193b6-0a98-421f-be76-cd6ca5de6d75-tmp\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.831805 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.829394 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-multus-socket-dir-parent\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.831805 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.829408 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kccnm\" (UniqueName: \"kubernetes.io/projected/c9625c83-f82b-4eec-9e5d-1529523bc741-kube-api-access-kccnm\") pod \"iptables-alerter-kk4zw\" (UID: \"c9625c83-f82b-4eec-9e5d-1529523bc741\") " pod="openshift-network-operator/iptables-alerter-kk4zw" Jul 12 08:21:18.831805 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.829453 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-run-systemd\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.831805 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.829480 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.831805 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.829507 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c9625c83-f82b-4eec-9e5d-1529523bc741-host-slash\") pod \"iptables-alerter-kk4zw\" (UID: \"c9625c83-f82b-4eec-9e5d-1529523bc741\") " pod="openshift-network-operator/iptables-alerter-kk4zw" Jul 12 08:21:18.831805 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.829539 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-ovnkube-config\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.832491 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.829555 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/37e6726a-f4aa-431c-8d02-961f53c45038-konnectivity-ca\") pod \"konnectivity-agent-8mwdg\" (UID: \"37e6726a-f4aa-431c-8d02-961f53c45038\") " pod="kube-system/konnectivity-agent-8mwdg" Jul 12 08:21:18.832491 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.829566 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/869193b6-0a98-421f-be76-cd6ca5de6d75-etc-sysctl-conf\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.832491 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.829672 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3dcfbd82-b903-426b-9573-7e68317111b0-host-run-netns\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.832491 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.829954 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3dcfbd82-b903-426b-9573-7e68317111b0-cni-binary-copy\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.832491 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.830084 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/869193b6-0a98-421f-be76-cd6ca5de6d75-etc-sysctl-conf\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.832491 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.830122 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3dcfbd82-b903-426b-9573-7e68317111b0-multus-daemon-config\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.833877 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.833848 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/869193b6-0a98-421f-be76-cd6ca5de6d75-tmp\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.833996 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.833888 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/869193b6-0a98-421f-be76-cd6ca5de6d75-etc-tuned\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.834418 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.834391 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/37e6726a-f4aa-431c-8d02-961f53c45038-agent-certs\") pod \"konnectivity-agent-8mwdg\" (UID: \"37e6726a-f4aa-431c-8d02-961f53c45038\") " pod="kube-system/konnectivity-agent-8mwdg" Jul 12 08:21:18.839298 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.839273 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fh2b2\" (UniqueName: \"kubernetes.io/projected/869193b6-0a98-421f-be76-cd6ca5de6d75-kube-api-access-fh2b2\") pod \"tuned-j8z49\" (UID: \"869193b6-0a98-421f-be76-cd6ca5de6d75\") " pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:18.842597 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.842502 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dnqm\" (UniqueName: \"kubernetes.io/projected/3dcfbd82-b903-426b-9573-7e68317111b0-kube-api-access-2dnqm\") pod \"multus-pfjqv\" (UID: \"3dcfbd82-b903-426b-9573-7e68317111b0\") " pod="openshift-multus/multus-pfjqv" Jul 12 08:21:18.929855 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.929817 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6b580b23-6e3f-47b0-bcf9-2aca08bc5578-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c8ncb\" (UID: \"6b580b23-6e3f-47b0-bcf9-2aca08bc5578\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c8ncb" Jul 12 08:21:18.930001 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.929867 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-run-openvswitch\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.930001 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.929920 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr7dk\" (UniqueName: \"kubernetes.io/projected/f66d4032-1ab2-4d52-874b-a7d0296525ee-kube-api-access-sr7dk\") pod \"network-check-target-hr58s\" (UID: \"f66d4032-1ab2-4d52-874b-a7d0296525ee\") " pod="openshift-network-diagnostics/network-check-target-hr58s" Jul 12 08:21:18.930001 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.929970 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6b580b23-6e3f-47b0-bcf9-2aca08bc5578-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c8ncb\" (UID: \"6b580b23-6e3f-47b0-bcf9-2aca08bc5578\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c8ncb" Jul 12 08:21:18.930190 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.929977 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-host-cni-bin\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.930190 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.930042 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-host-cni-bin\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.930190 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.930043 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-ovnkube-script-lib\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.930190 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.930080 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6b580b23-6e3f-47b0-bcf9-2aca08bc5578-registration-dir\") pod \"aws-ebs-csi-driver-node-c8ncb\" (UID: \"6b580b23-6e3f-47b0-bcf9-2aca08bc5578\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c8ncb" Jul 12 08:21:18.930190 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.930105 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a69a78ba-31f4-488d-a7a5-0d2a197ef775-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xvhw4\" (UID: \"a69a78ba-31f4-488d-a7a5-0d2a197ef775\") " pod="openshift-multus/multus-additional-cni-plugins-xvhw4" Jul 12 08:21:18.930190 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.930131 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-systemd-units\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.930190 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.930157 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9npxj\" (UniqueName: \"kubernetes.io/projected/ccef3090-87ec-4e72-a24c-5dd65f19c55b-kube-api-access-9npxj\") pod \"node-resolver-lwln2\" (UID: \"ccef3090-87ec-4e72-a24c-5dd65f19c55b\") " pod="openshift-dns/node-resolver-lwln2" Jul 12 08:21:18.930190 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.930181 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-host-cni-netd\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.930542 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.930210 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kccnm\" (UniqueName: \"kubernetes.io/projected/c9625c83-f82b-4eec-9e5d-1529523bc741-kube-api-access-kccnm\") pod \"iptables-alerter-kk4zw\" (UID: \"c9625c83-f82b-4eec-9e5d-1529523bc741\") " pod="openshift-network-operator/iptables-alerter-kk4zw" Jul 12 08:21:18.930542 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.930237 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-run-systemd\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.930542 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.930263 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.930542 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.930289 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c9625c83-f82b-4eec-9e5d-1529523bc741-host-slash\") pod \"iptables-alerter-kk4zw\" (UID: \"c9625c83-f82b-4eec-9e5d-1529523bc741\") " pod="openshift-network-operator/iptables-alerter-kk4zw" Jul 12 08:21:18.930542 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.930321 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-ovnkube-config\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.930542 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.930346 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6b580b23-6e3f-47b0-bcf9-2aca08bc5578-socket-dir\") pod \"aws-ebs-csi-driver-node-c8ncb\" (UID: \"6b580b23-6e3f-47b0-bcf9-2aca08bc5578\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c8ncb" Jul 12 08:21:18.930542 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.930371 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6b580b23-6e3f-47b0-bcf9-2aca08bc5578-etc-selinux\") pod \"aws-ebs-csi-driver-node-c8ncb\" (UID: \"6b580b23-6e3f-47b0-bcf9-2aca08bc5578\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c8ncb" Jul 12 08:21:18.930542 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.930397 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-host-kubelet\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.930542 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.930447 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-host-kubelet\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.930542 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.930494 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-run-systemd\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.930542 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.930537 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.931132 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.930579 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c9625c83-f82b-4eec-9e5d-1529523bc741-host-slash\") pod \"iptables-alerter-kk4zw\" (UID: \"c9625c83-f82b-4eec-9e5d-1529523bc741\") " pod="openshift-network-operator/iptables-alerter-kk4zw" Jul 12 08:21:18.931132 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.930595 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-ovnkube-script-lib\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.931132 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.930706 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-run-openvswitch\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.931132 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.930781 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6b580b23-6e3f-47b0-bcf9-2aca08bc5578-registration-dir\") pod \"aws-ebs-csi-driver-node-c8ncb\" (UID: \"6b580b23-6e3f-47b0-bcf9-2aca08bc5578\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c8ncb" Jul 12 08:21:18.931132 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.930874 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6b580b23-6e3f-47b0-bcf9-2aca08bc5578-socket-dir\") pod \"aws-ebs-csi-driver-node-c8ncb\" (UID: \"6b580b23-6e3f-47b0-bcf9-2aca08bc5578\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c8ncb" Jul 12 08:21:18.931132 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.930970 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6b580b23-6e3f-47b0-bcf9-2aca08bc5578-etc-selinux\") pod \"aws-ebs-csi-driver-node-c8ncb\" (UID: \"6b580b23-6e3f-47b0-bcf9-2aca08bc5578\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c8ncb" Jul 12 08:21:18.931132 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931017 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-systemd-units\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.931132 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931056 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-host-cni-netd\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.931132 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931087 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-var-lib-openvswitch\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.931132 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931093 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-ovnkube-config\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.931132 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931117 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ccef3090-87ec-4e72-a24c-5dd65f19c55b-hosts-file\") pod \"node-resolver-lwln2\" (UID: \"ccef3090-87ec-4e72-a24c-5dd65f19c55b\") " pod="openshift-dns/node-resolver-lwln2" Jul 12 08:21:18.931781 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931142 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6cd77bb0-5bbe-4d49-9e22-05dad8c650af-host\") pod \"node-ca-pzgtb\" (UID: \"6cd77bb0-5bbe-4d49-9e22-05dad8c650af\") " pod="openshift-image-registry/node-ca-pzgtb" Jul 12 08:21:18.931781 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931151 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-var-lib-openvswitch\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.931781 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931168 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a69a78ba-31f4-488d-a7a5-0d2a197ef775-cnibin\") pod \"multus-additional-cni-plugins-xvhw4\" (UID: \"a69a78ba-31f4-488d-a7a5-0d2a197ef775\") " pod="openshift-multus/multus-additional-cni-plugins-xvhw4" Jul 12 08:21:18.931781 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931194 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a69a78ba-31f4-488d-a7a5-0d2a197ef775-cni-binary-copy\") pod \"multus-additional-cni-plugins-xvhw4\" (UID: \"a69a78ba-31f4-488d-a7a5-0d2a197ef775\") " pod="openshift-multus/multus-additional-cni-plugins-xvhw4" Jul 12 08:21:18.931781 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931201 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ccef3090-87ec-4e72-a24c-5dd65f19c55b-hosts-file\") pod \"node-resolver-lwln2\" (UID: \"ccef3090-87ec-4e72-a24c-5dd65f19c55b\") " pod="openshift-dns/node-resolver-lwln2" Jul 12 08:21:18.931781 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931202 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6cd77bb0-5bbe-4d49-9e22-05dad8c650af-host\") pod \"node-ca-pzgtb\" (UID: \"6cd77bb0-5bbe-4d49-9e22-05dad8c650af\") " pod="openshift-image-registry/node-ca-pzgtb" Jul 12 08:21:18.931781 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931219 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a69a78ba-31f4-488d-a7a5-0d2a197ef775-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xvhw4\" (UID: \"a69a78ba-31f4-488d-a7a5-0d2a197ef775\") " pod="openshift-multus/multus-additional-cni-plugins-xvhw4" Jul 12 08:21:18.931781 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931239 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a69a78ba-31f4-488d-a7a5-0d2a197ef775-cnibin\") pod \"multus-additional-cni-plugins-xvhw4\" (UID: \"a69a78ba-31f4-488d-a7a5-0d2a197ef775\") " pod="openshift-multus/multus-additional-cni-plugins-xvhw4" Jul 12 08:21:18.931781 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931247 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c9625c83-f82b-4eec-9e5d-1529523bc741-iptables-alerter-script\") pod \"iptables-alerter-kk4zw\" (UID: \"c9625c83-f82b-4eec-9e5d-1529523bc741\") " pod="openshift-network-operator/iptables-alerter-kk4zw" Jul 12 08:21:18.931781 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931274 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ccef3090-87ec-4e72-a24c-5dd65f19c55b-tmp-dir\") pod \"node-resolver-lwln2\" (UID: \"ccef3090-87ec-4e72-a24c-5dd65f19c55b\") " pod="openshift-dns/node-resolver-lwln2" Jul 12 08:21:18.931781 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931298 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-host-slash\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.931781 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931322 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-host-run-ovn-kubernetes\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.931781 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931348 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-ovn-node-metrics-cert\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.931781 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931375 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f6lb\" (UniqueName: \"kubernetes.io/projected/6b580b23-6e3f-47b0-bcf9-2aca08bc5578-kube-api-access-4f6lb\") pod \"aws-ebs-csi-driver-node-c8ncb\" (UID: \"6b580b23-6e3f-47b0-bcf9-2aca08bc5578\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c8ncb" Jul 12 08:21:18.932604 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931377 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a69a78ba-31f4-488d-a7a5-0d2a197ef775-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xvhw4\" (UID: \"a69a78ba-31f4-488d-a7a5-0d2a197ef775\") " pod="openshift-multus/multus-additional-cni-plugins-xvhw4" Jul 12 08:21:18.932604 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931400 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-log-socket\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.932604 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931421 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-host-run-ovn-kubernetes\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.932604 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931428 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a69a78ba-31f4-488d-a7a5-0d2a197ef775-system-cni-dir\") pod \"multus-additional-cni-plugins-xvhw4\" (UID: \"a69a78ba-31f4-488d-a7a5-0d2a197ef775\") " pod="openshift-multus/multus-additional-cni-plugins-xvhw4" Jul 12 08:21:18.932604 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931472 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a69a78ba-31f4-488d-a7a5-0d2a197ef775-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xvhw4\" (UID: \"a69a78ba-31f4-488d-a7a5-0d2a197ef775\") " pod="openshift-multus/multus-additional-cni-plugins-xvhw4" Jul 12 08:21:18.932604 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931498 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-host-run-netns\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.932604 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931522 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-etc-openvswitch\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.932604 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931546 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-node-log\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.932604 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931573 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/508b3df5-2cbe-4cd6-a8fd-e6cb29188184-metrics-certs\") pod \"network-metrics-daemon-9ms85\" (UID: \"508b3df5-2cbe-4cd6-a8fd-e6cb29188184\") " pod="openshift-multus/network-metrics-daemon-9ms85" Jul 12 08:21:18.932604 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931596 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-env-overrides\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.932604 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931641 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a69a78ba-31f4-488d-a7a5-0d2a197ef775-os-release\") pod \"multus-additional-cni-plugins-xvhw4\" (UID: \"a69a78ba-31f4-488d-a7a5-0d2a197ef775\") " pod="openshift-multus/multus-additional-cni-plugins-xvhw4" Jul 12 08:21:18.932604 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931670 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5tt9\" (UniqueName: \"kubernetes.io/projected/508b3df5-2cbe-4cd6-a8fd-e6cb29188184-kube-api-access-p5tt9\") pod \"network-metrics-daemon-9ms85\" (UID: \"508b3df5-2cbe-4cd6-a8fd-e6cb29188184\") " pod="openshift-multus/network-metrics-daemon-9ms85" Jul 12 08:21:18.932604 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931696 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-run-ovn\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.932604 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931723 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6b580b23-6e3f-47b0-bcf9-2aca08bc5578-device-dir\") pod \"aws-ebs-csi-driver-node-c8ncb\" (UID: \"6b580b23-6e3f-47b0-bcf9-2aca08bc5578\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c8ncb" Jul 12 08:21:18.933314 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931748 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6b580b23-6e3f-47b0-bcf9-2aca08bc5578-sys-fs\") pod \"aws-ebs-csi-driver-node-c8ncb\" (UID: \"6b580b23-6e3f-47b0-bcf9-2aca08bc5578\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c8ncb" Jul 12 08:21:18.933314 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931755 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a69a78ba-31f4-488d-a7a5-0d2a197ef775-cni-binary-copy\") pod \"multus-additional-cni-plugins-xvhw4\" (UID: \"a69a78ba-31f4-488d-a7a5-0d2a197ef775\") " pod="openshift-multus/multus-additional-cni-plugins-xvhw4" Jul 12 08:21:18.933314 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931776 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6cd77bb0-5bbe-4d49-9e22-05dad8c650af-serviceca\") pod \"node-ca-pzgtb\" (UID: \"6cd77bb0-5bbe-4d49-9e22-05dad8c650af\") " pod="openshift-image-registry/node-ca-pzgtb" Jul 12 08:21:18.933314 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931803 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwzjr\" (UniqueName: \"kubernetes.io/projected/6cd77bb0-5bbe-4d49-9e22-05dad8c650af-kube-api-access-lwzjr\") pod \"node-ca-pzgtb\" (UID: \"6cd77bb0-5bbe-4d49-9e22-05dad8c650af\") " pod="openshift-image-registry/node-ca-pzgtb" Jul 12 08:21:18.933314 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931824 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-host-slash\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.933314 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931829 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2klr5\" (UniqueName: \"kubernetes.io/projected/a69a78ba-31f4-488d-a7a5-0d2a197ef775-kube-api-access-2klr5\") pod \"multus-additional-cni-plugins-xvhw4\" (UID: \"a69a78ba-31f4-488d-a7a5-0d2a197ef775\") " pod="openshift-multus/multus-additional-cni-plugins-xvhw4" Jul 12 08:21:18.933314 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931872 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5k8nw\" (UniqueName: \"kubernetes.io/projected/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-kube-api-access-5k8nw\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.933314 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.931865 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c9625c83-f82b-4eec-9e5d-1529523bc741-iptables-alerter-script\") pod \"iptables-alerter-kk4zw\" (UID: \"c9625c83-f82b-4eec-9e5d-1529523bc741\") " pod="openshift-network-operator/iptables-alerter-kk4zw" Jul 12 08:21:18.933314 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:18.931966 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 08:21:18.933314 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:18.932024 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/508b3df5-2cbe-4cd6-a8fd-e6cb29188184-metrics-certs podName:508b3df5-2cbe-4cd6-a8fd-e6cb29188184 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:19.432006346 +0000 UTC m=+3.092801991 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/508b3df5-2cbe-4cd6-a8fd-e6cb29188184-metrics-certs") pod "network-metrics-daemon-9ms85" (UID: "508b3df5-2cbe-4cd6-a8fd-e6cb29188184") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 08:21:18.933314 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.932118 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a69a78ba-31f4-488d-a7a5-0d2a197ef775-os-release\") pod \"multus-additional-cni-plugins-xvhw4\" (UID: \"a69a78ba-31f4-488d-a7a5-0d2a197ef775\") " pod="openshift-multus/multus-additional-cni-plugins-xvhw4" Jul 12 08:21:18.933314 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.932164 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6b580b23-6e3f-47b0-bcf9-2aca08bc5578-sys-fs\") pod \"aws-ebs-csi-driver-node-c8ncb\" (UID: \"6b580b23-6e3f-47b0-bcf9-2aca08bc5578\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c8ncb" Jul 12 08:21:18.933314 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.932235 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6b580b23-6e3f-47b0-bcf9-2aca08bc5578-device-dir\") pod \"aws-ebs-csi-driver-node-c8ncb\" (UID: \"6b580b23-6e3f-47b0-bcf9-2aca08bc5578\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c8ncb" Jul 12 08:21:18.933314 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.932361 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-log-socket\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.934064 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.932407 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ccef3090-87ec-4e72-a24c-5dd65f19c55b-tmp-dir\") pod \"node-resolver-lwln2\" (UID: \"ccef3090-87ec-4e72-a24c-5dd65f19c55b\") " pod="openshift-dns/node-resolver-lwln2" Jul 12 08:21:18.934064 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.932417 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a69a78ba-31f4-488d-a7a5-0d2a197ef775-system-cni-dir\") pod \"multus-additional-cni-plugins-xvhw4\" (UID: \"a69a78ba-31f4-488d-a7a5-0d2a197ef775\") " pod="openshift-multus/multus-additional-cni-plugins-xvhw4" Jul 12 08:21:18.934064 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.932465 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-node-log\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.934064 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.932533 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6cd77bb0-5bbe-4d49-9e22-05dad8c650af-serviceca\") pod \"node-ca-pzgtb\" (UID: \"6cd77bb0-5bbe-4d49-9e22-05dad8c650af\") " pod="openshift-image-registry/node-ca-pzgtb" Jul 12 08:21:18.934064 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.932584 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-run-ovn\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.934064 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.932703 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-host-run-netns\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.934064 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.932758 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-etc-openvswitch\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.934064 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.933011 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a69a78ba-31f4-488d-a7a5-0d2a197ef775-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xvhw4\" (UID: \"a69a78ba-31f4-488d-a7a5-0d2a197ef775\") " pod="openshift-multus/multus-additional-cni-plugins-xvhw4" Jul 12 08:21:18.934064 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.933664 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a69a78ba-31f4-488d-a7a5-0d2a197ef775-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xvhw4\" (UID: \"a69a78ba-31f4-488d-a7a5-0d2a197ef775\") " pod="openshift-multus/multus-additional-cni-plugins-xvhw4" Jul 12 08:21:18.934490 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.934110 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-env-overrides\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.935427 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.935404 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-ovn-node-metrics-cert\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.943430 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:18.943406 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 08:21:18.943430 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:18.943432 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 08:21:18.943595 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:18.943446 2566 projected.go:196] Error preparing data for projected volume kube-api-access-sr7dk for pod openshift-network-diagnostics/network-check-target-hr58s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 08:21:18.943595 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:18.943525 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f66d4032-1ab2-4d52-874b-a7d0296525ee-kube-api-access-sr7dk podName:f66d4032-1ab2-4d52-874b-a7d0296525ee nodeName:}" failed. No retries permitted until 2026-07-12 08:21:19.443507747 +0000 UTC m=+3.104303408 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-sr7dk" (UniqueName: "kubernetes.io/projected/f66d4032-1ab2-4d52-874b-a7d0296525ee-kube-api-access-sr7dk") pod "network-check-target-hr58s" (UID: "f66d4032-1ab2-4d52-874b-a7d0296525ee") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 08:21:18.944867 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.944841 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2klr5\" (UniqueName: \"kubernetes.io/projected/a69a78ba-31f4-488d-a7a5-0d2a197ef775-kube-api-access-2klr5\") pod \"multus-additional-cni-plugins-xvhw4\" (UID: \"a69a78ba-31f4-488d-a7a5-0d2a197ef775\") " pod="openshift-multus/multus-additional-cni-plugins-xvhw4" Jul 12 08:21:18.945616 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.945571 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5tt9\" (UniqueName: \"kubernetes.io/projected/508b3df5-2cbe-4cd6-a8fd-e6cb29188184-kube-api-access-p5tt9\") pod \"network-metrics-daemon-9ms85\" (UID: \"508b3df5-2cbe-4cd6-a8fd-e6cb29188184\") " pod="openshift-multus/network-metrics-daemon-9ms85" Jul 12 08:21:18.946178 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.946125 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5k8nw\" (UniqueName: \"kubernetes.io/projected/abe468eb-f6d3-4af4-bf5e-a3c029a404ab-kube-api-access-5k8nw\") pod \"ovnkube-node-lfbsq\" (UID: \"abe468eb-f6d3-4af4-bf5e-a3c029a404ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:18.946286 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.946180 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9npxj\" (UniqueName: \"kubernetes.io/projected/ccef3090-87ec-4e72-a24c-5dd65f19c55b-kube-api-access-9npxj\") pod \"node-resolver-lwln2\" (UID: \"ccef3090-87ec-4e72-a24c-5dd65f19c55b\") " pod="openshift-dns/node-resolver-lwln2" Jul 12 08:21:18.947000 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.946945 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwzjr\" (UniqueName: \"kubernetes.io/projected/6cd77bb0-5bbe-4d49-9e22-05dad8c650af-kube-api-access-lwzjr\") pod \"node-ca-pzgtb\" (UID: \"6cd77bb0-5bbe-4d49-9e22-05dad8c650af\") " pod="openshift-image-registry/node-ca-pzgtb" Jul 12 08:21:18.947301 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.947276 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kccnm\" (UniqueName: \"kubernetes.io/projected/c9625c83-f82b-4eec-9e5d-1529523bc741-kube-api-access-kccnm\") pod \"iptables-alerter-kk4zw\" (UID: \"c9625c83-f82b-4eec-9e5d-1529523bc741\") " pod="openshift-network-operator/iptables-alerter-kk4zw" Jul 12 08:21:18.947591 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:18.947555 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f6lb\" (UniqueName: \"kubernetes.io/projected/6b580b23-6e3f-47b0-bcf9-2aca08bc5578-kube-api-access-4f6lb\") pod \"aws-ebs-csi-driver-node-c8ncb\" (UID: \"6b580b23-6e3f-47b0-bcf9-2aca08bc5578\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c8ncb" Jul 12 08:21:19.024515 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:19.024450 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-138.ec2.internal" event={"ID":"4f7c17a66495e6fef40567703d162ecc","Type":"ContainerStarted","Data":"97906fee2d083db1aa06b88e1daa8626ba063c93e01c993d8caf6c6cfd73fa00"} Jul 12 08:21:19.026561 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:19.026503 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-138.ec2.internal" event={"ID":"73eb6c7cf5825b2f7fddf7b8ec918121","Type":"ContainerStarted","Data":"f07831ef3b4d1c73e5a1dc8697905df39f7a2a45493357a16d91a7940c5596bf"} Jul 12 08:21:19.030761 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:19.030735 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8mwdg" Jul 12 08:21:19.042609 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:19.042567 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-j8z49" Jul 12 08:21:19.043410 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:19.043382 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37e6726a_f4aa_431c_8d02_961f53c45038.slice/crio-22653d7e4c83547f7dabacb2101e1b578b8b62d00b934138fa34f9dbb333793f WatchSource:0}: Error finding container 22653d7e4c83547f7dabacb2101e1b578b8b62d00b934138fa34f9dbb333793f: Status 404 returned error can't find the container with id 22653d7e4c83547f7dabacb2101e1b578b8b62d00b934138fa34f9dbb333793f Jul 12 08:21:19.053347 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:19.053314 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod869193b6_0a98_421f_be76_cd6ca5de6d75.slice/crio-0c6bc586872df2d918fd4535fef4372f31be75ff15ffc4447c32972c3f949f4f WatchSource:0}: Error finding container 0c6bc586872df2d918fd4535fef4372f31be75ff15ffc4447c32972c3f949f4f: Status 404 returned error can't find the container with id 0c6bc586872df2d918fd4535fef4372f31be75ff15ffc4447c32972c3f949f4f Jul 12 08:21:19.053454 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:19.053444 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pfjqv" Jul 12 08:21:19.054476 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:19.054453 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 08:21:19.062133 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:19.061941 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kk4zw" Jul 12 08:21:19.064802 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:19.064776 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dcfbd82_b903_426b_9573_7e68317111b0.slice/crio-b1fde1bbe6a0b29312febfb7fdff0c532b9782212c765fdc37648ad63f04f110 WatchSource:0}: Error finding container b1fde1bbe6a0b29312febfb7fdff0c532b9782212c765fdc37648ad63f04f110: Status 404 returned error can't find the container with id b1fde1bbe6a0b29312febfb7fdff0c532b9782212c765fdc37648ad63f04f110 Jul 12 08:21:19.070877 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:19.070788 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c8ncb" Jul 12 08:21:19.072062 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:19.072042 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9625c83_f82b_4eec_9e5d_1529523bc741.slice/crio-220b6d4553f9cb43cc1bd9a8a27501ce8d79247e193c4d956fc2fe1a137cbb24 WatchSource:0}: Error finding container 220b6d4553f9cb43cc1bd9a8a27501ce8d79247e193c4d956fc2fe1a137cbb24: Status 404 returned error can't find the container with id 220b6d4553f9cb43cc1bd9a8a27501ce8d79247e193c4d956fc2fe1a137cbb24 Jul 12 08:21:19.078723 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:19.078700 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lwln2" Jul 12 08:21:19.086653 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:19.086610 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pzgtb" Jul 12 08:21:19.096053 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:19.096035 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xvhw4" Jul 12 08:21:19.101906 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:19.101884 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:19.335508 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:19.335413 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f993375-bf43-40a2-b8c6-558a97dcba75-original-pull-secret\") pod \"global-pull-secret-syncer-w4cfw\" (UID: \"0f993375-bf43-40a2-b8c6-558a97dcba75\") " pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:21:19.335684 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:19.335621 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 08:21:19.335747 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:19.335699 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f993375-bf43-40a2-b8c6-558a97dcba75-original-pull-secret podName:0f993375-bf43-40a2-b8c6-558a97dcba75 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:20.335678236 +0000 UTC m=+3.996473886 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0f993375-bf43-40a2-b8c6-558a97dcba75-original-pull-secret") pod "global-pull-secret-syncer-w4cfw" (UID: "0f993375-bf43-40a2-b8c6-558a97dcba75") : object "kube-system"/"original-pull-secret" not registered Jul 12 08:21:19.436304 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:19.436168 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/508b3df5-2cbe-4cd6-a8fd-e6cb29188184-metrics-certs\") pod \"network-metrics-daemon-9ms85\" (UID: \"508b3df5-2cbe-4cd6-a8fd-e6cb29188184\") " pod="openshift-multus/network-metrics-daemon-9ms85" Jul 12 08:21:19.436494 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:19.436341 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 08:21:19.436494 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:19.436413 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/508b3df5-2cbe-4cd6-a8fd-e6cb29188184-metrics-certs podName:508b3df5-2cbe-4cd6-a8fd-e6cb29188184 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:20.43639216 +0000 UTC m=+4.097187801 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/508b3df5-2cbe-4cd6-a8fd-e6cb29188184-metrics-certs") pod "network-metrics-daemon-9ms85" (UID: "508b3df5-2cbe-4cd6-a8fd-e6cb29188184") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 08:21:19.537203 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:19.537166 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr7dk\" (UniqueName: \"kubernetes.io/projected/f66d4032-1ab2-4d52-874b-a7d0296525ee-kube-api-access-sr7dk\") pod \"network-check-target-hr58s\" (UID: \"f66d4032-1ab2-4d52-874b-a7d0296525ee\") " pod="openshift-network-diagnostics/network-check-target-hr58s" Jul 12 08:21:19.537377 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:19.537331 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 08:21:19.537377 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:19.537364 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 08:21:19.537377 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:19.537379 2566 projected.go:196] Error preparing data for projected volume kube-api-access-sr7dk for pod openshift-network-diagnostics/network-check-target-hr58s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 08:21:19.537509 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:19.537446 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f66d4032-1ab2-4d52-874b-a7d0296525ee-kube-api-access-sr7dk podName:f66d4032-1ab2-4d52-874b-a7d0296525ee nodeName:}" failed. No retries permitted until 2026-07-12 08:21:20.537425243 +0000 UTC m=+4.198220910 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-sr7dk" (UniqueName: "kubernetes.io/projected/f66d4032-1ab2-4d52-874b-a7d0296525ee-kube-api-access-sr7dk") pod "network-check-target-hr58s" (UID: "f66d4032-1ab2-4d52-874b-a7d0296525ee") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 08:21:19.618051 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:19.617965 2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 08:21:19.766053 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:19.765996 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-11 08:16:18 +0000 UTC" deadline="2028-04-22 05:10:53.348967292 +0000 UTC" Jul 12 08:21:19.766053 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:19.766039 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15596h49m33.582933625s" Jul 12 08:21:19.903745 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:19.903674 2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 08:21:20.020547 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:20.020377 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:21:20.020547 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:20.020409 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ms85" Jul 12 08:21:20.020547 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:20.020521 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4cfw" podUID="0f993375-bf43-40a2-b8c6-558a97dcba75" Jul 12 08:21:20.020851 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:20.020664 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ms85" podUID="508b3df5-2cbe-4cd6-a8fd-e6cb29188184" Jul 12 08:21:20.029529 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:20.029495 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kk4zw" event={"ID":"c9625c83-f82b-4eec-9e5d-1529523bc741","Type":"ContainerStarted","Data":"220b6d4553f9cb43cc1bd9a8a27501ce8d79247e193c4d956fc2fe1a137cbb24"} Jul 12 08:21:20.031582 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:20.031512 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pfjqv" event={"ID":"3dcfbd82-b903-426b-9573-7e68317111b0","Type":"ContainerStarted","Data":"b1fde1bbe6a0b29312febfb7fdff0c532b9782212c765fdc37648ad63f04f110"} Jul 12 08:21:20.033176 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:20.033144 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-j8z49" event={"ID":"869193b6-0a98-421f-be76-cd6ca5de6d75","Type":"ContainerStarted","Data":"0c6bc586872df2d918fd4535fef4372f31be75ff15ffc4447c32972c3f949f4f"} Jul 12 08:21:20.034711 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:20.034687 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8mwdg" event={"ID":"37e6726a-f4aa-431c-8d02-961f53c45038","Type":"ContainerStarted","Data":"22653d7e4c83547f7dabacb2101e1b578b8b62d00b934138fa34f9dbb333793f"} Jul 12 08:21:20.227847 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:20.227802 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b580b23_6e3f_47b0_bcf9_2aca08bc5578.slice/crio-3c1b1fe80b0d679c09e5bb9677deb465e98992321617314707210e1ef0336856 WatchSource:0}: Error finding container 3c1b1fe80b0d679c09e5bb9677deb465e98992321617314707210e1ef0336856: Status 404 returned error can't find the container with id 3c1b1fe80b0d679c09e5bb9677deb465e98992321617314707210e1ef0336856 Jul 12 08:21:20.232030 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:20.231983 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cd77bb0_5bbe_4d49_9e22_05dad8c650af.slice/crio-7385bdfb45f49c3cc8b6f6bc690fa777e88fcd65c5b781348e0de720e4530e77 WatchSource:0}: Error finding container 7385bdfb45f49c3cc8b6f6bc690fa777e88fcd65c5b781348e0de720e4530e77: Status 404 returned error can't find the container with id 7385bdfb45f49c3cc8b6f6bc690fa777e88fcd65c5b781348e0de720e4530e77 Jul 12 08:21:20.234168 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:20.234145 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabe468eb_f6d3_4af4_bf5e_a3c029a404ab.slice/crio-cbb1f32e4e039e1a59f7a8d9c7e0451516648505a680fede28cbd469af45c815 WatchSource:0}: Error finding container cbb1f32e4e039e1a59f7a8d9c7e0451516648505a680fede28cbd469af45c815: Status 404 returned error can't find the container with id cbb1f32e4e039e1a59f7a8d9c7e0451516648505a680fede28cbd469af45c815 Jul 12 08:21:20.239150 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:20.239059 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda69a78ba_31f4_488d_a7a5_0d2a197ef775.slice/crio-de50949a2faf8599c9970b3bbd577dee0592a4d5f5e4878cfda322711a001353 WatchSource:0}: Error finding container de50949a2faf8599c9970b3bbd577dee0592a4d5f5e4878cfda322711a001353: Status 404 returned error can't find the container with id de50949a2faf8599c9970b3bbd577dee0592a4d5f5e4878cfda322711a001353 Jul 12 08:21:20.239586 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:20.239492 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccef3090_87ec_4e72_a24c_5dd65f19c55b.slice/crio-6c023925faaaf22b93e3d33be891c7145208e2d41d008f3ed8b9424ac8a53a72 WatchSource:0}: Error finding container 6c023925faaaf22b93e3d33be891c7145208e2d41d008f3ed8b9424ac8a53a72: Status 404 returned error can't find the container with id 6c023925faaaf22b93e3d33be891c7145208e2d41d008f3ed8b9424ac8a53a72 Jul 12 08:21:20.343371 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:20.342526 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f993375-bf43-40a2-b8c6-558a97dcba75-original-pull-secret\") pod \"global-pull-secret-syncer-w4cfw\" (UID: \"0f993375-bf43-40a2-b8c6-558a97dcba75\") " pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:21:20.343371 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:20.342928 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 08:21:20.343371 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:20.342995 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f993375-bf43-40a2-b8c6-558a97dcba75-original-pull-secret podName:0f993375-bf43-40a2-b8c6-558a97dcba75 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:22.342975456 +0000 UTC m=+6.003771118 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0f993375-bf43-40a2-b8c6-558a97dcba75-original-pull-secret") pod "global-pull-secret-syncer-w4cfw" (UID: "0f993375-bf43-40a2-b8c6-558a97dcba75") : object "kube-system"/"original-pull-secret" not registered Jul 12 08:21:20.444123 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:20.444054 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/508b3df5-2cbe-4cd6-a8fd-e6cb29188184-metrics-certs\") pod \"network-metrics-daemon-9ms85\" (UID: \"508b3df5-2cbe-4cd6-a8fd-e6cb29188184\") " pod="openshift-multus/network-metrics-daemon-9ms85" Jul 12 08:21:20.444308 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:20.444244 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 08:21:20.444364 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:20.444324 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/508b3df5-2cbe-4cd6-a8fd-e6cb29188184-metrics-certs podName:508b3df5-2cbe-4cd6-a8fd-e6cb29188184 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:22.44430266 +0000 UTC m=+6.105098321 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/508b3df5-2cbe-4cd6-a8fd-e6cb29188184-metrics-certs") pod "network-metrics-daemon-9ms85" (UID: "508b3df5-2cbe-4cd6-a8fd-e6cb29188184") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 08:21:20.544915 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:20.544885 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr7dk\" (UniqueName: \"kubernetes.io/projected/f66d4032-1ab2-4d52-874b-a7d0296525ee-kube-api-access-sr7dk\") pod \"network-check-target-hr58s\" (UID: \"f66d4032-1ab2-4d52-874b-a7d0296525ee\") " pod="openshift-network-diagnostics/network-check-target-hr58s" Jul 12 08:21:20.545077 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:20.545061 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 08:21:20.545138 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:20.545083 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 08:21:20.545138 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:20.545096 2566 projected.go:196] Error preparing data for projected volume kube-api-access-sr7dk for pod openshift-network-diagnostics/network-check-target-hr58s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 08:21:20.545247 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:20.545155 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f66d4032-1ab2-4d52-874b-a7d0296525ee-kube-api-access-sr7dk podName:f66d4032-1ab2-4d52-874b-a7d0296525ee nodeName:}" failed. No retries permitted until 2026-07-12 08:21:22.54513607 +0000 UTC m=+6.205931718 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-sr7dk" (UniqueName: "kubernetes.io/projected/f66d4032-1ab2-4d52-874b-a7d0296525ee-kube-api-access-sr7dk") pod "network-check-target-hr58s" (UID: "f66d4032-1ab2-4d52-874b-a7d0296525ee") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 08:21:20.769112 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:20.767097 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-11 08:16:18 +0000 UTC" deadline="2028-04-22 17:02:03.209835148 +0000 UTC" Jul 12 08:21:20.769112 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:20.767158 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15608h40m42.442680968s" Jul 12 08:21:21.022966 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:21.022892 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hr58s" Jul 12 08:21:21.023126 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:21.023015 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hr58s" podUID="f66d4032-1ab2-4d52-874b-a7d0296525ee" Jul 12 08:21:21.046251 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:21.046122 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" event={"ID":"abe468eb-f6d3-4af4-bf5e-a3c029a404ab","Type":"ContainerStarted","Data":"cbb1f32e4e039e1a59f7a8d9c7e0451516648505a680fede28cbd469af45c815"} Jul 12 08:21:21.052727 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:21.052654 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pzgtb" event={"ID":"6cd77bb0-5bbe-4d49-9e22-05dad8c650af","Type":"ContainerStarted","Data":"7385bdfb45f49c3cc8b6f6bc690fa777e88fcd65c5b781348e0de720e4530e77"} Jul 12 08:21:21.059945 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:21.059881 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-138.ec2.internal" event={"ID":"4f7c17a66495e6fef40567703d162ecc","Type":"ContainerStarted","Data":"58c9e9d784a3e259b6a754e54e12364b4dfec253ccb31c4f0cacf8157dd29d32"} Jul 12 08:21:21.067224 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:21.066647 2566 generic.go:356] "Generic (PLEG): container finished" podID="73eb6c7cf5825b2f7fddf7b8ec918121" containerID="43b654ae07ab0144ad62ebd38d1281f46942b2665a4f73a2a6ba27fcc56162c1" exitCode=0 Jul 12 08:21:21.067224 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:21.066909 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-138.ec2.internal" event={"ID":"73eb6c7cf5825b2f7fddf7b8ec918121","Type":"ContainerDied","Data":"43b654ae07ab0144ad62ebd38d1281f46942b2665a4f73a2a6ba27fcc56162c1"} Jul 12 08:21:21.076738 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:21.076710 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lwln2" event={"ID":"ccef3090-87ec-4e72-a24c-5dd65f19c55b","Type":"ContainerStarted","Data":"6c023925faaaf22b93e3d33be891c7145208e2d41d008f3ed8b9424ac8a53a72"} Jul 12 08:21:21.082757 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:21.082702 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xvhw4" event={"ID":"a69a78ba-31f4-488d-a7a5-0d2a197ef775","Type":"ContainerStarted","Data":"de50949a2faf8599c9970b3bbd577dee0592a4d5f5e4878cfda322711a001353"} Jul 12 08:21:21.087158 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:21.087130 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c8ncb" event={"ID":"6b580b23-6e3f-47b0-bcf9-2aca08bc5578","Type":"ContainerStarted","Data":"3c1b1fe80b0d679c09e5bb9677deb465e98992321617314707210e1ef0336856"} Jul 12 08:21:21.091997 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:21.091947 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-138.ec2.internal" podStartSLOduration=3.09193018 podStartE2EDuration="3.09193018s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 08:21:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:21:21.07463817 +0000 UTC m=+4.735433832" watchObservedRunningTime="2026-07-12 08:21:21.09193018 +0000 UTC m=+4.752725843" Jul 12 08:21:22.021284 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:22.020806 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:21:22.021284 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:22.020806 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ms85" Jul 12 08:21:22.021284 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:22.020948 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4cfw" podUID="0f993375-bf43-40a2-b8c6-558a97dcba75" Jul 12 08:21:22.021284 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:22.021003 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ms85" podUID="508b3df5-2cbe-4cd6-a8fd-e6cb29188184" Jul 12 08:21:22.104368 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:22.104275 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-138.ec2.internal" event={"ID":"73eb6c7cf5825b2f7fddf7b8ec918121","Type":"ContainerStarted","Data":"89cf73da115129de6f71c891b0334aa6f207194564fdc2057b40b3bf9b72c53b"} Jul 12 08:21:22.359702 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:22.359625 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f993375-bf43-40a2-b8c6-558a97dcba75-original-pull-secret\") pod \"global-pull-secret-syncer-w4cfw\" (UID: \"0f993375-bf43-40a2-b8c6-558a97dcba75\") " pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:21:22.359869 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:22.359785 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 08:21:22.359869 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:22.359846 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f993375-bf43-40a2-b8c6-558a97dcba75-original-pull-secret podName:0f993375-bf43-40a2-b8c6-558a97dcba75 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:26.359829036 +0000 UTC m=+10.020624679 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0f993375-bf43-40a2-b8c6-558a97dcba75-original-pull-secret") pod "global-pull-secret-syncer-w4cfw" (UID: "0f993375-bf43-40a2-b8c6-558a97dcba75") : object "kube-system"/"original-pull-secret" not registered Jul 12 08:21:22.461493 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:22.460960 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/508b3df5-2cbe-4cd6-a8fd-e6cb29188184-metrics-certs\") pod \"network-metrics-daemon-9ms85\" (UID: \"508b3df5-2cbe-4cd6-a8fd-e6cb29188184\") " pod="openshift-multus/network-metrics-daemon-9ms85" Jul 12 08:21:22.461493 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:22.461090 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 08:21:22.461493 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:22.461152 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/508b3df5-2cbe-4cd6-a8fd-e6cb29188184-metrics-certs podName:508b3df5-2cbe-4cd6-a8fd-e6cb29188184 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:26.461134825 +0000 UTC m=+10.121930468 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/508b3df5-2cbe-4cd6-a8fd-e6cb29188184-metrics-certs") pod "network-metrics-daemon-9ms85" (UID: "508b3df5-2cbe-4cd6-a8fd-e6cb29188184") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 08:21:22.562260 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:22.562215 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr7dk\" (UniqueName: \"kubernetes.io/projected/f66d4032-1ab2-4d52-874b-a7d0296525ee-kube-api-access-sr7dk\") pod \"network-check-target-hr58s\" (UID: \"f66d4032-1ab2-4d52-874b-a7d0296525ee\") " pod="openshift-network-diagnostics/network-check-target-hr58s" Jul 12 08:21:22.562438 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:22.562368 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 08:21:22.562438 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:22.562387 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 08:21:22.562438 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:22.562402 2566 projected.go:196] Error preparing data for projected volume kube-api-access-sr7dk for pod openshift-network-diagnostics/network-check-target-hr58s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 08:21:22.562637 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:22.562457 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f66d4032-1ab2-4d52-874b-a7d0296525ee-kube-api-access-sr7dk podName:f66d4032-1ab2-4d52-874b-a7d0296525ee nodeName:}" failed. No retries permitted until 2026-07-12 08:21:26.562439969 +0000 UTC m=+10.223235622 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-sr7dk" (UniqueName: "kubernetes.io/projected/f66d4032-1ab2-4d52-874b-a7d0296525ee-kube-api-access-sr7dk") pod "network-check-target-hr58s" (UID: "f66d4032-1ab2-4d52-874b-a7d0296525ee") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 08:21:23.022581 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:23.022488 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hr58s" Jul 12 08:21:23.023034 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:23.022619 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hr58s" podUID="f66d4032-1ab2-4d52-874b-a7d0296525ee" Jul 12 08:21:24.021082 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:24.020659 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:21:24.021082 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:24.020706 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ms85" Jul 12 08:21:24.021082 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:24.020789 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4cfw" podUID="0f993375-bf43-40a2-b8c6-558a97dcba75" Jul 12 08:21:24.021082 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:24.020941 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ms85" podUID="508b3df5-2cbe-4cd6-a8fd-e6cb29188184" Jul 12 08:21:25.020537 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:25.020424 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hr58s" Jul 12 08:21:25.020993 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:25.020579 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hr58s" podUID="f66d4032-1ab2-4d52-874b-a7d0296525ee" Jul 12 08:21:26.020286 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:26.020226 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:21:26.020286 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:26.020256 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ms85" Jul 12 08:21:26.020598 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:26.020364 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4cfw" podUID="0f993375-bf43-40a2-b8c6-558a97dcba75" Jul 12 08:21:26.020598 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:26.020514 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ms85" podUID="508b3df5-2cbe-4cd6-a8fd-e6cb29188184" Jul 12 08:21:26.392884 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:26.392813 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f993375-bf43-40a2-b8c6-558a97dcba75-original-pull-secret\") pod \"global-pull-secret-syncer-w4cfw\" (UID: \"0f993375-bf43-40a2-b8c6-558a97dcba75\") " pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:21:26.393021 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:26.392978 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 08:21:26.393086 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:26.393044 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f993375-bf43-40a2-b8c6-558a97dcba75-original-pull-secret podName:0f993375-bf43-40a2-b8c6-558a97dcba75 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:34.393025094 +0000 UTC m=+18.053820737 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0f993375-bf43-40a2-b8c6-558a97dcba75-original-pull-secret") pod "global-pull-secret-syncer-w4cfw" (UID: "0f993375-bf43-40a2-b8c6-558a97dcba75") : object "kube-system"/"original-pull-secret" not registered Jul 12 08:21:26.493507 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:26.493440 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/508b3df5-2cbe-4cd6-a8fd-e6cb29188184-metrics-certs\") pod \"network-metrics-daemon-9ms85\" (UID: \"508b3df5-2cbe-4cd6-a8fd-e6cb29188184\") " pod="openshift-multus/network-metrics-daemon-9ms85" Jul 12 08:21:26.493685 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:26.493632 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 08:21:26.493742 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:26.493709 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/508b3df5-2cbe-4cd6-a8fd-e6cb29188184-metrics-certs podName:508b3df5-2cbe-4cd6-a8fd-e6cb29188184 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:34.493686209 +0000 UTC m=+18.154481850 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/508b3df5-2cbe-4cd6-a8fd-e6cb29188184-metrics-certs") pod "network-metrics-daemon-9ms85" (UID: "508b3df5-2cbe-4cd6-a8fd-e6cb29188184") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 08:21:26.593988 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:26.593934 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr7dk\" (UniqueName: \"kubernetes.io/projected/f66d4032-1ab2-4d52-874b-a7d0296525ee-kube-api-access-sr7dk\") pod \"network-check-target-hr58s\" (UID: \"f66d4032-1ab2-4d52-874b-a7d0296525ee\") " pod="openshift-network-diagnostics/network-check-target-hr58s" Jul 12 08:21:26.594166 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:26.594112 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 08:21:26.594166 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:26.594134 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 08:21:26.594166 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:26.594146 2566 projected.go:196] Error preparing data for projected volume kube-api-access-sr7dk for pod openshift-network-diagnostics/network-check-target-hr58s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 08:21:26.594376 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:26.594196 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f66d4032-1ab2-4d52-874b-a7d0296525ee-kube-api-access-sr7dk podName:f66d4032-1ab2-4d52-874b-a7d0296525ee nodeName:}" failed. No retries permitted until 2026-07-12 08:21:34.594179348 +0000 UTC m=+18.254975001 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-sr7dk" (UniqueName: "kubernetes.io/projected/f66d4032-1ab2-4d52-874b-a7d0296525ee-kube-api-access-sr7dk") pod "network-check-target-hr58s" (UID: "f66d4032-1ab2-4d52-874b-a7d0296525ee") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 08:21:27.021712 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:27.021683 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hr58s" Jul 12 08:21:27.022183 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:27.021764 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hr58s" podUID="f66d4032-1ab2-4d52-874b-a7d0296525ee" Jul 12 08:21:28.021292 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:28.021251 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ms85" Jul 12 08:21:28.021436 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:28.021397 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ms85" podUID="508b3df5-2cbe-4cd6-a8fd-e6cb29188184" Jul 12 08:21:28.021520 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:28.021493 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:21:28.021664 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:28.021627 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4cfw" podUID="0f993375-bf43-40a2-b8c6-558a97dcba75" Jul 12 08:21:29.020318 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:29.020290 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hr58s" Jul 12 08:21:29.020760 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:29.020412 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hr58s" podUID="f66d4032-1ab2-4d52-874b-a7d0296525ee" Jul 12 08:21:30.021182 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:30.021152 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ms85" Jul 12 08:21:30.021182 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:30.021169 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:21:30.021625 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:30.021291 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ms85" podUID="508b3df5-2cbe-4cd6-a8fd-e6cb29188184" Jul 12 08:21:30.021625 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:30.021397 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4cfw" podUID="0f993375-bf43-40a2-b8c6-558a97dcba75" Jul 12 08:21:31.021012 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:31.020978 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hr58s" Jul 12 08:21:31.021207 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:31.021124 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hr58s" podUID="f66d4032-1ab2-4d52-874b-a7d0296525ee" Jul 12 08:21:32.020475 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:32.020441 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:21:32.020647 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:32.020444 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ms85" Jul 12 08:21:32.020647 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:32.020583 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4cfw" podUID="0f993375-bf43-40a2-b8c6-558a97dcba75" Jul 12 08:21:32.020758 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:32.020717 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ms85" podUID="508b3df5-2cbe-4cd6-a8fd-e6cb29188184" Jul 12 08:21:33.021360 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:33.021317 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hr58s" Jul 12 08:21:33.021847 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:33.021440 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hr58s" podUID="f66d4032-1ab2-4d52-874b-a7d0296525ee" Jul 12 08:21:34.020408 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:34.020380 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:21:34.020642 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:34.020424 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ms85" Jul 12 08:21:34.020642 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:34.020563 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4cfw" podUID="0f993375-bf43-40a2-b8c6-558a97dcba75" Jul 12 08:21:34.020744 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:34.020666 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ms85" podUID="508b3df5-2cbe-4cd6-a8fd-e6cb29188184" Jul 12 08:21:34.456880 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:34.456805 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f993375-bf43-40a2-b8c6-558a97dcba75-original-pull-secret\") pod \"global-pull-secret-syncer-w4cfw\" (UID: \"0f993375-bf43-40a2-b8c6-558a97dcba75\") " pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:21:34.457275 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:34.456939 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 08:21:34.457275 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:34.456997 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f993375-bf43-40a2-b8c6-558a97dcba75-original-pull-secret podName:0f993375-bf43-40a2-b8c6-558a97dcba75 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:50.456981685 +0000 UTC m=+34.117777324 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0f993375-bf43-40a2-b8c6-558a97dcba75-original-pull-secret") pod "global-pull-secret-syncer-w4cfw" (UID: "0f993375-bf43-40a2-b8c6-558a97dcba75") : object "kube-system"/"original-pull-secret" not registered Jul 12 08:21:34.557686 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:34.557653 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/508b3df5-2cbe-4cd6-a8fd-e6cb29188184-metrics-certs\") pod \"network-metrics-daemon-9ms85\" (UID: \"508b3df5-2cbe-4cd6-a8fd-e6cb29188184\") " pod="openshift-multus/network-metrics-daemon-9ms85" Jul 12 08:21:34.557823 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:34.557806 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 08:21:34.557875 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:34.557864 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/508b3df5-2cbe-4cd6-a8fd-e6cb29188184-metrics-certs podName:508b3df5-2cbe-4cd6-a8fd-e6cb29188184 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:50.557846743 +0000 UTC m=+34.218642382 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/508b3df5-2cbe-4cd6-a8fd-e6cb29188184-metrics-certs") pod "network-metrics-daemon-9ms85" (UID: "508b3df5-2cbe-4cd6-a8fd-e6cb29188184") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 08:21:34.658797 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:34.658763 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr7dk\" (UniqueName: \"kubernetes.io/projected/f66d4032-1ab2-4d52-874b-a7d0296525ee-kube-api-access-sr7dk\") pod \"network-check-target-hr58s\" (UID: \"f66d4032-1ab2-4d52-874b-a7d0296525ee\") " pod="openshift-network-diagnostics/network-check-target-hr58s" Jul 12 08:21:34.658952 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:34.658923 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 08:21:34.658952 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:34.658947 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 08:21:34.659048 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:34.658960 2566 projected.go:196] Error preparing data for projected volume kube-api-access-sr7dk for pod openshift-network-diagnostics/network-check-target-hr58s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 08:21:34.659048 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:34.659014 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f66d4032-1ab2-4d52-874b-a7d0296525ee-kube-api-access-sr7dk podName:f66d4032-1ab2-4d52-874b-a7d0296525ee nodeName:}" failed. No retries permitted until 2026-07-12 08:21:50.658999091 +0000 UTC m=+34.319794735 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-sr7dk" (UniqueName: "kubernetes.io/projected/f66d4032-1ab2-4d52-874b-a7d0296525ee-kube-api-access-sr7dk") pod "network-check-target-hr58s" (UID: "f66d4032-1ab2-4d52-874b-a7d0296525ee") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 08:21:35.020255 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:35.020225 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hr58s" Jul 12 08:21:35.020422 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:35.020349 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hr58s" podUID="f66d4032-1ab2-4d52-874b-a7d0296525ee" Jul 12 08:21:36.020404 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:36.020368 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ms85" Jul 12 08:21:36.020880 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:36.020407 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:21:36.020880 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:36.020514 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ms85" podUID="508b3df5-2cbe-4cd6-a8fd-e6cb29188184" Jul 12 08:21:36.020880 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:36.020661 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4cfw" podUID="0f993375-bf43-40a2-b8c6-558a97dcba75" Jul 12 08:21:37.023558 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:37.023429 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hr58s" Jul 12 08:21:37.024872 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:37.024817 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hr58s" podUID="f66d4032-1ab2-4d52-874b-a7d0296525ee" Jul 12 08:21:38.020454 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:38.020423 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:21:38.020582 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:38.020423 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ms85" Jul 12 08:21:38.020582 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:38.020527 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4cfw" podUID="0f993375-bf43-40a2-b8c6-558a97dcba75" Jul 12 08:21:38.020654 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:38.020611 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ms85" podUID="508b3df5-2cbe-4cd6-a8fd-e6cb29188184" Jul 12 08:21:39.021678 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:39.021207 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hr58s" Jul 12 08:21:39.022349 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:39.021748 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hr58s" podUID="f66d4032-1ab2-4d52-874b-a7d0296525ee" Jul 12 08:21:39.140078 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:39.140046 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lwln2" event={"ID":"ccef3090-87ec-4e72-a24c-5dd65f19c55b","Type":"ContainerStarted","Data":"0e09339738c67e689c891cdbb80dd68720f722c761bfaabae13599ed54d828de"} Jul 12 08:21:39.141584 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:39.141543 2566 generic.go:356] "Generic (PLEG): container finished" podID="a69a78ba-31f4-488d-a7a5-0d2a197ef775" containerID="5e7bf7a156c399fe47564eafcd65c924ccd7e58e84ac778af9eb58ababe40f21" exitCode=0 Jul 12 08:21:39.141689 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:39.141630 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xvhw4" event={"ID":"a69a78ba-31f4-488d-a7a5-0d2a197ef775","Type":"ContainerDied","Data":"5e7bf7a156c399fe47564eafcd65c924ccd7e58e84ac778af9eb58ababe40f21"} Jul 12 08:21:39.143165 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:39.143146 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c8ncb" event={"ID":"6b580b23-6e3f-47b0-bcf9-2aca08bc5578","Type":"ContainerStarted","Data":"49f3c1fd2a30e0faece6f00c9cde48b720df09e6b1a901215b23872df1ac3c5b"} Jul 12 08:21:39.144680 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:39.144657 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pfjqv" event={"ID":"3dcfbd82-b903-426b-9573-7e68317111b0","Type":"ContainerStarted","Data":"39a15519b6b4daf0f87df4b92c42bbc83dbfda130175744ef17581c8f2866127"} Jul 12 08:21:39.145923 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:39.145902 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-j8z49" event={"ID":"869193b6-0a98-421f-be76-cd6ca5de6d75","Type":"ContainerStarted","Data":"504f4f0ea59cc9e7bd3ce6aaf616eae16effe935c5e84502677a935bf6e789bc"} Jul 12 08:21:39.147150 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:39.147119 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8mwdg" event={"ID":"37e6726a-f4aa-431c-8d02-961f53c45038","Type":"ContainerStarted","Data":"3bf42b8f36db783d47c12fc8b2a212745b781a080418cacc90e6de5c7ce433f8"} Jul 12 08:21:39.149644 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:39.149628 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lfbsq_abe468eb-f6d3-4af4-bf5e-a3c029a404ab/ovn-acl-logging/0.log" Jul 12 08:21:39.149925 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:39.149907 2566 generic.go:356] "Generic (PLEG): container finished" podID="abe468eb-f6d3-4af4-bf5e-a3c029a404ab" containerID="22320586146b4cbdc9dd199fa8016886be960b7cb209aa038f2eeda6f0d7cd57" exitCode=1 Jul 12 08:21:39.149969 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:39.149922 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" event={"ID":"abe468eb-f6d3-4af4-bf5e-a3c029a404ab","Type":"ContainerStarted","Data":"ed313748683a258726bc94a598d2eeef82ac0cececc371c014c10680a1787373"} Jul 12 08:21:39.150018 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:39.149987 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" event={"ID":"abe468eb-f6d3-4af4-bf5e-a3c029a404ab","Type":"ContainerStarted","Data":"4b0940f9feb58ec7e7eea6e6b37a534e119dc916114ada832f2c9b60a19d0258"} Jul 12 08:21:39.150069 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:39.150020 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" event={"ID":"abe468eb-f6d3-4af4-bf5e-a3c029a404ab","Type":"ContainerStarted","Data":"ad17ad2d7f9dd86434ee90e93eff027a70f46e2331936adf8d3949b689841779"} Jul 12 08:21:39.150069 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:39.150060 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" event={"ID":"abe468eb-f6d3-4af4-bf5e-a3c029a404ab","Type":"ContainerStarted","Data":"f6ff6a11626d565b1c18e1864e1a85a35ff601bc28ff076baba025b3ac6ec6b9"} Jul 12 08:21:39.150157 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:39.150090 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" event={"ID":"abe468eb-f6d3-4af4-bf5e-a3c029a404ab","Type":"ContainerDied","Data":"22320586146b4cbdc9dd199fa8016886be960b7cb209aa038f2eeda6f0d7cd57"} Jul 12 08:21:39.150157 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:39.150120 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" event={"ID":"abe468eb-f6d3-4af4-bf5e-a3c029a404ab","Type":"ContainerStarted","Data":"be8088cc49a4128bb88905c7fe27d1de572b269a247e519c2f3bbf86b06f6626"} Jul 12 08:21:39.151051 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:39.151033 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pzgtb" event={"ID":"6cd77bb0-5bbe-4d49-9e22-05dad8c650af","Type":"ContainerStarted","Data":"059274255d3f9799e0f08ba5947405a1fcb075d9425cbd18da2ac5ede141bc18"} Jul 12 08:21:39.154449 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:39.154417 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-lwln2" podStartSLOduration=4.35129295 podStartE2EDuration="22.154406803s" totalImagesPullingTime="17.803113853s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 08:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:21:39.154186728 +0000 UTC m=+22.814982387" watchObservedRunningTime="2026-07-12 08:21:39.154406803 +0000 UTC m=+22.815202462" Jul 12 08:21:39.154535 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:39.154502 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-138.ec2.internal" podStartSLOduration=21.154497981 podStartE2EDuration="21.154497981s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 08:21:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:21:22.119003053 +0000 UTC m=+5.779798714" watchObservedRunningTime="2026-07-12 08:21:39.154497981 +0000 UTC m=+22.815293642" Jul 12 08:21:39.182074 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:39.182043 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8mwdg" podStartSLOduration=3.183199761 podStartE2EDuration="22.182032009s" totalImagesPullingTime="18.998832248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:21:39.166825212 +0000 UTC m=+22.827620873" watchObservedRunningTime="2026-07-12 08:21:39.182032009 +0000 UTC m=+22.842827732" Jul 12 08:21:39.182254 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:39.182236 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-pfjqv" podStartSLOduration=3.172327714 podStartE2EDuration="22.182229538s" totalImagesPullingTime="19.009901824s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 08:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:21:39.181845689 +0000 UTC m=+22.842641389" watchObservedRunningTime="2026-07-12 08:21:39.182229538 +0000 UTC m=+22.843025198" Jul 12 08:21:39.199261 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:39.199233 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-j8z49" podStartSLOduration=3.184191393 podStartE2EDuration="22.199222907s" totalImagesPullingTime="19.015031514s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 08:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:21:39.199112755 +0000 UTC m=+22.859908416" watchObservedRunningTime="2026-07-12 08:21:39.199222907 +0000 UTC m=+22.860018567" Jul 12 08:21:39.238469 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:39.238435 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-pzgtb" podStartSLOduration=4.405388232 podStartE2EDuration="22.238424707s" totalImagesPullingTime="17.833036475s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 08:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:21:39.214425082 +0000 UTC m=+22.875220971" watchObservedRunningTime="2026-07-12 08:21:39.238424707 +0000 UTC m=+22.899220368" Jul 12 08:21:39.622064 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:39.622036 2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 12 08:21:39.821763 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:39.821608 2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-12T08:21:39.622060327Z","UUID":"12bb9fe6-631f-488f-a799-56f53808f27b","Handler":null,"Name":"","Endpoint":""} Jul 12 08:21:39.823408 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:39.823384 2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 12 08:21:39.823519 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:39.823428 2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 12 08:21:40.021176 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:40.021144 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ms85" Jul 12 08:21:40.021443 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:40.021188 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:21:40.021443 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:40.021269 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ms85" podUID="508b3df5-2cbe-4cd6-a8fd-e6cb29188184" Jul 12 08:21:40.021443 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:40.021395 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4cfw" podUID="0f993375-bf43-40a2-b8c6-558a97dcba75" Jul 12 08:21:40.155232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:40.155143 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kk4zw" event={"ID":"c9625c83-f82b-4eec-9e5d-1529523bc741","Type":"ContainerStarted","Data":"fef5a45f84d648b987b5f0706d4fc9d3a328d592d3e025ad136cbe17daabfadb"} Jul 12 08:21:40.157932 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:40.157770 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c8ncb" event={"ID":"6b580b23-6e3f-47b0-bcf9-2aca08bc5578","Type":"ContainerStarted","Data":"56d44e5ccea7a43d6d8da9e80a4b4157dc1708b31e4f55260f9988c6b4ba43b7"} Jul 12 08:21:40.168902 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:40.168858 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kk4zw" podStartSLOduration=4.175653726 podStartE2EDuration="23.168844396s" totalImagesPullingTime="18.99319067s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 08:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:21:40.168474713 +0000 UTC m=+23.829270370" watchObservedRunningTime="2026-07-12 08:21:40.168844396 +0000 UTC m=+23.829640057" Jul 12 08:21:41.021050 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:41.020883 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hr58s" Jul 12 08:21:41.021250 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:41.021145 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hr58s" podUID="f66d4032-1ab2-4d52-874b-a7d0296525ee" Jul 12 08:21:41.162788 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:41.162714 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lfbsq_abe468eb-f6d3-4af4-bf5e-a3c029a404ab/ovn-acl-logging/0.log" Jul 12 08:21:41.163308 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:41.163138 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" event={"ID":"abe468eb-f6d3-4af4-bf5e-a3c029a404ab","Type":"ContainerStarted","Data":"468b3c95d165bef7b6736e03ddb61341a62724adf6fda718e955c3944862c618"} Jul 12 08:21:41.165497 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:41.165469 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c8ncb" event={"ID":"6b580b23-6e3f-47b0-bcf9-2aca08bc5578","Type":"ContainerStarted","Data":"dbb818ff9bf54cf900da52fd9a65bcca47ec417594fdc0725b31f525c6237f44"} Jul 12 08:21:41.182889 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:41.182846 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c8ncb" podStartSLOduration=3.859029519 podStartE2EDuration="24.182832305s" totalImagesPullingTime="20.323802786s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 08:21:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:21:41.182543318 +0000 UTC m=+24.843338979" watchObservedRunningTime="2026-07-12 08:21:41.182832305 +0000 UTC m=+24.843628006" Jul 12 08:21:42.020777 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:42.020743 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:21:42.020989 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:42.020743 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ms85" Jul 12 08:21:42.020989 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:42.020898 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4cfw" podUID="0f993375-bf43-40a2-b8c6-558a97dcba75" Jul 12 08:21:42.020989 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:42.020945 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ms85" podUID="508b3df5-2cbe-4cd6-a8fd-e6cb29188184" Jul 12 08:21:43.020489 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:43.020466 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hr58s" Jul 12 08:21:43.021023 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:43.020591 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hr58s" podUID="f66d4032-1ab2-4d52-874b-a7d0296525ee" Jul 12 08:21:43.874144 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:43.874105 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8mwdg" Jul 12 08:21:43.874764 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:43.874739 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8mwdg" Jul 12 08:21:44.021121 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:44.021097 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:21:44.021665 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:44.021255 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4cfw" podUID="0f993375-bf43-40a2-b8c6-558a97dcba75" Jul 12 08:21:44.021665 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:44.021298 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ms85" Jul 12 08:21:44.021665 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:44.021395 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ms85" podUID="508b3df5-2cbe-4cd6-a8fd-e6cb29188184" Jul 12 08:21:44.174827 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:44.174701 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lfbsq_abe468eb-f6d3-4af4-bf5e-a3c029a404ab/ovn-acl-logging/0.log" Jul 12 08:21:44.175212 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:44.175178 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" event={"ID":"abe468eb-f6d3-4af4-bf5e-a3c029a404ab","Type":"ContainerStarted","Data":"e9ecac86d915e4dff78da9e6da2397e221aea6ef0219943e1ab3eeb20bf4924f"} Jul 12 08:21:44.175496 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:44.175472 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8mwdg" Jul 12 08:21:44.175916 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:44.175522 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:44.175916 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:44.175620 2566 scope.go:117] "RemoveContainer" containerID="22320586146b4cbdc9dd199fa8016886be960b7cb209aa038f2eeda6f0d7cd57" Jul 12 08:21:44.175916 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:44.175698 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:44.176109 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:44.176092 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8mwdg" Jul 12 08:21:44.190091 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:44.190072 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:45.020941 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:45.020913 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hr58s" Jul 12 08:21:45.021069 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:45.021014 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hr58s" podUID="f66d4032-1ab2-4d52-874b-a7d0296525ee" Jul 12 08:21:45.178631 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:45.178596 2566 generic.go:356] "Generic (PLEG): container finished" podID="a69a78ba-31f4-488d-a7a5-0d2a197ef775" containerID="b601ff2498d6cef28fcbc42970c8727cd6e74c30c7f5083e11f7d3073644819c" exitCode=0 Jul 12 08:21:45.179101 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:45.178655 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xvhw4" event={"ID":"a69a78ba-31f4-488d-a7a5-0d2a197ef775","Type":"ContainerDied","Data":"b601ff2498d6cef28fcbc42970c8727cd6e74c30c7f5083e11f7d3073644819c"} Jul 12 08:21:45.182385 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:45.182366 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lfbsq_abe468eb-f6d3-4af4-bf5e-a3c029a404ab/ovn-acl-logging/0.log" Jul 12 08:21:45.182739 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:45.182722 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" event={"ID":"abe468eb-f6d3-4af4-bf5e-a3c029a404ab","Type":"ContainerStarted","Data":"75e2746c1acc60ea12feacde0b126583e7aa4bbc59ed66bd45d3c28f4f7e6ed1"} Jul 12 08:21:45.182912 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:45.182894 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:45.197786 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:45.197768 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:21:45.226278 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:45.226236 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" podStartSLOduration=10.215466469 podStartE2EDuration="28.226224679s" totalImagesPullingTime="18.01075821s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 08:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:21:45.22565785 +0000 UTC m=+28.886453523" watchObservedRunningTime="2026-07-12 08:21:45.226224679 +0000 UTC m=+28.887020340" Jul 12 08:21:46.020817 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:46.020782 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ms85" Jul 12 08:21:46.020817 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:46.020820 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:21:46.020996 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:46.020892 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ms85" podUID="508b3df5-2cbe-4cd6-a8fd-e6cb29188184" Jul 12 08:21:46.020996 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:46.020946 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4cfw" podUID="0f993375-bf43-40a2-b8c6-558a97dcba75" Jul 12 08:21:46.034683 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:46.034654 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-w4cfw"] Jul 12 08:21:46.041706 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:46.041638 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9ms85"] Jul 12 08:21:46.041847 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:46.041740 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hr58s"] Jul 12 08:21:46.041847 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:46.041833 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hr58s" Jul 12 08:21:46.042660 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:46.041988 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hr58s" podUID="f66d4032-1ab2-4d52-874b-a7d0296525ee" Jul 12 08:21:46.184411 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:46.184381 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ms85" Jul 12 08:21:46.184411 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:46.184408 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:21:46.184831 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:46.184491 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ms85" podUID="508b3df5-2cbe-4cd6-a8fd-e6cb29188184" Jul 12 08:21:46.184871 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:46.184832 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4cfw" podUID="0f993375-bf43-40a2-b8c6-558a97dcba75" Jul 12 08:21:47.188910 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:47.188880 2566 generic.go:356] "Generic (PLEG): container finished" podID="a69a78ba-31f4-488d-a7a5-0d2a197ef775" containerID="a8bf34da67ed3827f37117060978404a2bacd29764b818e6f38fbc3133a98d22" exitCode=0 Jul 12 08:21:47.189234 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:47.188954 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xvhw4" event={"ID":"a69a78ba-31f4-488d-a7a5-0d2a197ef775","Type":"ContainerDied","Data":"a8bf34da67ed3827f37117060978404a2bacd29764b818e6f38fbc3133a98d22"} Jul 12 08:21:48.021182 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:48.021016 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:21:48.021331 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:48.021073 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hr58s" Jul 12 08:21:48.021331 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:48.021247 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4cfw" podUID="0f993375-bf43-40a2-b8c6-558a97dcba75" Jul 12 08:21:48.021331 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:48.021324 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hr58s" podUID="f66d4032-1ab2-4d52-874b-a7d0296525ee" Jul 12 08:21:48.021445 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:48.021076 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ms85" Jul 12 08:21:48.021445 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:48.021385 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ms85" podUID="508b3df5-2cbe-4cd6-a8fd-e6cb29188184" Jul 12 08:21:49.194749 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:49.194718 2566 generic.go:356] "Generic (PLEG): container finished" podID="a69a78ba-31f4-488d-a7a5-0d2a197ef775" containerID="4ef9c11a298c3c980eedb2055f75b2d8175e0b84fdc389f94ca10b9ca98b5781" exitCode=0 Jul 12 08:21:49.195362 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:49.194772 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xvhw4" event={"ID":"a69a78ba-31f4-488d-a7a5-0d2a197ef775","Type":"ContainerDied","Data":"4ef9c11a298c3c980eedb2055f75b2d8175e0b84fdc389f94ca10b9ca98b5781"} Jul 12 08:21:50.020379 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:50.020342 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hr58s" Jul 12 08:21:50.020565 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:50.020464 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ms85" Jul 12 08:21:50.020565 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:50.020477 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hr58s" podUID="f66d4032-1ab2-4d52-874b-a7d0296525ee" Jul 12 08:21:50.020675 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:50.020592 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ms85" podUID="508b3df5-2cbe-4cd6-a8fd-e6cb29188184" Jul 12 08:21:50.020675 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:50.020608 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:21:50.020745 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:50.020696 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4cfw" podUID="0f993375-bf43-40a2-b8c6-558a97dcba75" Jul 12 08:21:50.480350 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:50.480283 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f993375-bf43-40a2-b8c6-558a97dcba75-original-pull-secret\") pod \"global-pull-secret-syncer-w4cfw\" (UID: \"0f993375-bf43-40a2-b8c6-558a97dcba75\") " pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:21:50.480788 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:50.480394 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 08:21:50.480788 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:50.480448 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f993375-bf43-40a2-b8c6-558a97dcba75-original-pull-secret podName:0f993375-bf43-40a2-b8c6-558a97dcba75 nodeName:}" failed. No retries permitted until 2026-07-12 08:22:22.480432857 +0000 UTC m=+66.141228496 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0f993375-bf43-40a2-b8c6-558a97dcba75-original-pull-secret") pod "global-pull-secret-syncer-w4cfw" (UID: "0f993375-bf43-40a2-b8c6-558a97dcba75") : object "kube-system"/"original-pull-secret" not registered Jul 12 08:21:50.580682 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:50.580652 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/508b3df5-2cbe-4cd6-a8fd-e6cb29188184-metrics-certs\") pod \"network-metrics-daemon-9ms85\" (UID: \"508b3df5-2cbe-4cd6-a8fd-e6cb29188184\") " pod="openshift-multus/network-metrics-daemon-9ms85" Jul 12 08:21:50.580858 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:50.580787 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 08:21:50.580858 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:50.580846 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/508b3df5-2cbe-4cd6-a8fd-e6cb29188184-metrics-certs podName:508b3df5-2cbe-4cd6-a8fd-e6cb29188184 nodeName:}" failed. No retries permitted until 2026-07-12 08:22:22.580830223 +0000 UTC m=+66.241625878 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/508b3df5-2cbe-4cd6-a8fd-e6cb29188184-metrics-certs") pod "network-metrics-daemon-9ms85" (UID: "508b3df5-2cbe-4cd6-a8fd-e6cb29188184") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 08:21:50.681589 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:50.681511 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr7dk\" (UniqueName: \"kubernetes.io/projected/f66d4032-1ab2-4d52-874b-a7d0296525ee-kube-api-access-sr7dk\") pod \"network-check-target-hr58s\" (UID: \"f66d4032-1ab2-4d52-874b-a7d0296525ee\") " pod="openshift-network-diagnostics/network-check-target-hr58s" Jul 12 08:21:50.681869 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:50.681661 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 08:21:50.681869 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:50.681679 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 08:21:50.681869 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:50.681693 2566 projected.go:196] Error preparing data for projected volume kube-api-access-sr7dk for pod openshift-network-diagnostics/network-check-target-hr58s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 08:21:50.681869 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:50.681741 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f66d4032-1ab2-4d52-874b-a7d0296525ee-kube-api-access-sr7dk podName:f66d4032-1ab2-4d52-874b-a7d0296525ee nodeName:}" failed. No retries permitted until 2026-07-12 08:22:22.681727347 +0000 UTC m=+66.342522990 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-sr7dk" (UniqueName: "kubernetes.io/projected/f66d4032-1ab2-4d52-874b-a7d0296525ee-kube-api-access-sr7dk") pod "network-check-target-hr58s" (UID: "f66d4032-1ab2-4d52-874b-a7d0296525ee") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 08:21:51.147201 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.147170 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-138.ec2.internal" event="NodeReady" Jul 12 08:21:51.147432 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.147307 2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 12 08:21:51.181618 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.181585 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-k9kgv"] Jul 12 08:21:51.183614 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.183592 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5bd7886769-cpjl8"] Jul 12 08:21:51.183781 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.183725 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-k9kgv" Jul 12 08:21:51.185637 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.185488 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:21:51.185816 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.185763 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4z9qg"] Jul 12 08:21:51.187230 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.187202 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-p5fvd\"" Jul 12 08:21:51.187497 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.187476 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 12 08:21:51.187785 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.187767 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kxpkk"] Jul 12 08:21:51.187957 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.187938 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4z9qg" Jul 12 08:21:51.188912 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.188887 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 12 08:21:51.189167 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.189149 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 12 08:21:51.189475 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.189457 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n9s6q"] Jul 12 08:21:51.189562 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.189478 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-wckvx\"" Jul 12 08:21:51.189562 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.189531 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 12 08:21:51.189665 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.189609 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kxpkk" Jul 12 08:21:51.191348 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.191330 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-k2ltl"] Jul 12 08:21:51.191448 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.191432 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n9s6q" Jul 12 08:21:51.193128 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.193110 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94b94"] Jul 12 08:21:51.193365 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.193297 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 12 08:21:51.193594 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.193443 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 12 08:21:51.193594 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.193541 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 12 08:21:51.193594 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.193568 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 12 08:21:51.194279 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.193785 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-k2ltl" Jul 12 08:21:51.194279 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.194167 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 12 08:21:51.194711 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.194407 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 12 08:21:51.194769 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.194759 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-l6hkt\"" Jul 12 08:21:51.195156 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.194974 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 12 08:21:51.195156 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.195003 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 12 08:21:51.195156 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.195028 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-rvkmd"] Jul 12 08:21:51.195361 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.195269 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 12 08:21:51.196224 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.195476 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94b94" Jul 12 08:21:51.196224 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.195922 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 12 08:21:51.196224 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.196005 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-79zxt\"" Jul 12 08:21:51.196224 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.196064 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-s7pkc\"" Jul 12 08:21:51.196224 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.196106 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 12 08:21:51.196224 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.196140 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 12 08:21:51.201490 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.198002 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-k9kgv"] Jul 12 08:21:51.201490 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.198104 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-rvkmd" Jul 12 08:21:51.201490 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.198463 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 12 08:21:51.201490 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.199269 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 12 08:21:51.201490 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.199534 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 12 08:21:51.201490 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.200440 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 12 08:21:51.201490 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.200774 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 12 08:21:51.201490 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.200965 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-h7fqn\"" Jul 12 08:21:51.201490 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.201316 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 12 08:21:51.201920 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.201837 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 12 08:21:51.201971 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.201923 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 12 08:21:51.202162 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.202147 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 12 08:21:51.202290 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.202183 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-g5dtw\"" Jul 12 08:21:51.203484 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.203278 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 12 08:21:51.203484 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.203333 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 12 08:21:51.203484 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.203390 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 12 08:21:51.203976 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.203954 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-sfd26\"" Jul 12 08:21:51.208263 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.208212 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 12 08:21:51.212193 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.212079 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 12 08:21:51.213977 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.213959 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4z9qg"] Jul 12 08:21:51.215253 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.215189 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 12 08:21:51.215253 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.215232 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5bd7886769-cpjl8"] Jul 12 08:21:51.216263 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.216241 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-k2ltl"] Jul 12 08:21:51.217371 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.217354 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-rvkmd"] Jul 12 08:21:51.218669 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.218592 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94b94"] Jul 12 08:21:51.220236 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.219304 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kxpkk"] Jul 12 08:21:51.221485 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.221388 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-b9fxd"] Jul 12 08:21:51.223633 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.223533 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-8t6pv"] Jul 12 08:21:51.223721 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.223667 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-b9fxd" Jul 12 08:21:51.226008 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.225988 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 12 08:21:51.226268 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.226250 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 12 08:21:51.227382 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.226480 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-b8989f8c6-v5dgq"] Jul 12 08:21:51.227382 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.226782 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-8t6pv" Jul 12 08:21:51.227382 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.226968 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-kwmw2\"" Jul 12 08:21:51.229583 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.228640 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-b756k"] Jul 12 08:21:51.229583 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.229048 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-b8989f8c6-v5dgq" Jul 12 08:21:51.230477 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.230302 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-bl4p7\"" Jul 12 08:21:51.230477 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.230301 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 12 08:21:51.230971 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.230814 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 12 08:21:51.231210 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.231195 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4p4w4"] Jul 12 08:21:51.231463 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.231447 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b756k" Jul 12 08:21:51.232169 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.232151 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 12 08:21:51.233137 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.232383 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 12 08:21:51.233137 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.232655 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 12 08:21:51.233137 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.232682 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-qfn84\"" Jul 12 08:21:51.233137 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.232838 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 12 08:21:51.233137 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.232998 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 12 08:21:51.233137 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.233045 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 12 08:21:51.233773 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.233571 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 12 08:21:51.234047 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.234029 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 12 08:21:51.234200 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.234180 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9lnkq\"" Jul 12 08:21:51.235748 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.235732 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n9s6q"] Jul 12 08:21:51.235939 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.235918 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-b9fxd"] Jul 12 08:21:51.236093 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.235948 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4p4w4"] Jul 12 08:21:51.236093 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.235871 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4p4w4" Jul 12 08:21:51.236746 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.236529 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-8t6pv"] Jul 12 08:21:51.236746 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.236579 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b756k"] Jul 12 08:21:51.236746 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.236602 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-b8989f8c6-v5dgq"] Jul 12 08:21:51.238411 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.238384 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 12 08:21:51.240304 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.238714 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 12 08:21:51.240304 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.239019 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 12 08:21:51.240304 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.239125 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-x9vpt\"" Jul 12 08:21:51.286700 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.286670 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5cf8c8a8-825a-404d-b968-dfb4f4f466df-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-n9s6q\" (UID: \"5cf8c8a8-825a-404d-b968-dfb4f4f466df\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n9s6q" Jul 12 08:21:51.286795 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.286733 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh858\" (UniqueName: \"kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-kube-api-access-rh858\") pod \"image-registry-5bd7886769-cpjl8\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:21:51.286795 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.286754 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f8037f69-5405-41e4-9473-f2e22adbbb17-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-kxpkk\" (UID: \"f8037f69-5405-41e4-9473-f2e22adbbb17\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kxpkk" Jul 12 08:21:51.286795 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.286771 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-bound-sa-token\") pod \"image-registry-5bd7886769-cpjl8\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:21:51.286795 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.286789 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e142390b-f773-488a-92de-6883d5f4ad70-tmp\") pod \"insights-operator-68d9dd94cf-rvkmd\" (UID: \"e142390b-f773-488a-92de-6883d5f4ad70\") " pod="openshift-insights/insights-operator-68d9dd94cf-rvkmd" Jul 12 08:21:51.286945 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.286804 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t444\" (UniqueName: \"kubernetes.io/projected/5cf8c8a8-825a-404d-b968-dfb4f4f466df-kube-api-access-4t444\") pod \"kube-storage-version-migrator-operator-799cb7f958-n9s6q\" (UID: \"5cf8c8a8-825a-404d-b968-dfb4f4f466df\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n9s6q" Jul 12 08:21:51.286945 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.286818 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79wcp\" (UniqueName: \"kubernetes.io/projected/bc3f8e97-c9ab-4370-b437-8a47514c869c-kube-api-access-79wcp\") pod \"console-operator-7866497494-k2ltl\" (UID: \"bc3f8e97-c9ab-4370-b437-8a47514c869c\") " pod="openshift-console-operator/console-operator-7866497494-k2ltl" Jul 12 08:21:51.286945 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.286838 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/56a89c85-753b-4d5c-b012-a89e921d6400-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-94b94\" (UID: \"56a89c85-753b-4d5c-b012-a89e921d6400\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94b94" Jul 12 08:21:51.286945 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.286852 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jddx\" (UniqueName: \"kubernetes.io/projected/29040aa3-9ad7-4c27-9ed5-fcc5339a5e28-kube-api-access-8jddx\") pod \"volume-data-source-validator-78495597f6-k9kgv\" (UID: \"29040aa3-9ad7-4c27-9ed5-fcc5339a5e28\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-k9kgv" Jul 12 08:21:51.286945 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.286881 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2slb\" (UniqueName: \"kubernetes.io/projected/56a89c85-753b-4d5c-b012-a89e921d6400-kube-api-access-w2slb\") pod \"cluster-samples-operator-65db858894-94b94\" (UID: \"56a89c85-753b-4d5c-b012-a89e921d6400\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94b94" Jul 12 08:21:51.286945 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.286915 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f98d0302-b975-4e38-a0d2-1f37eb5679f3-installation-pull-secrets\") pod \"image-registry-5bd7886769-cpjl8\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:21:51.287230 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.286950 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/40be23cf-4432-4ea2-9979-3ae525ba3ef9-serving-cert\") pod \"service-ca-operator-7695fc6c8c-4z9qg\" (UID: \"40be23cf-4432-4ea2-9979-3ae525ba3ef9\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4z9qg" Jul 12 08:21:51.287230 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.286994 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bc3f8e97-c9ab-4370-b437-8a47514c869c-serving-cert\") pod \"console-operator-7866497494-k2ltl\" (UID: \"bc3f8e97-c9ab-4370-b437-8a47514c869c\") " pod="openshift-console-operator/console-operator-7866497494-k2ltl" Jul 12 08:21:51.287230 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.287038 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f8037f69-5405-41e4-9473-f2e22adbbb17-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-kxpkk\" (UID: \"f8037f69-5405-41e4-9473-f2e22adbbb17\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kxpkk" Jul 12 08:21:51.287230 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.287099 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f98d0302-b975-4e38-a0d2-1f37eb5679f3-registry-certificates\") pod \"image-registry-5bd7886769-cpjl8\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:21:51.287230 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.287124 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzrnv\" (UniqueName: \"kubernetes.io/projected/40be23cf-4432-4ea2-9979-3ae525ba3ef9-kube-api-access-xzrnv\") pod \"service-ca-operator-7695fc6c8c-4z9qg\" (UID: \"40be23cf-4432-4ea2-9979-3ae525ba3ef9\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4z9qg" Jul 12 08:21:51.287230 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.287173 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bc3f8e97-c9ab-4370-b437-8a47514c869c-trusted-ca\") pod \"console-operator-7866497494-k2ltl\" (UID: \"bc3f8e97-c9ab-4370-b437-8a47514c869c\") " pod="openshift-console-operator/console-operator-7866497494-k2ltl" Jul 12 08:21:51.287230 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.287200 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/e142390b-f773-488a-92de-6883d5f4ad70-snapshots\") pod \"insights-operator-68d9dd94cf-rvkmd\" (UID: \"e142390b-f773-488a-92de-6883d5f4ad70\") " pod="openshift-insights/insights-operator-68d9dd94cf-rvkmd" Jul 12 08:21:51.287230 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.287224 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e142390b-f773-488a-92de-6883d5f4ad70-serving-cert\") pod \"insights-operator-68d9dd94cf-rvkmd\" (UID: \"e142390b-f773-488a-92de-6883d5f4ad70\") " pod="openshift-insights/insights-operator-68d9dd94cf-rvkmd" Jul 12 08:21:51.287584 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.287247 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bc3f8e97-c9ab-4370-b437-8a47514c869c-config\") pod \"console-operator-7866497494-k2ltl\" (UID: \"bc3f8e97-c9ab-4370-b437-8a47514c869c\") " pod="openshift-console-operator/console-operator-7866497494-k2ltl" Jul 12 08:21:51.287584 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.287283 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgj5s\" (UniqueName: \"kubernetes.io/projected/f8037f69-5405-41e4-9473-f2e22adbbb17-kube-api-access-pgj5s\") pod \"cluster-monitoring-operator-5f7f494dd4-kxpkk\" (UID: \"f8037f69-5405-41e4-9473-f2e22adbbb17\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kxpkk" Jul 12 08:21:51.287584 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.287324 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f98d0302-b975-4e38-a0d2-1f37eb5679f3-trusted-ca\") pod \"image-registry-5bd7886769-cpjl8\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:21:51.287584 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.287353 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f98d0302-b975-4e38-a0d2-1f37eb5679f3-image-registry-private-configuration\") pod \"image-registry-5bd7886769-cpjl8\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:21:51.287584 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.287371 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-registry-tls\") pod \"image-registry-5bd7886769-cpjl8\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:21:51.287584 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.287403 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5cf8c8a8-825a-404d-b968-dfb4f4f466df-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-n9s6q\" (UID: \"5cf8c8a8-825a-404d-b968-dfb4f4f466df\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n9s6q" Jul 12 08:21:51.287584 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.287448 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzdjb\" (UniqueName: \"kubernetes.io/projected/e142390b-f773-488a-92de-6883d5f4ad70-kube-api-access-xzdjb\") pod \"insights-operator-68d9dd94cf-rvkmd\" (UID: \"e142390b-f773-488a-92de-6883d5f4ad70\") " pod="openshift-insights/insights-operator-68d9dd94cf-rvkmd" Jul 12 08:21:51.287584 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.287510 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f98d0302-b975-4e38-a0d2-1f37eb5679f3-ca-trust-extracted\") pod \"image-registry-5bd7886769-cpjl8\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:21:51.287584 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.287540 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/40be23cf-4432-4ea2-9979-3ae525ba3ef9-config\") pod \"service-ca-operator-7695fc6c8c-4z9qg\" (UID: \"40be23cf-4432-4ea2-9979-3ae525ba3ef9\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4z9qg" Jul 12 08:21:51.287893 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.287595 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e142390b-f773-488a-92de-6883d5f4ad70-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-rvkmd\" (UID: \"e142390b-f773-488a-92de-6883d5f4ad70\") " pod="openshift-insights/insights-operator-68d9dd94cf-rvkmd" Jul 12 08:21:51.287893 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.287619 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e142390b-f773-488a-92de-6883d5f4ad70-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-rvkmd\" (UID: \"e142390b-f773-488a-92de-6883d5f4ad70\") " pod="openshift-insights/insights-operator-68d9dd94cf-rvkmd" Jul 12 08:21:51.349570 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.349516 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:21:51.388056 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.388020 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e142390b-f773-488a-92de-6883d5f4ad70-serving-cert\") pod \"insights-operator-68d9dd94cf-rvkmd\" (UID: \"e142390b-f773-488a-92de-6883d5f4ad70\") " pod="openshift-insights/insights-operator-68d9dd94cf-rvkmd" Jul 12 08:21:51.388056 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.388053 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bc3f8e97-c9ab-4370-b437-8a47514c869c-config\") pod \"console-operator-7866497494-k2ltl\" (UID: \"bc3f8e97-c9ab-4370-b437-8a47514c869c\") " pod="openshift-console-operator/console-operator-7866497494-k2ltl" Jul 12 08:21:51.388284 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.388079 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgj5s\" (UniqueName: \"kubernetes.io/projected/f8037f69-5405-41e4-9473-f2e22adbbb17-kube-api-access-pgj5s\") pod \"cluster-monitoring-operator-5f7f494dd4-kxpkk\" (UID: \"f8037f69-5405-41e4-9473-f2e22adbbb17\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kxpkk" Jul 12 08:21:51.388284 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.388105 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/96c69912-dd41-420f-9c38-a7b64b14d50f-nginx-conf\") pod \"networking-console-plugin-6cf856758c-8t6pv\" (UID: \"96c69912-dd41-420f-9c38-a7b64b14d50f\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8t6pv" Jul 12 08:21:51.388284 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.388144 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f98d0302-b975-4e38-a0d2-1f37eb5679f3-trusted-ca\") pod \"image-registry-5bd7886769-cpjl8\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:21:51.388284 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.388173 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f98d0302-b975-4e38-a0d2-1f37eb5679f3-image-registry-private-configuration\") pod \"image-registry-5bd7886769-cpjl8\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:21:51.388284 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.388210 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-registry-tls\") pod \"image-registry-5bd7886769-cpjl8\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:21:51.388284 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.388236 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/088c2a49-9c89-44f2-b75a-68dbeda985a6-tmp-dir\") pod \"dns-default-b756k\" (UID: \"088c2a49-9c89-44f2-b75a-68dbeda985a6\") " pod="openshift-dns/dns-default-b756k" Jul 12 08:21:51.388284 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.388261 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4czjl\" (UniqueName: \"kubernetes.io/projected/373c0f0d-eabe-4ad9-af2c-51f5cc0e1c3d-kube-api-access-4czjl\") pod \"network-check-source-869dcfcdcd-b9fxd\" (UID: \"373c0f0d-eabe-4ad9-af2c-51f5cc0e1c3d\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-b9fxd" Jul 12 08:21:51.388595 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.388288 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5cf8c8a8-825a-404d-b968-dfb4f4f466df-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-n9s6q\" (UID: \"5cf8c8a8-825a-404d-b968-dfb4f4f466df\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n9s6q" Jul 12 08:21:51.388595 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.388315 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkdhw\" (UniqueName: \"kubernetes.io/projected/f5e63ffc-280a-49f6-8861-6b5bcf16ac29-kube-api-access-nkdhw\") pod \"ingress-canary-4p4w4\" (UID: \"f5e63ffc-280a-49f6-8861-6b5bcf16ac29\") " pod="openshift-ingress-canary/ingress-canary-4p4w4" Jul 12 08:21:51.388595 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:51.388341 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 08:21:51.388595 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:51.388363 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bd7886769-cpjl8: secret "image-registry-tls" not found Jul 12 08:21:51.388595 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:51.388469 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-registry-tls podName:f98d0302-b975-4e38-a0d2-1f37eb5679f3 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:51.88844853 +0000 UTC m=+35.549244176 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-registry-tls") pod "image-registry-5bd7886769-cpjl8" (UID: "f98d0302-b975-4e38-a0d2-1f37eb5679f3") : secret "image-registry-tls" not found Jul 12 08:21:51.388595 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.388344 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzdjb\" (UniqueName: \"kubernetes.io/projected/e142390b-f773-488a-92de-6883d5f4ad70-kube-api-access-xzdjb\") pod \"insights-operator-68d9dd94cf-rvkmd\" (UID: \"e142390b-f773-488a-92de-6883d5f4ad70\") " pod="openshift-insights/insights-operator-68d9dd94cf-rvkmd" Jul 12 08:21:51.388861 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.388611 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgfdd\" (UniqueName: \"kubernetes.io/projected/088c2a49-9c89-44f2-b75a-68dbeda985a6-kube-api-access-rgfdd\") pod \"dns-default-b756k\" (UID: \"088c2a49-9c89-44f2-b75a-68dbeda985a6\") " pod="openshift-dns/dns-default-b756k" Jul 12 08:21:51.388861 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.388664 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bd2626f-14a5-450e-ae9a-6968c5aa9055-metrics-certs\") pod \"router-default-b8989f8c6-v5dgq\" (UID: \"1bd2626f-14a5-450e-ae9a-6968c5aa9055\") " pod="openshift-ingress/router-default-b8989f8c6-v5dgq" Jul 12 08:21:51.388861 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.388695 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f98d0302-b975-4e38-a0d2-1f37eb5679f3-ca-trust-extracted\") pod \"image-registry-5bd7886769-cpjl8\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:21:51.388861 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.388718 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/40be23cf-4432-4ea2-9979-3ae525ba3ef9-config\") pod \"service-ca-operator-7695fc6c8c-4z9qg\" (UID: \"40be23cf-4432-4ea2-9979-3ae525ba3ef9\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4z9qg" Jul 12 08:21:51.388861 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.388747 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e142390b-f773-488a-92de-6883d5f4ad70-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-rvkmd\" (UID: \"e142390b-f773-488a-92de-6883d5f4ad70\") " pod="openshift-insights/insights-operator-68d9dd94cf-rvkmd" Jul 12 08:21:51.388861 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.388771 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e142390b-f773-488a-92de-6883d5f4ad70-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-rvkmd\" (UID: \"e142390b-f773-488a-92de-6883d5f4ad70\") " pod="openshift-insights/insights-operator-68d9dd94cf-rvkmd" Jul 12 08:21:51.388861 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.388794 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/088c2a49-9c89-44f2-b75a-68dbeda985a6-metrics-tls\") pod \"dns-default-b756k\" (UID: \"088c2a49-9c89-44f2-b75a-68dbeda985a6\") " pod="openshift-dns/dns-default-b756k" Jul 12 08:21:51.388861 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.388826 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5cf8c8a8-825a-404d-b968-dfb4f4f466df-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-n9s6q\" (UID: \"5cf8c8a8-825a-404d-b968-dfb4f4f466df\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n9s6q" Jul 12 08:21:51.389175 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.388867 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh858\" (UniqueName: \"kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-kube-api-access-rh858\") pod \"image-registry-5bd7886769-cpjl8\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:21:51.389175 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.388898 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f8037f69-5405-41e4-9473-f2e22adbbb17-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-kxpkk\" (UID: \"f8037f69-5405-41e4-9473-f2e22adbbb17\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kxpkk" Jul 12 08:21:51.389175 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.388926 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-bound-sa-token\") pod \"image-registry-5bd7886769-cpjl8\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:21:51.389175 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.388949 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/1bd2626f-14a5-450e-ae9a-6968c5aa9055-default-certificate\") pod \"router-default-b8989f8c6-v5dgq\" (UID: \"1bd2626f-14a5-450e-ae9a-6968c5aa9055\") " pod="openshift-ingress/router-default-b8989f8c6-v5dgq" Jul 12 08:21:51.389175 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.388969 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e142390b-f773-488a-92de-6883d5f4ad70-tmp\") pod \"insights-operator-68d9dd94cf-rvkmd\" (UID: \"e142390b-f773-488a-92de-6883d5f4ad70\") " pod="openshift-insights/insights-operator-68d9dd94cf-rvkmd" Jul 12 08:21:51.389175 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.388987 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t444\" (UniqueName: \"kubernetes.io/projected/5cf8c8a8-825a-404d-b968-dfb4f4f466df-kube-api-access-4t444\") pod \"kube-storage-version-migrator-operator-799cb7f958-n9s6q\" (UID: \"5cf8c8a8-825a-404d-b968-dfb4f4f466df\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n9s6q" Jul 12 08:21:51.389175 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.389002 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79wcp\" (UniqueName: \"kubernetes.io/projected/bc3f8e97-c9ab-4370-b437-8a47514c869c-kube-api-access-79wcp\") pod \"console-operator-7866497494-k2ltl\" (UID: \"bc3f8e97-c9ab-4370-b437-8a47514c869c\") " pod="openshift-console-operator/console-operator-7866497494-k2ltl" Jul 12 08:21:51.389175 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.389020 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/96c69912-dd41-420f-9c38-a7b64b14d50f-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8t6pv\" (UID: \"96c69912-dd41-420f-9c38-a7b64b14d50f\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8t6pv" Jul 12 08:21:51.389175 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.389035 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bc3f8e97-c9ab-4370-b437-8a47514c869c-config\") pod \"console-operator-7866497494-k2ltl\" (UID: \"bc3f8e97-c9ab-4370-b437-8a47514c869c\") " pod="openshift-console-operator/console-operator-7866497494-k2ltl" Jul 12 08:21:51.389175 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.389047 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/56a89c85-753b-4d5c-b012-a89e921d6400-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-94b94\" (UID: \"56a89c85-753b-4d5c-b012-a89e921d6400\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94b94" Jul 12 08:21:51.389175 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.389047 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5cf8c8a8-825a-404d-b968-dfb4f4f466df-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-n9s6q\" (UID: \"5cf8c8a8-825a-404d-b968-dfb4f4f466df\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n9s6q" Jul 12 08:21:51.389175 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.389062 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jddx\" (UniqueName: \"kubernetes.io/projected/29040aa3-9ad7-4c27-9ed5-fcc5339a5e28-kube-api-access-8jddx\") pod \"volume-data-source-validator-78495597f6-k9kgv\" (UID: \"29040aa3-9ad7-4c27-9ed5-fcc5339a5e28\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-k9kgv" Jul 12 08:21:51.389651 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.389092 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/1bd2626f-14a5-450e-ae9a-6968c5aa9055-stats-auth\") pod \"router-default-b8989f8c6-v5dgq\" (UID: \"1bd2626f-14a5-450e-ae9a-6968c5aa9055\") " pod="openshift-ingress/router-default-b8989f8c6-v5dgq" Jul 12 08:21:51.389651 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.389124 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2slb\" (UniqueName: \"kubernetes.io/projected/56a89c85-753b-4d5c-b012-a89e921d6400-kube-api-access-w2slb\") pod \"cluster-samples-operator-65db858894-94b94\" (UID: \"56a89c85-753b-4d5c-b012-a89e921d6400\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94b94" Jul 12 08:21:51.389651 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.389148 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7bnn\" (UniqueName: \"kubernetes.io/projected/1bd2626f-14a5-450e-ae9a-6968c5aa9055-kube-api-access-t7bnn\") pod \"router-default-b8989f8c6-v5dgq\" (UID: \"1bd2626f-14a5-450e-ae9a-6968c5aa9055\") " pod="openshift-ingress/router-default-b8989f8c6-v5dgq" Jul 12 08:21:51.389651 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.389170 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f98d0302-b975-4e38-a0d2-1f37eb5679f3-installation-pull-secrets\") pod \"image-registry-5bd7886769-cpjl8\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:21:51.389651 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.389192 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/40be23cf-4432-4ea2-9979-3ae525ba3ef9-serving-cert\") pod \"service-ca-operator-7695fc6c8c-4z9qg\" (UID: \"40be23cf-4432-4ea2-9979-3ae525ba3ef9\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4z9qg" Jul 12 08:21:51.389651 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.389212 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bc3f8e97-c9ab-4370-b437-8a47514c869c-serving-cert\") pod \"console-operator-7866497494-k2ltl\" (UID: \"bc3f8e97-c9ab-4370-b437-8a47514c869c\") " pod="openshift-console-operator/console-operator-7866497494-k2ltl" Jul 12 08:21:51.389651 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.389256 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f8037f69-5405-41e4-9473-f2e22adbbb17-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-kxpkk\" (UID: \"f8037f69-5405-41e4-9473-f2e22adbbb17\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kxpkk" Jul 12 08:21:51.389651 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.389298 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f98d0302-b975-4e38-a0d2-1f37eb5679f3-registry-certificates\") pod \"image-registry-5bd7886769-cpjl8\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:21:51.389651 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:51.389316 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 12 08:21:51.389651 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.389320 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f5e63ffc-280a-49f6-8861-6b5bcf16ac29-cert\") pod \"ingress-canary-4p4w4\" (UID: \"f5e63ffc-280a-49f6-8861-6b5bcf16ac29\") " pod="openshift-ingress-canary/ingress-canary-4p4w4" Jul 12 08:21:51.389651 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.389331 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f98d0302-b975-4e38-a0d2-1f37eb5679f3-trusted-ca\") pod \"image-registry-5bd7886769-cpjl8\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:21:51.389651 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.389351 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzrnv\" (UniqueName: \"kubernetes.io/projected/40be23cf-4432-4ea2-9979-3ae525ba3ef9-kube-api-access-xzrnv\") pod \"service-ca-operator-7695fc6c8c-4z9qg\" (UID: \"40be23cf-4432-4ea2-9979-3ae525ba3ef9\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4z9qg" Jul 12 08:21:51.389651 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:51.389366 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8037f69-5405-41e4-9473-f2e22adbbb17-cluster-monitoring-operator-tls podName:f8037f69-5405-41e4-9473-f2e22adbbb17 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:51.88935307 +0000 UTC m=+35.550148712 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f8037f69-5405-41e4-9473-f2e22adbbb17-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-kxpkk" (UID: "f8037f69-5405-41e4-9473-f2e22adbbb17") : secret "cluster-monitoring-operator-tls" not found Jul 12 08:21:51.390218 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:51.389844 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 12 08:21:51.390218 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:51.389888 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56a89c85-753b-4d5c-b012-a89e921d6400-samples-operator-tls podName:56a89c85-753b-4d5c-b012-a89e921d6400 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:51.889874699 +0000 UTC m=+35.550670350 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/56a89c85-753b-4d5c-b012-a89e921d6400-samples-operator-tls") pod "cluster-samples-operator-65db858894-94b94" (UID: "56a89c85-753b-4d5c-b012-a89e921d6400") : secret "samples-operator-tls" not found Jul 12 08:21:51.390218 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.389915 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/40be23cf-4432-4ea2-9979-3ae525ba3ef9-config\") pod \"service-ca-operator-7695fc6c8c-4z9qg\" (UID: \"40be23cf-4432-4ea2-9979-3ae525ba3ef9\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4z9qg" Jul 12 08:21:51.390218 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.389918 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/088c2a49-9c89-44f2-b75a-68dbeda985a6-config-volume\") pod \"dns-default-b756k\" (UID: \"088c2a49-9c89-44f2-b75a-68dbeda985a6\") " pod="openshift-dns/dns-default-b756k" Jul 12 08:21:51.390218 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.389971 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bc3f8e97-c9ab-4370-b437-8a47514c869c-trusted-ca\") pod \"console-operator-7866497494-k2ltl\" (UID: \"bc3f8e97-c9ab-4370-b437-8a47514c869c\") " pod="openshift-console-operator/console-operator-7866497494-k2ltl" Jul 12 08:21:51.390218 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.390000 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bd2626f-14a5-450e-ae9a-6968c5aa9055-service-ca-bundle\") pod \"router-default-b8989f8c6-v5dgq\" (UID: \"1bd2626f-14a5-450e-ae9a-6968c5aa9055\") " pod="openshift-ingress/router-default-b8989f8c6-v5dgq" Jul 12 08:21:51.390218 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.390111 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e142390b-f773-488a-92de-6883d5f4ad70-tmp\") pod \"insights-operator-68d9dd94cf-rvkmd\" (UID: \"e142390b-f773-488a-92de-6883d5f4ad70\") " pod="openshift-insights/insights-operator-68d9dd94cf-rvkmd" Jul 12 08:21:51.390532 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.390235 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f98d0302-b975-4e38-a0d2-1f37eb5679f3-ca-trust-extracted\") pod \"image-registry-5bd7886769-cpjl8\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:21:51.390611 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.390541 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e142390b-f773-488a-92de-6883d5f4ad70-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-rvkmd\" (UID: \"e142390b-f773-488a-92de-6883d5f4ad70\") " pod="openshift-insights/insights-operator-68d9dd94cf-rvkmd" Jul 12 08:21:51.391511 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.391453 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f8037f69-5405-41e4-9473-f2e22adbbb17-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-kxpkk\" (UID: \"f8037f69-5405-41e4-9473-f2e22adbbb17\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kxpkk" Jul 12 08:21:51.391511 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.391488 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e142390b-f773-488a-92de-6883d5f4ad70-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-rvkmd\" (UID: \"e142390b-f773-488a-92de-6883d5f4ad70\") " pod="openshift-insights/insights-operator-68d9dd94cf-rvkmd" Jul 12 08:21:51.392291 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.391768 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/e142390b-f773-488a-92de-6883d5f4ad70-snapshots\") pod \"insights-operator-68d9dd94cf-rvkmd\" (UID: \"e142390b-f773-488a-92de-6883d5f4ad70\") " pod="openshift-insights/insights-operator-68d9dd94cf-rvkmd" Jul 12 08:21:51.392291 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.391899 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bc3f8e97-c9ab-4370-b437-8a47514c869c-trusted-ca\") pod \"console-operator-7866497494-k2ltl\" (UID: \"bc3f8e97-c9ab-4370-b437-8a47514c869c\") " pod="openshift-console-operator/console-operator-7866497494-k2ltl" Jul 12 08:21:51.392291 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.391957 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f98d0302-b975-4e38-a0d2-1f37eb5679f3-registry-certificates\") pod \"image-registry-5bd7886769-cpjl8\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:21:51.392291 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.392237 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/e142390b-f773-488a-92de-6883d5f4ad70-snapshots\") pod \"insights-operator-68d9dd94cf-rvkmd\" (UID: \"e142390b-f773-488a-92de-6883d5f4ad70\") " pod="openshift-insights/insights-operator-68d9dd94cf-rvkmd" Jul 12 08:21:51.393492 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.393465 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/40be23cf-4432-4ea2-9979-3ae525ba3ef9-serving-cert\") pod \"service-ca-operator-7695fc6c8c-4z9qg\" (UID: \"40be23cf-4432-4ea2-9979-3ae525ba3ef9\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4z9qg" Jul 12 08:21:51.393799 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.393778 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f98d0302-b975-4e38-a0d2-1f37eb5679f3-installation-pull-secrets\") pod \"image-registry-5bd7886769-cpjl8\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:21:51.393896 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.393832 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f98d0302-b975-4e38-a0d2-1f37eb5679f3-image-registry-private-configuration\") pod \"image-registry-5bd7886769-cpjl8\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:21:51.394163 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.394138 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5cf8c8a8-825a-404d-b968-dfb4f4f466df-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-n9s6q\" (UID: \"5cf8c8a8-825a-404d-b968-dfb4f4f466df\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n9s6q" Jul 12 08:21:51.394262 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.394165 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e142390b-f773-488a-92de-6883d5f4ad70-serving-cert\") pod \"insights-operator-68d9dd94cf-rvkmd\" (UID: \"e142390b-f773-488a-92de-6883d5f4ad70\") " pod="openshift-insights/insights-operator-68d9dd94cf-rvkmd" Jul 12 08:21:51.394407 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.394383 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bc3f8e97-c9ab-4370-b437-8a47514c869c-serving-cert\") pod \"console-operator-7866497494-k2ltl\" (UID: \"bc3f8e97-c9ab-4370-b437-8a47514c869c\") " pod="openshift-console-operator/console-operator-7866497494-k2ltl" Jul 12 08:21:51.404421 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.403477 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgj5s\" (UniqueName: \"kubernetes.io/projected/f8037f69-5405-41e4-9473-f2e22adbbb17-kube-api-access-pgj5s\") pod \"cluster-monitoring-operator-5f7f494dd4-kxpkk\" (UID: \"f8037f69-5405-41e4-9473-f2e22adbbb17\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kxpkk" Jul 12 08:21:51.405762 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.405741 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jddx\" (UniqueName: \"kubernetes.io/projected/29040aa3-9ad7-4c27-9ed5-fcc5339a5e28-kube-api-access-8jddx\") pod \"volume-data-source-validator-78495597f6-k9kgv\" (UID: \"29040aa3-9ad7-4c27-9ed5-fcc5339a5e28\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-k9kgv" Jul 12 08:21:51.407406 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.407366 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzrnv\" (UniqueName: \"kubernetes.io/projected/40be23cf-4432-4ea2-9979-3ae525ba3ef9-kube-api-access-xzrnv\") pod \"service-ca-operator-7695fc6c8c-4z9qg\" (UID: \"40be23cf-4432-4ea2-9979-3ae525ba3ef9\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4z9qg" Jul 12 08:21:51.407532 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.407506 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-bound-sa-token\") pod \"image-registry-5bd7886769-cpjl8\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:21:51.407802 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.407777 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2slb\" (UniqueName: \"kubernetes.io/projected/56a89c85-753b-4d5c-b012-a89e921d6400-kube-api-access-w2slb\") pod \"cluster-samples-operator-65db858894-94b94\" (UID: \"56a89c85-753b-4d5c-b012-a89e921d6400\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94b94" Jul 12 08:21:51.408053 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.408030 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzdjb\" (UniqueName: \"kubernetes.io/projected/e142390b-f773-488a-92de-6883d5f4ad70-kube-api-access-xzdjb\") pod \"insights-operator-68d9dd94cf-rvkmd\" (UID: \"e142390b-f773-488a-92de-6883d5f4ad70\") " pod="openshift-insights/insights-operator-68d9dd94cf-rvkmd" Jul 12 08:21:51.408213 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.408191 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79wcp\" (UniqueName: \"kubernetes.io/projected/bc3f8e97-c9ab-4370-b437-8a47514c869c-kube-api-access-79wcp\") pod \"console-operator-7866497494-k2ltl\" (UID: \"bc3f8e97-c9ab-4370-b437-8a47514c869c\") " pod="openshift-console-operator/console-operator-7866497494-k2ltl" Jul 12 08:21:51.408763 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.408742 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t444\" (UniqueName: \"kubernetes.io/projected/5cf8c8a8-825a-404d-b968-dfb4f4f466df-kube-api-access-4t444\") pod \"kube-storage-version-migrator-operator-799cb7f958-n9s6q\" (UID: \"5cf8c8a8-825a-404d-b968-dfb4f4f466df\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n9s6q" Jul 12 08:21:51.410028 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.410000 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh858\" (UniqueName: \"kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-kube-api-access-rh858\") pod \"image-registry-5bd7886769-cpjl8\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:21:51.467048 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.467011 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:21:51.492983 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.492950 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/088c2a49-9c89-44f2-b75a-68dbeda985a6-config-volume\") pod \"dns-default-b756k\" (UID: \"088c2a49-9c89-44f2-b75a-68dbeda985a6\") " pod="openshift-dns/dns-default-b756k" Jul 12 08:21:51.492983 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.492984 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bd2626f-14a5-450e-ae9a-6968c5aa9055-service-ca-bundle\") pod \"router-default-b8989f8c6-v5dgq\" (UID: \"1bd2626f-14a5-450e-ae9a-6968c5aa9055\") " pod="openshift-ingress/router-default-b8989f8c6-v5dgq" Jul 12 08:21:51.493497 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.493006 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/96c69912-dd41-420f-9c38-a7b64b14d50f-nginx-conf\") pod \"networking-console-plugin-6cf856758c-8t6pv\" (UID: \"96c69912-dd41-420f-9c38-a7b64b14d50f\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8t6pv" Jul 12 08:21:51.493497 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:51.493467 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1bd2626f-14a5-450e-ae9a-6968c5aa9055-service-ca-bundle podName:1bd2626f-14a5-450e-ae9a-6968c5aa9055 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:51.99344063 +0000 UTC m=+35.654236292 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1bd2626f-14a5-450e-ae9a-6968c5aa9055-service-ca-bundle") pod "router-default-b8989f8c6-v5dgq" (UID: "1bd2626f-14a5-450e-ae9a-6968c5aa9055") : configmap references non-existent config key: service-ca.crt Jul 12 08:21:51.493633 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.493500 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/088c2a49-9c89-44f2-b75a-68dbeda985a6-tmp-dir\") pod \"dns-default-b756k\" (UID: \"088c2a49-9c89-44f2-b75a-68dbeda985a6\") " pod="openshift-dns/dns-default-b756k" Jul 12 08:21:51.493633 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.493578 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4czjl\" (UniqueName: \"kubernetes.io/projected/373c0f0d-eabe-4ad9-af2c-51f5cc0e1c3d-kube-api-access-4czjl\") pod \"network-check-source-869dcfcdcd-b9fxd\" (UID: \"373c0f0d-eabe-4ad9-af2c-51f5cc0e1c3d\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-b9fxd" Jul 12 08:21:51.493633 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.493617 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkdhw\" (UniqueName: \"kubernetes.io/projected/f5e63ffc-280a-49f6-8861-6b5bcf16ac29-kube-api-access-nkdhw\") pod \"ingress-canary-4p4w4\" (UID: \"f5e63ffc-280a-49f6-8861-6b5bcf16ac29\") " pod="openshift-ingress-canary/ingress-canary-4p4w4" Jul 12 08:21:51.493777 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.493658 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgfdd\" (UniqueName: \"kubernetes.io/projected/088c2a49-9c89-44f2-b75a-68dbeda985a6-kube-api-access-rgfdd\") pod \"dns-default-b756k\" (UID: \"088c2a49-9c89-44f2-b75a-68dbeda985a6\") " pod="openshift-dns/dns-default-b756k" Jul 12 08:21:51.493777 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.493724 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bd2626f-14a5-450e-ae9a-6968c5aa9055-metrics-certs\") pod \"router-default-b8989f8c6-v5dgq\" (UID: \"1bd2626f-14a5-450e-ae9a-6968c5aa9055\") " pod="openshift-ingress/router-default-b8989f8c6-v5dgq" Jul 12 08:21:51.493777 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.493769 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/088c2a49-9c89-44f2-b75a-68dbeda985a6-metrics-tls\") pod \"dns-default-b756k\" (UID: \"088c2a49-9c89-44f2-b75a-68dbeda985a6\") " pod="openshift-dns/dns-default-b756k" Jul 12 08:21:51.494385 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.493851 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/088c2a49-9c89-44f2-b75a-68dbeda985a6-tmp-dir\") pod \"dns-default-b756k\" (UID: \"088c2a49-9c89-44f2-b75a-68dbeda985a6\") " pod="openshift-dns/dns-default-b756k" Jul 12 08:21:51.494385 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:51.493899 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 12 08:21:51.494385 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:51.493961 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bd2626f-14a5-450e-ae9a-6968c5aa9055-metrics-certs podName:1bd2626f-14a5-450e-ae9a-6968c5aa9055 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:51.993944241 +0000 UTC m=+35.654739881 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bd2626f-14a5-450e-ae9a-6968c5aa9055-metrics-certs") pod "router-default-b8989f8c6-v5dgq" (UID: "1bd2626f-14a5-450e-ae9a-6968c5aa9055") : secret "router-metrics-certs-default" not found Jul 12 08:21:51.494385 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:51.494041 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 08:21:51.494385 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:51.494108 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/088c2a49-9c89-44f2-b75a-68dbeda985a6-metrics-tls podName:088c2a49-9c89-44f2-b75a-68dbeda985a6 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:51.994093811 +0000 UTC m=+35.654889455 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/088c2a49-9c89-44f2-b75a-68dbeda985a6-metrics-tls") pod "dns-default-b756k" (UID: "088c2a49-9c89-44f2-b75a-68dbeda985a6") : secret "dns-default-metrics-tls" not found Jul 12 08:21:51.494385 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.494208 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/088c2a49-9c89-44f2-b75a-68dbeda985a6-config-volume\") pod \"dns-default-b756k\" (UID: \"088c2a49-9c89-44f2-b75a-68dbeda985a6\") " pod="openshift-dns/dns-default-b756k" Jul 12 08:21:51.494385 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.494236 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/1bd2626f-14a5-450e-ae9a-6968c5aa9055-default-certificate\") pod \"router-default-b8989f8c6-v5dgq\" (UID: \"1bd2626f-14a5-450e-ae9a-6968c5aa9055\") " pod="openshift-ingress/router-default-b8989f8c6-v5dgq" Jul 12 08:21:51.494385 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.494210 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/96c69912-dd41-420f-9c38-a7b64b14d50f-nginx-conf\") pod \"networking-console-plugin-6cf856758c-8t6pv\" (UID: \"96c69912-dd41-420f-9c38-a7b64b14d50f\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8t6pv" Jul 12 08:21:51.494385 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.494280 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/96c69912-dd41-420f-9c38-a7b64b14d50f-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8t6pv\" (UID: \"96c69912-dd41-420f-9c38-a7b64b14d50f\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8t6pv" Jul 12 08:21:51.494385 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.494332 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/1bd2626f-14a5-450e-ae9a-6968c5aa9055-stats-auth\") pod \"router-default-b8989f8c6-v5dgq\" (UID: \"1bd2626f-14a5-450e-ae9a-6968c5aa9055\") " pod="openshift-ingress/router-default-b8989f8c6-v5dgq" Jul 12 08:21:51.494385 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.494371 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7bnn\" (UniqueName: \"kubernetes.io/projected/1bd2626f-14a5-450e-ae9a-6968c5aa9055-kube-api-access-t7bnn\") pod \"router-default-b8989f8c6-v5dgq\" (UID: \"1bd2626f-14a5-450e-ae9a-6968c5aa9055\") " pod="openshift-ingress/router-default-b8989f8c6-v5dgq" Jul 12 08:21:51.495215 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.494417 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f5e63ffc-280a-49f6-8861-6b5bcf16ac29-cert\") pod \"ingress-canary-4p4w4\" (UID: \"f5e63ffc-280a-49f6-8861-6b5bcf16ac29\") " pod="openshift-ingress-canary/ingress-canary-4p4w4" Jul 12 08:21:51.495215 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:51.494593 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 08:21:51.495215 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:51.494656 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96c69912-dd41-420f-9c38-a7b64b14d50f-networking-console-plugin-cert podName:96c69912-dd41-420f-9c38-a7b64b14d50f nodeName:}" failed. No retries permitted until 2026-07-12 08:21:51.994635638 +0000 UTC m=+35.655431278 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/96c69912-dd41-420f-9c38-a7b64b14d50f-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-8t6pv" (UID: "96c69912-dd41-420f-9c38-a7b64b14d50f") : secret "networking-console-plugin-cert" not found Jul 12 08:21:51.498281 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.497481 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/1bd2626f-14a5-450e-ae9a-6968c5aa9055-default-certificate\") pod \"router-default-b8989f8c6-v5dgq\" (UID: \"1bd2626f-14a5-450e-ae9a-6968c5aa9055\") " pod="openshift-ingress/router-default-b8989f8c6-v5dgq" Jul 12 08:21:51.499265 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.499230 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-k9kgv" Jul 12 08:21:51.499498 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.499478 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/1bd2626f-14a5-450e-ae9a-6968c5aa9055-stats-auth\") pod \"router-default-b8989f8c6-v5dgq\" (UID: \"1bd2626f-14a5-450e-ae9a-6968c5aa9055\") " pod="openshift-ingress/router-default-b8989f8c6-v5dgq" Jul 12 08:21:51.500108 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.500091 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f5e63ffc-280a-49f6-8861-6b5bcf16ac29-cert\") pod \"ingress-canary-4p4w4\" (UID: \"f5e63ffc-280a-49f6-8861-6b5bcf16ac29\") " pod="openshift-ingress-canary/ingress-canary-4p4w4" Jul 12 08:21:51.506959 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.506937 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkdhw\" (UniqueName: \"kubernetes.io/projected/f5e63ffc-280a-49f6-8861-6b5bcf16ac29-kube-api-access-nkdhw\") pod \"ingress-canary-4p4w4\" (UID: \"f5e63ffc-280a-49f6-8861-6b5bcf16ac29\") " pod="openshift-ingress-canary/ingress-canary-4p4w4" Jul 12 08:21:51.507122 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.507102 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgfdd\" (UniqueName: \"kubernetes.io/projected/088c2a49-9c89-44f2-b75a-68dbeda985a6-kube-api-access-rgfdd\") pod \"dns-default-b756k\" (UID: \"088c2a49-9c89-44f2-b75a-68dbeda985a6\") " pod="openshift-dns/dns-default-b756k" Jul 12 08:21:51.507370 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.507344 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4czjl\" (UniqueName: \"kubernetes.io/projected/373c0f0d-eabe-4ad9-af2c-51f5cc0e1c3d-kube-api-access-4czjl\") pod \"network-check-source-869dcfcdcd-b9fxd\" (UID: \"373c0f0d-eabe-4ad9-af2c-51f5cc0e1c3d\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-b9fxd" Jul 12 08:21:51.509724 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.509703 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7bnn\" (UniqueName: \"kubernetes.io/projected/1bd2626f-14a5-450e-ae9a-6968c5aa9055-kube-api-access-t7bnn\") pod \"router-default-b8989f8c6-v5dgq\" (UID: \"1bd2626f-14a5-450e-ae9a-6968c5aa9055\") " pod="openshift-ingress/router-default-b8989f8c6-v5dgq" Jul 12 08:21:51.521883 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.521865 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4z9qg" Jul 12 08:21:51.542671 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.542644 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n9s6q" Jul 12 08:21:51.549385 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.549176 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-k2ltl" Jul 12 08:21:51.563694 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.563358 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-rvkmd" Jul 12 08:21:51.569846 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.569743 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-b9fxd" Jul 12 08:21:51.612229 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.609417 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4p4w4" Jul 12 08:21:51.703334 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.703221 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-k9kgv"] Jul 12 08:21:51.724051 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:51.723999 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29040aa3_9ad7_4c27_9ed5_fcc5339a5e28.slice/crio-a7497786f8ed7c9279ec92cb9edc012f16b77af2c00bbdc8e27a6357790d96ee WatchSource:0}: Error finding container a7497786f8ed7c9279ec92cb9edc012f16b77af2c00bbdc8e27a6357790d96ee: Status 404 returned error can't find the container with id a7497786f8ed7c9279ec92cb9edc012f16b77af2c00bbdc8e27a6357790d96ee Jul 12 08:21:51.748575 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.747887 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4z9qg"] Jul 12 08:21:51.754986 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:51.754949 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40be23cf_4432_4ea2_9979_3ae525ba3ef9.slice/crio-e9744fc6065281c1b08c4766ff613ee724856ea9bfb958fe6133ba7817846d0c WatchSource:0}: Error finding container e9744fc6065281c1b08c4766ff613ee724856ea9bfb958fe6133ba7817846d0c: Status 404 returned error can't find the container with id e9744fc6065281c1b08c4766ff613ee724856ea9bfb958fe6133ba7817846d0c Jul 12 08:21:51.792685 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.792483 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-k2ltl"] Jul 12 08:21:51.794422 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.794189 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n9s6q"] Jul 12 08:21:51.794626 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:51.794596 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc3f8e97_c9ab_4370_b437_8a47514c869c.slice/crio-f2753948238fc204cbcea853bbd41b0a536a797fb0e5f55981904f8089dc0361 WatchSource:0}: Error finding container f2753948238fc204cbcea853bbd41b0a536a797fb0e5f55981904f8089dc0361: Status 404 returned error can't find the container with id f2753948238fc204cbcea853bbd41b0a536a797fb0e5f55981904f8089dc0361 Jul 12 08:21:51.797737 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:51.797486 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cf8c8a8_825a_404d_b968_dfb4f4f466df.slice/crio-d309abea3ba767f2b760e34c2d3b89d478d71dd85cc8e9e66481c28f91757f1a WatchSource:0}: Error finding container d309abea3ba767f2b760e34c2d3b89d478d71dd85cc8e9e66481c28f91757f1a: Status 404 returned error can't find the container with id d309abea3ba767f2b760e34c2d3b89d478d71dd85cc8e9e66481c28f91757f1a Jul 12 08:21:51.798296 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.798272 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-b9fxd"] Jul 12 08:21:51.801444 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:51.801421 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod373c0f0d_eabe_4ad9_af2c_51f5cc0e1c3d.slice/crio-0a6abcaf9d399bd27fbc46ed7529639e90700eb4df3e3f5e95f31159ce9c6a86 WatchSource:0}: Error finding container 0a6abcaf9d399bd27fbc46ed7529639e90700eb4df3e3f5e95f31159ce9c6a86: Status 404 returned error can't find the container with id 0a6abcaf9d399bd27fbc46ed7529639e90700eb4df3e3f5e95f31159ce9c6a86 Jul 12 08:21:51.824251 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.824212 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-rvkmd"] Jul 12 08:21:51.827681 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:51.827656 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode142390b_f773_488a_92de_6883d5f4ad70.slice/crio-f7331e3db29d562cbf0bed6848f7ffb3b6cd4b80a8cd6005b13a7295cf05dffc WatchSource:0}: Error finding container f7331e3db29d562cbf0bed6848f7ffb3b6cd4b80a8cd6005b13a7295cf05dffc: Status 404 returned error can't find the container with id f7331e3db29d562cbf0bed6848f7ffb3b6cd4b80a8cd6005b13a7295cf05dffc Jul 12 08:21:51.838938 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.838918 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4p4w4"] Jul 12 08:21:51.841927 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:21:51.841898 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5e63ffc_280a_49f6_8861_6b5bcf16ac29.slice/crio-16af3438d057ebe9882c865b514838383007f8648321560cb7547c1827544ef5 WatchSource:0}: Error finding container 16af3438d057ebe9882c865b514838383007f8648321560cb7547c1827544ef5: Status 404 returned error can't find the container with id 16af3438d057ebe9882c865b514838383007f8648321560cb7547c1827544ef5 Jul 12 08:21:51.913235 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.913163 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-registry-tls\") pod \"image-registry-5bd7886769-cpjl8\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:21:51.913391 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.913264 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f8037f69-5405-41e4-9473-f2e22adbbb17-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-kxpkk\" (UID: \"f8037f69-5405-41e4-9473-f2e22adbbb17\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kxpkk" Jul 12 08:21:51.913391 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:51.913301 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/56a89c85-753b-4d5c-b012-a89e921d6400-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-94b94\" (UID: \"56a89c85-753b-4d5c-b012-a89e921d6400\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94b94" Jul 12 08:21:51.913391 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:51.913330 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 08:21:51.913391 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:51.913353 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bd7886769-cpjl8: secret "image-registry-tls" not found Jul 12 08:21:51.913611 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:51.913404 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-registry-tls podName:f98d0302-b975-4e38-a0d2-1f37eb5679f3 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:52.913388019 +0000 UTC m=+36.574183658 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-registry-tls") pod "image-registry-5bd7886769-cpjl8" (UID: "f98d0302-b975-4e38-a0d2-1f37eb5679f3") : secret "image-registry-tls" not found Jul 12 08:21:51.913611 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:51.913404 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 12 08:21:51.913611 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:51.913416 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 12 08:21:51.913611 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:51.913441 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8037f69-5405-41e4-9473-f2e22adbbb17-cluster-monitoring-operator-tls podName:f8037f69-5405-41e4-9473-f2e22adbbb17 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:52.91343234 +0000 UTC m=+36.574227979 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f8037f69-5405-41e4-9473-f2e22adbbb17-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-kxpkk" (UID: "f8037f69-5405-41e4-9473-f2e22adbbb17") : secret "cluster-monitoring-operator-tls" not found Jul 12 08:21:51.913611 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:51.913456 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56a89c85-753b-4d5c-b012-a89e921d6400-samples-operator-tls podName:56a89c85-753b-4d5c-b012-a89e921d6400 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:52.913449183 +0000 UTC m=+36.574244822 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/56a89c85-753b-4d5c-b012-a89e921d6400-samples-operator-tls") pod "cluster-samples-operator-65db858894-94b94" (UID: "56a89c85-753b-4d5c-b012-a89e921d6400") : secret "samples-operator-tls" not found Jul 12 08:21:52.014596 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:52.014568 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/96c69912-dd41-420f-9c38-a7b64b14d50f-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8t6pv\" (UID: \"96c69912-dd41-420f-9c38-a7b64b14d50f\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8t6pv" Jul 12 08:21:52.014811 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:52.014630 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bd2626f-14a5-450e-ae9a-6968c5aa9055-service-ca-bundle\") pod \"router-default-b8989f8c6-v5dgq\" (UID: \"1bd2626f-14a5-450e-ae9a-6968c5aa9055\") " pod="openshift-ingress/router-default-b8989f8c6-v5dgq" Jul 12 08:21:52.014811 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:52.014725 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bd2626f-14a5-450e-ae9a-6968c5aa9055-metrics-certs\") pod \"router-default-b8989f8c6-v5dgq\" (UID: \"1bd2626f-14a5-450e-ae9a-6968c5aa9055\") " pod="openshift-ingress/router-default-b8989f8c6-v5dgq" Jul 12 08:21:52.014811 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:52.014728 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 08:21:52.014811 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:52.014764 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1bd2626f-14a5-450e-ae9a-6968c5aa9055-service-ca-bundle podName:1bd2626f-14a5-450e-ae9a-6968c5aa9055 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:53.014743133 +0000 UTC m=+36.675538789 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1bd2626f-14a5-450e-ae9a-6968c5aa9055-service-ca-bundle") pod "router-default-b8989f8c6-v5dgq" (UID: "1bd2626f-14a5-450e-ae9a-6968c5aa9055") : configmap references non-existent config key: service-ca.crt Jul 12 08:21:52.015021 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:52.014818 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96c69912-dd41-420f-9c38-a7b64b14d50f-networking-console-plugin-cert podName:96c69912-dd41-420f-9c38-a7b64b14d50f nodeName:}" failed. No retries permitted until 2026-07-12 08:21:53.01479906 +0000 UTC m=+36.675594714 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/96c69912-dd41-420f-9c38-a7b64b14d50f-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-8t6pv" (UID: "96c69912-dd41-420f-9c38-a7b64b14d50f") : secret "networking-console-plugin-cert" not found Jul 12 08:21:52.015021 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:52.014841 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 12 08:21:52.015021 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:52.014855 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/088c2a49-9c89-44f2-b75a-68dbeda985a6-metrics-tls\") pod \"dns-default-b756k\" (UID: \"088c2a49-9c89-44f2-b75a-68dbeda985a6\") " pod="openshift-dns/dns-default-b756k" Jul 12 08:21:52.015021 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:52.014877 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bd2626f-14a5-450e-ae9a-6968c5aa9055-metrics-certs podName:1bd2626f-14a5-450e-ae9a-6968c5aa9055 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:53.014865633 +0000 UTC m=+36.675661295 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bd2626f-14a5-450e-ae9a-6968c5aa9055-metrics-certs") pod "router-default-b8989f8c6-v5dgq" (UID: "1bd2626f-14a5-450e-ae9a-6968c5aa9055") : secret "router-metrics-certs-default" not found Jul 12 08:21:52.015021 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:52.014915 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 08:21:52.015021 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:52.014977 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/088c2a49-9c89-44f2-b75a-68dbeda985a6-metrics-tls podName:088c2a49-9c89-44f2-b75a-68dbeda985a6 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:53.014969944 +0000 UTC m=+36.675765587 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/088c2a49-9c89-44f2-b75a-68dbeda985a6-metrics-tls") pod "dns-default-b756k" (UID: "088c2a49-9c89-44f2-b75a-68dbeda985a6") : secret "dns-default-metrics-tls" not found Jul 12 08:21:52.020225 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:52.020200 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ms85" Jul 12 08:21:52.020381 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:52.020360 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hr58s" Jul 12 08:21:52.020381 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:52.020380 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:21:52.022709 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:52.022586 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 12 08:21:52.022709 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:52.022596 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-d6mvs\"" Jul 12 08:21:52.022709 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:52.022617 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8cc5r\"" Jul 12 08:21:52.022709 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:52.022627 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 12 08:21:52.215035 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:52.214903 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4p4w4" event={"ID":"f5e63ffc-280a-49f6-8861-6b5bcf16ac29","Type":"ContainerStarted","Data":"16af3438d057ebe9882c865b514838383007f8648321560cb7547c1827544ef5"} Jul 12 08:21:52.217314 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:52.217284 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-rvkmd" event={"ID":"e142390b-f773-488a-92de-6883d5f4ad70","Type":"ContainerStarted","Data":"f7331e3db29d562cbf0bed6848f7ffb3b6cd4b80a8cd6005b13a7295cf05dffc"} Jul 12 08:21:52.219448 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:52.219372 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n9s6q" event={"ID":"5cf8c8a8-825a-404d-b968-dfb4f4f466df","Type":"ContainerStarted","Data":"d309abea3ba767f2b760e34c2d3b89d478d71dd85cc8e9e66481c28f91757f1a"} Jul 12 08:21:52.222045 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:52.221983 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4z9qg" event={"ID":"40be23cf-4432-4ea2-9979-3ae525ba3ef9","Type":"ContainerStarted","Data":"e9744fc6065281c1b08c4766ff613ee724856ea9bfb958fe6133ba7817846d0c"} Jul 12 08:21:52.225989 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:52.225957 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-b9fxd" event={"ID":"373c0f0d-eabe-4ad9-af2c-51f5cc0e1c3d","Type":"ContainerStarted","Data":"0a6abcaf9d399bd27fbc46ed7529639e90700eb4df3e3f5e95f31159ce9c6a86"} Jul 12 08:21:52.229995 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:52.229932 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-k2ltl" event={"ID":"bc3f8e97-c9ab-4370-b437-8a47514c869c","Type":"ContainerStarted","Data":"f2753948238fc204cbcea853bbd41b0a536a797fb0e5f55981904f8089dc0361"} Jul 12 08:21:52.236327 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:52.236302 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-k9kgv" event={"ID":"29040aa3-9ad7-4c27-9ed5-fcc5339a5e28","Type":"ContainerStarted","Data":"a7497786f8ed7c9279ec92cb9edc012f16b77af2c00bbdc8e27a6357790d96ee"} Jul 12 08:21:52.923528 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:52.923487 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/56a89c85-753b-4d5c-b012-a89e921d6400-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-94b94\" (UID: \"56a89c85-753b-4d5c-b012-a89e921d6400\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94b94" Jul 12 08:21:52.924044 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:52.923629 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-registry-tls\") pod \"image-registry-5bd7886769-cpjl8\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:21:52.924044 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:52.923743 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f8037f69-5405-41e4-9473-f2e22adbbb17-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-kxpkk\" (UID: \"f8037f69-5405-41e4-9473-f2e22adbbb17\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kxpkk" Jul 12 08:21:52.924044 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:52.923888 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 12 08:21:52.924044 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:52.923950 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8037f69-5405-41e4-9473-f2e22adbbb17-cluster-monitoring-operator-tls podName:f8037f69-5405-41e4-9473-f2e22adbbb17 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:54.923931479 +0000 UTC m=+38.584727132 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f8037f69-5405-41e4-9473-f2e22adbbb17-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-kxpkk" (UID: "f8037f69-5405-41e4-9473-f2e22adbbb17") : secret "cluster-monitoring-operator-tls" not found Jul 12 08:21:52.924478 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:52.924388 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 12 08:21:52.924478 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:52.924422 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 08:21:52.924478 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:52.924445 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bd7886769-cpjl8: secret "image-registry-tls" not found Jul 12 08:21:52.924478 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:52.924450 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56a89c85-753b-4d5c-b012-a89e921d6400-samples-operator-tls podName:56a89c85-753b-4d5c-b012-a89e921d6400 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:54.924434484 +0000 UTC m=+38.585230138 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/56a89c85-753b-4d5c-b012-a89e921d6400-samples-operator-tls") pod "cluster-samples-operator-65db858894-94b94" (UID: "56a89c85-753b-4d5c-b012-a89e921d6400") : secret "samples-operator-tls" not found Jul 12 08:21:52.924770 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:52.924499 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-registry-tls podName:f98d0302-b975-4e38-a0d2-1f37eb5679f3 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:54.924481208 +0000 UTC m=+38.585276865 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-registry-tls") pod "image-registry-5bd7886769-cpjl8" (UID: "f98d0302-b975-4e38-a0d2-1f37eb5679f3") : secret "image-registry-tls" not found Jul 12 08:21:53.024577 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:53.024147 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bd2626f-14a5-450e-ae9a-6968c5aa9055-service-ca-bundle\") pod \"router-default-b8989f8c6-v5dgq\" (UID: \"1bd2626f-14a5-450e-ae9a-6968c5aa9055\") " pod="openshift-ingress/router-default-b8989f8c6-v5dgq" Jul 12 08:21:53.024577 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:53.024261 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bd2626f-14a5-450e-ae9a-6968c5aa9055-metrics-certs\") pod \"router-default-b8989f8c6-v5dgq\" (UID: \"1bd2626f-14a5-450e-ae9a-6968c5aa9055\") " pod="openshift-ingress/router-default-b8989f8c6-v5dgq" Jul 12 08:21:53.024577 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:53.024297 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/088c2a49-9c89-44f2-b75a-68dbeda985a6-metrics-tls\") pod \"dns-default-b756k\" (UID: \"088c2a49-9c89-44f2-b75a-68dbeda985a6\") " pod="openshift-dns/dns-default-b756k" Jul 12 08:21:53.024577 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:53.024354 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/96c69912-dd41-420f-9c38-a7b64b14d50f-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8t6pv\" (UID: \"96c69912-dd41-420f-9c38-a7b64b14d50f\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8t6pv" Jul 12 08:21:53.024577 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:53.024523 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 08:21:53.024989 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:53.024741 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 12 08:21:53.024989 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:53.024809 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bd2626f-14a5-450e-ae9a-6968c5aa9055-metrics-certs podName:1bd2626f-14a5-450e-ae9a-6968c5aa9055 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:55.024789607 +0000 UTC m=+38.685585268 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bd2626f-14a5-450e-ae9a-6968c5aa9055-metrics-certs") pod "router-default-b8989f8c6-v5dgq" (UID: "1bd2626f-14a5-450e-ae9a-6968c5aa9055") : secret "router-metrics-certs-default" not found Jul 12 08:21:53.025245 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:53.025179 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 08:21:53.025245 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:53.025200 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1bd2626f-14a5-450e-ae9a-6968c5aa9055-service-ca-bundle podName:1bd2626f-14a5-450e-ae9a-6968c5aa9055 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:55.025181632 +0000 UTC m=+38.685977279 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1bd2626f-14a5-450e-ae9a-6968c5aa9055-service-ca-bundle") pod "router-default-b8989f8c6-v5dgq" (UID: "1bd2626f-14a5-450e-ae9a-6968c5aa9055") : configmap references non-existent config key: service-ca.crt Jul 12 08:21:53.025245 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:53.025226 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/088c2a49-9c89-44f2-b75a-68dbeda985a6-metrics-tls podName:088c2a49-9c89-44f2-b75a-68dbeda985a6 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:55.025216442 +0000 UTC m=+38.686012084 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/088c2a49-9c89-44f2-b75a-68dbeda985a6-metrics-tls") pod "dns-default-b756k" (UID: "088c2a49-9c89-44f2-b75a-68dbeda985a6") : secret "dns-default-metrics-tls" not found Jul 12 08:21:53.025476 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:53.025453 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96c69912-dd41-420f-9c38-a7b64b14d50f-networking-console-plugin-cert podName:96c69912-dd41-420f-9c38-a7b64b14d50f nodeName:}" failed. No retries permitted until 2026-07-12 08:21:55.025439542 +0000 UTC m=+38.686235181 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/96c69912-dd41-420f-9c38-a7b64b14d50f-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-8t6pv" (UID: "96c69912-dd41-420f-9c38-a7b64b14d50f") : secret "networking-console-plugin-cert" not found Jul 12 08:21:54.363320 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:54.363278 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:21:54.417435 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:54.417393 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:21:54.942559 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:54.942513 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-registry-tls\") pod \"image-registry-5bd7886769-cpjl8\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:21:54.942728 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:54.942620 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f8037f69-5405-41e4-9473-f2e22adbbb17-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-kxpkk\" (UID: \"f8037f69-5405-41e4-9473-f2e22adbbb17\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kxpkk" Jul 12 08:21:54.942728 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:54.942658 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/56a89c85-753b-4d5c-b012-a89e921d6400-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-94b94\" (UID: \"56a89c85-753b-4d5c-b012-a89e921d6400\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94b94" Jul 12 08:21:54.942728 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:54.942656 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 08:21:54.942728 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:54.942710 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 12 08:21:54.942728 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:54.942719 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bd7886769-cpjl8: secret "image-registry-tls" not found Jul 12 08:21:54.942977 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:54.942761 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 12 08:21:54.942977 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:54.942773 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-registry-tls podName:f98d0302-b975-4e38-a0d2-1f37eb5679f3 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:58.942755246 +0000 UTC m=+42.603550887 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-registry-tls") pod "image-registry-5bd7886769-cpjl8" (UID: "f98d0302-b975-4e38-a0d2-1f37eb5679f3") : secret "image-registry-tls" not found Jul 12 08:21:54.942977 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:54.942791 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8037f69-5405-41e4-9473-f2e22adbbb17-cluster-monitoring-operator-tls podName:f8037f69-5405-41e4-9473-f2e22adbbb17 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:58.942781363 +0000 UTC m=+42.603577002 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f8037f69-5405-41e4-9473-f2e22adbbb17-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-kxpkk" (UID: "f8037f69-5405-41e4-9473-f2e22adbbb17") : secret "cluster-monitoring-operator-tls" not found Jul 12 08:21:54.942977 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:54.942805 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56a89c85-753b-4d5c-b012-a89e921d6400-samples-operator-tls podName:56a89c85-753b-4d5c-b012-a89e921d6400 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:58.942797919 +0000 UTC m=+42.603593562 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/56a89c85-753b-4d5c-b012-a89e921d6400-samples-operator-tls") pod "cluster-samples-operator-65db858894-94b94" (UID: "56a89c85-753b-4d5c-b012-a89e921d6400") : secret "samples-operator-tls" not found Jul 12 08:21:55.043310 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:55.043270 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bd2626f-14a5-450e-ae9a-6968c5aa9055-service-ca-bundle\") pod \"router-default-b8989f8c6-v5dgq\" (UID: \"1bd2626f-14a5-450e-ae9a-6968c5aa9055\") " pod="openshift-ingress/router-default-b8989f8c6-v5dgq" Jul 12 08:21:55.043486 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:55.043458 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1bd2626f-14a5-450e-ae9a-6968c5aa9055-service-ca-bundle podName:1bd2626f-14a5-450e-ae9a-6968c5aa9055 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:59.04343427 +0000 UTC m=+42.704229915 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1bd2626f-14a5-450e-ae9a-6968c5aa9055-service-ca-bundle") pod "router-default-b8989f8c6-v5dgq" (UID: "1bd2626f-14a5-450e-ae9a-6968c5aa9055") : configmap references non-existent config key: service-ca.crt Jul 12 08:21:55.043604 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:55.043512 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bd2626f-14a5-450e-ae9a-6968c5aa9055-metrics-certs\") pod \"router-default-b8989f8c6-v5dgq\" (UID: \"1bd2626f-14a5-450e-ae9a-6968c5aa9055\") " pod="openshift-ingress/router-default-b8989f8c6-v5dgq" Jul 12 08:21:55.043604 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:55.043572 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/088c2a49-9c89-44f2-b75a-68dbeda985a6-metrics-tls\") pod \"dns-default-b756k\" (UID: \"088c2a49-9c89-44f2-b75a-68dbeda985a6\") " pod="openshift-dns/dns-default-b756k" Jul 12 08:21:55.043718 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:55.043631 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/96c69912-dd41-420f-9c38-a7b64b14d50f-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8t6pv\" (UID: \"96c69912-dd41-420f-9c38-a7b64b14d50f\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8t6pv" Jul 12 08:21:55.043768 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:55.043738 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 08:21:55.043812 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:55.043778 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96c69912-dd41-420f-9c38-a7b64b14d50f-networking-console-plugin-cert podName:96c69912-dd41-420f-9c38-a7b64b14d50f nodeName:}" failed. No retries permitted until 2026-07-12 08:21:59.043764878 +0000 UTC m=+42.704560522 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/96c69912-dd41-420f-9c38-a7b64b14d50f-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-8t6pv" (UID: "96c69912-dd41-420f-9c38-a7b64b14d50f") : secret "networking-console-plugin-cert" not found Jul 12 08:21:55.043867 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:55.043832 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 08:21:55.043867 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:55.043841 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 12 08:21:55.043867 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:55.043861 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/088c2a49-9c89-44f2-b75a-68dbeda985a6-metrics-tls podName:088c2a49-9c89-44f2-b75a-68dbeda985a6 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:59.043850151 +0000 UTC m=+42.704645791 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/088c2a49-9c89-44f2-b75a-68dbeda985a6-metrics-tls") pod "dns-default-b756k" (UID: "088c2a49-9c89-44f2-b75a-68dbeda985a6") : secret "dns-default-metrics-tls" not found Jul 12 08:21:55.044004 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:55.043897 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bd2626f-14a5-450e-ae9a-6968c5aa9055-metrics-certs podName:1bd2626f-14a5-450e-ae9a-6968c5aa9055 nodeName:}" failed. No retries permitted until 2026-07-12 08:21:59.043879717 +0000 UTC m=+42.704675373 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bd2626f-14a5-450e-ae9a-6968c5aa9055-metrics-certs") pod "router-default-b8989f8c6-v5dgq" (UID: "1bd2626f-14a5-450e-ae9a-6968c5aa9055") : secret "router-metrics-certs-default" not found Jul 12 08:21:56.720747 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:56.720508 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:21:56.820127 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:56.820089 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:21:58.978559 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:58.978517 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f8037f69-5405-41e4-9473-f2e22adbbb17-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-kxpkk\" (UID: \"f8037f69-5405-41e4-9473-f2e22adbbb17\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kxpkk" Jul 12 08:21:58.978931 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:58.978590 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/56a89c85-753b-4d5c-b012-a89e921d6400-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-94b94\" (UID: \"56a89c85-753b-4d5c-b012-a89e921d6400\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94b94" Jul 12 08:21:58.978931 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:58.978718 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 12 08:21:58.978931 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:58.978752 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-registry-tls\") pod \"image-registry-5bd7886769-cpjl8\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:21:58.978931 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:58.978715 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 12 08:21:58.978931 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:58.978781 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56a89c85-753b-4d5c-b012-a89e921d6400-samples-operator-tls podName:56a89c85-753b-4d5c-b012-a89e921d6400 nodeName:}" failed. No retries permitted until 2026-07-12 08:22:06.978763074 +0000 UTC m=+50.639558715 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/56a89c85-753b-4d5c-b012-a89e921d6400-samples-operator-tls") pod "cluster-samples-operator-65db858894-94b94" (UID: "56a89c85-753b-4d5c-b012-a89e921d6400") : secret "samples-operator-tls" not found Jul 12 08:21:58.978931 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:58.978825 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8037f69-5405-41e4-9473-f2e22adbbb17-cluster-monitoring-operator-tls podName:f8037f69-5405-41e4-9473-f2e22adbbb17 nodeName:}" failed. No retries permitted until 2026-07-12 08:22:06.97880319 +0000 UTC m=+50.639598853 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f8037f69-5405-41e4-9473-f2e22adbbb17-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-kxpkk" (UID: "f8037f69-5405-41e4-9473-f2e22adbbb17") : secret "cluster-monitoring-operator-tls" not found Jul 12 08:21:58.978931 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:58.978851 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 08:21:58.978931 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:58.978865 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bd7886769-cpjl8: secret "image-registry-tls" not found Jul 12 08:21:58.978931 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:58.978917 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-registry-tls podName:f98d0302-b975-4e38-a0d2-1f37eb5679f3 nodeName:}" failed. No retries permitted until 2026-07-12 08:22:06.978902267 +0000 UTC m=+50.639697923 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-registry-tls") pod "image-registry-5bd7886769-cpjl8" (UID: "f98d0302-b975-4e38-a0d2-1f37eb5679f3") : secret "image-registry-tls" not found Jul 12 08:21:59.079777 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:59.079744 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bd2626f-14a5-450e-ae9a-6968c5aa9055-service-ca-bundle\") pod \"router-default-b8989f8c6-v5dgq\" (UID: \"1bd2626f-14a5-450e-ae9a-6968c5aa9055\") " pod="openshift-ingress/router-default-b8989f8c6-v5dgq" Jul 12 08:21:59.079965 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:59.079854 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bd2626f-14a5-450e-ae9a-6968c5aa9055-metrics-certs\") pod \"router-default-b8989f8c6-v5dgq\" (UID: \"1bd2626f-14a5-450e-ae9a-6968c5aa9055\") " pod="openshift-ingress/router-default-b8989f8c6-v5dgq" Jul 12 08:21:59.079965 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:59.079877 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/088c2a49-9c89-44f2-b75a-68dbeda985a6-metrics-tls\") pod \"dns-default-b756k\" (UID: \"088c2a49-9c89-44f2-b75a-68dbeda985a6\") " pod="openshift-dns/dns-default-b756k" Jul 12 08:21:59.079965 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:59.079937 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1bd2626f-14a5-450e-ae9a-6968c5aa9055-service-ca-bundle podName:1bd2626f-14a5-450e-ae9a-6968c5aa9055 nodeName:}" failed. No retries permitted until 2026-07-12 08:22:07.079914034 +0000 UTC m=+50.740709675 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1bd2626f-14a5-450e-ae9a-6968c5aa9055-service-ca-bundle") pod "router-default-b8989f8c6-v5dgq" (UID: "1bd2626f-14a5-450e-ae9a-6968c5aa9055") : configmap references non-existent config key: service-ca.crt Jul 12 08:21:59.080129 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:59.079975 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/96c69912-dd41-420f-9c38-a7b64b14d50f-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8t6pv\" (UID: \"96c69912-dd41-420f-9c38-a7b64b14d50f\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8t6pv" Jul 12 08:21:59.080129 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:59.079993 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 08:21:59.080129 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:59.080011 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 12 08:21:59.080129 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:59.080063 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/088c2a49-9c89-44f2-b75a-68dbeda985a6-metrics-tls podName:088c2a49-9c89-44f2-b75a-68dbeda985a6 nodeName:}" failed. No retries permitted until 2026-07-12 08:22:07.080049134 +0000 UTC m=+50.740844773 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/088c2a49-9c89-44f2-b75a-68dbeda985a6-metrics-tls") pod "dns-default-b756k" (UID: "088c2a49-9c89-44f2-b75a-68dbeda985a6") : secret "dns-default-metrics-tls" not found Jul 12 08:21:59.080129 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:59.080080 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bd2626f-14a5-450e-ae9a-6968c5aa9055-metrics-certs podName:1bd2626f-14a5-450e-ae9a-6968c5aa9055 nodeName:}" failed. No retries permitted until 2026-07-12 08:22:07.080071304 +0000 UTC m=+50.740866947 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bd2626f-14a5-450e-ae9a-6968c5aa9055-metrics-certs") pod "router-default-b8989f8c6-v5dgq" (UID: "1bd2626f-14a5-450e-ae9a-6968c5aa9055") : secret "router-metrics-certs-default" not found Jul 12 08:21:59.080129 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:59.080115 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 08:21:59.080407 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:21:59.080152 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96c69912-dd41-420f-9c38-a7b64b14d50f-networking-console-plugin-cert podName:96c69912-dd41-420f-9c38-a7b64b14d50f nodeName:}" failed. No retries permitted until 2026-07-12 08:22:07.080140434 +0000 UTC m=+50.740936077 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/96c69912-dd41-420f-9c38-a7b64b14d50f-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-8t6pv" (UID: "96c69912-dd41-420f-9c38-a7b64b14d50f") : secret "networking-console-plugin-cert" not found Jul 12 08:21:59.200667 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:59.200582 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:21:59.301807 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:21:59.301769 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:00.575105 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:00.575046 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:00.675705 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:00.675623 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:01.267984 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.267946 2566 generic.go:356] "Generic (PLEG): container finished" podID="a69a78ba-31f4-488d-a7a5-0d2a197ef775" containerID="45e21b08b95ec89169d60497ea09897ff5ff8773efdc65b022045812d97e3159" exitCode=0 Jul 12 08:22:01.268161 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.268063 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xvhw4" event={"ID":"a69a78ba-31f4-488d-a7a5-0d2a197ef775","Type":"ContainerDied","Data":"45e21b08b95ec89169d60497ea09897ff5ff8773efdc65b022045812d97e3159"} Jul 12 08:22:01.269645 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.269617 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4p4w4" event={"ID":"f5e63ffc-280a-49f6-8861-6b5bcf16ac29","Type":"ContainerStarted","Data":"602e9b32c358847df1a88d26d9d1d9449a4db147629038b7ef35570fd6a244f6"} Jul 12 08:22:01.271643 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.271618 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-rvkmd" event={"ID":"e142390b-f773-488a-92de-6883d5f4ad70","Type":"ContainerStarted","Data":"7fc22fdfb89d8d9b8b484653cdf945a00ecc41aefa91320d277109ef560b7a52"} Jul 12 08:22:01.273135 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.273115 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n9s6q" event={"ID":"5cf8c8a8-825a-404d-b968-dfb4f4f466df","Type":"ContainerStarted","Data":"487de75e0d2c54697dc41019aeca0c18b1e8356bd42097b45853a1be779ae041"} Jul 12 08:22:01.275045 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.274995 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4z9qg" event={"ID":"40be23cf-4432-4ea2-9979-3ae525ba3ef9","Type":"ContainerStarted","Data":"2a3c25fa32c316b5b1ee68883eb403fafbf9b65f19067370b0a7c19ef4a7f2df"} Jul 12 08:22:01.277200 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.277173 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-b9fxd" event={"ID":"373c0f0d-eabe-4ad9-af2c-51f5cc0e1c3d","Type":"ContainerStarted","Data":"d5271a247b5feb8353ee9d0e282635c608f18bd89714ddbfbb01a23a418342c0"} Jul 12 08:22:01.278839 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.278821 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-k2ltl_bc3f8e97-c9ab-4370-b437-8a47514c869c/console-operator/0.log" Jul 12 08:22:01.278936 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.278858 2566 generic.go:356] "Generic (PLEG): container finished" podID="bc3f8e97-c9ab-4370-b437-8a47514c869c" containerID="c55bef8deb98e06ab5958af5282d59c249f20cb8a6a232c04ea687c00b392995" exitCode=255 Jul 12 08:22:01.278936 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.278925 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-k2ltl" event={"ID":"bc3f8e97-c9ab-4370-b437-8a47514c869c","Type":"ContainerDied","Data":"c55bef8deb98e06ab5958af5282d59c249f20cb8a6a232c04ea687c00b392995"} Jul 12 08:22:01.279119 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.279096 2566 scope.go:117] "RemoveContainer" containerID="c55bef8deb98e06ab5958af5282d59c249f20cb8a6a232c04ea687c00b392995" Jul 12 08:22:01.280741 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.280693 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-k9kgv" event={"ID":"29040aa3-9ad7-4c27-9ed5-fcc5339a5e28","Type":"ContainerStarted","Data":"46e6f551577502df9f50b516cb3d99eaf6b6d1eecca20549e9d869a44bcbaeaf"} Jul 12 08:22:01.303466 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.303421 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4z9qg" podStartSLOduration=34.738460431 podStartE2EDuration="43.303404053s" totalImagesPullingTime="8.564943622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:21:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:22:01.302215335 +0000 UTC m=+44.963011012" watchObservedRunningTime="2026-07-12 08:22:01.303404053 +0000 UTC m=+44.964199720" Jul 12 08:22:01.317368 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.317323 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n9s6q" podStartSLOduration=34.791817223 podStartE2EDuration="43.317309498s" totalImagesPullingTime="8.525492275s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:21:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:22:01.316952456 +0000 UTC m=+44.977748117" watchObservedRunningTime="2026-07-12 08:22:01.317309498 +0000 UTC m=+44.978105142" Jul 12 08:22:01.350381 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.350338 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-rvkmd" podStartSLOduration=34.81992616 podStartE2EDuration="43.350306902s" totalImagesPullingTime="8.530380742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:21:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:22:01.348611931 +0000 UTC m=+45.009407590" watchObservedRunningTime="2026-07-12 08:22:01.350306902 +0000 UTC m=+45.011102565" Jul 12 08:22:01.364191 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.363527 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4p4w4" podStartSLOduration=1.882552049 podStartE2EDuration="10.363508768s" totalImagesPullingTime="8.480956719s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:21:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:22:01.362228653 +0000 UTC m=+45.023024330" watchObservedRunningTime="2026-07-12 08:22:01.363508768 +0000 UTC m=+45.024304431" Jul 12 08:22:01.399430 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.399238 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-k9kgv" podStartSLOduration=34.802420928 podStartE2EDuration="43.399218365s" totalImagesPullingTime="8.596797437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:21:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:22:01.398985746 +0000 UTC m=+45.059781403" watchObservedRunningTime="2026-07-12 08:22:01.399218365 +0000 UTC m=+45.060014020" Jul 12 08:22:01.399430 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.399331 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-b9fxd" podStartSLOduration=34.878233434 podStartE2EDuration="43.399324859s" totalImagesPullingTime="8.521091425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:21:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:22:01.381000564 +0000 UTC m=+45.041796225" watchObservedRunningTime="2026-07-12 08:22:01.399324859 +0000 UTC m=+45.060120520" Jul 12 08:22:01.496193 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.496041 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:01.549738 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.549711 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-k2ltl" Jul 12 08:22:01.549836 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.549760 2566 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-k2ltl" Jul 12 08:22:01.610271 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.610231 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:01.662177 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.662137 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:01.842937 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.842854 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-98qmc"] Jul 12 08:22:01.871142 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.871108 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:01.880739 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.880711 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-98qmc"] Jul 12 08:22:01.880855 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.880836 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-98qmc" Jul 12 08:22:01.883034 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.883010 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 12 08:22:01.883139 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.883123 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 12 08:22:01.883505 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.883302 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qlrjx\"" Jul 12 08:22:01.906060 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.906037 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-98qmc\" (UID: \"636c08ec-7cba-4f5e-a5b5-4db9f35be60a\") " pod="openshift-insights/insights-runtime-extractor-98qmc" Jul 12 08:22:01.906161 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.906079 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbfvx\" (UniqueName: \"kubernetes.io/projected/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-kube-api-access-bbfvx\") pod \"insights-runtime-extractor-98qmc\" (UID: \"636c08ec-7cba-4f5e-a5b5-4db9f35be60a\") " pod="openshift-insights/insights-runtime-extractor-98qmc" Jul 12 08:22:01.906161 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.906144 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-data-volume\") pod \"insights-runtime-extractor-98qmc\" (UID: \"636c08ec-7cba-4f5e-a5b5-4db9f35be60a\") " pod="openshift-insights/insights-runtime-extractor-98qmc" Jul 12 08:22:01.906256 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.906168 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-98qmc\" (UID: \"636c08ec-7cba-4f5e-a5b5-4db9f35be60a\") " pod="openshift-insights/insights-runtime-extractor-98qmc" Jul 12 08:22:01.906309 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.906259 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-crio-socket\") pod \"insights-runtime-extractor-98qmc\" (UID: \"636c08ec-7cba-4f5e-a5b5-4db9f35be60a\") " pod="openshift-insights/insights-runtime-extractor-98qmc" Jul 12 08:22:01.926472 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:01.926445 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:02.006793 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:02.006764 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-crio-socket\") pod \"insights-runtime-extractor-98qmc\" (UID: \"636c08ec-7cba-4f5e-a5b5-4db9f35be60a\") " pod="openshift-insights/insights-runtime-extractor-98qmc" Jul 12 08:22:02.006935 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:02.006811 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-98qmc\" (UID: \"636c08ec-7cba-4f5e-a5b5-4db9f35be60a\") " pod="openshift-insights/insights-runtime-extractor-98qmc" Jul 12 08:22:02.006935 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:02.006854 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbfvx\" (UniqueName: \"kubernetes.io/projected/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-kube-api-access-bbfvx\") pod \"insights-runtime-extractor-98qmc\" (UID: \"636c08ec-7cba-4f5e-a5b5-4db9f35be60a\") " pod="openshift-insights/insights-runtime-extractor-98qmc" Jul 12 08:22:02.006935 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:02.006913 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-data-volume\") pod \"insights-runtime-extractor-98qmc\" (UID: \"636c08ec-7cba-4f5e-a5b5-4db9f35be60a\") " pod="openshift-insights/insights-runtime-extractor-98qmc" Jul 12 08:22:02.007087 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:02.006944 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-98qmc\" (UID: \"636c08ec-7cba-4f5e-a5b5-4db9f35be60a\") " pod="openshift-insights/insights-runtime-extractor-98qmc" Jul 12 08:22:02.007087 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:22:02.006978 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 08:22:02.007087 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:02.007007 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-crio-socket\") pod \"insights-runtime-extractor-98qmc\" (UID: \"636c08ec-7cba-4f5e-a5b5-4db9f35be60a\") " pod="openshift-insights/insights-runtime-extractor-98qmc" Jul 12 08:22:02.007087 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:22:02.007045 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-kube-rbac-proxy-cm podName:636c08ec-7cba-4f5e-a5b5-4db9f35be60a nodeName:}" failed. No retries permitted until 2026-07-12 08:22:02.507024646 +0000 UTC m=+46.167820288 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-98qmc" (UID: "636c08ec-7cba-4f5e-a5b5-4db9f35be60a") : configmap "kube-rbac-proxy" not found Jul 12 08:22:02.007276 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:22:02.007100 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 08:22:02.007276 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:22:02.007153 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-insights-runtime-extractor-tls podName:636c08ec-7cba-4f5e-a5b5-4db9f35be60a nodeName:}" failed. No retries permitted until 2026-07-12 08:22:02.507136785 +0000 UTC m=+46.167932432 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-98qmc" (UID: "636c08ec-7cba-4f5e-a5b5-4db9f35be60a") : secret "insights-runtime-extractor-tls" not found Jul 12 08:22:02.007348 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:02.007275 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-data-volume\") pod \"insights-runtime-extractor-98qmc\" (UID: \"636c08ec-7cba-4f5e-a5b5-4db9f35be60a\") " pod="openshift-insights/insights-runtime-extractor-98qmc" Jul 12 08:22:02.016013 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:02.015989 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbfvx\" (UniqueName: \"kubernetes.io/projected/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-kube-api-access-bbfvx\") pod \"insights-runtime-extractor-98qmc\" (UID: \"636c08ec-7cba-4f5e-a5b5-4db9f35be60a\") " pod="openshift-insights/insights-runtime-extractor-98qmc" Jul 12 08:22:02.287345 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:02.287308 2566 generic.go:356] "Generic (PLEG): container finished" podID="a69a78ba-31f4-488d-a7a5-0d2a197ef775" containerID="9ef538929927b8d142e0dcbc403680f9e1120c1567cceb0e1aab29a1fdcbce53" exitCode=0 Jul 12 08:22:02.287524 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:02.287394 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xvhw4" event={"ID":"a69a78ba-31f4-488d-a7a5-0d2a197ef775","Type":"ContainerDied","Data":"9ef538929927b8d142e0dcbc403680f9e1120c1567cceb0e1aab29a1fdcbce53"} Jul 12 08:22:02.289176 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:02.289154 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-k2ltl_bc3f8e97-c9ab-4370-b437-8a47514c869c/console-operator/1.log" Jul 12 08:22:02.289580 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:02.289565 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-k2ltl_bc3f8e97-c9ab-4370-b437-8a47514c869c/console-operator/0.log" Jul 12 08:22:02.289650 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:02.289596 2566 generic.go:356] "Generic (PLEG): container finished" podID="bc3f8e97-c9ab-4370-b437-8a47514c869c" containerID="18aa076c4b23d23244d435411d5bff8063d00bec01156f9e00e96ed40324dbc5" exitCode=255 Jul 12 08:22:02.289650 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:02.289641 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-k2ltl" event={"ID":"bc3f8e97-c9ab-4370-b437-8a47514c869c","Type":"ContainerDied","Data":"18aa076c4b23d23244d435411d5bff8063d00bec01156f9e00e96ed40324dbc5"} Jul 12 08:22:02.289730 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:02.289680 2566 scope.go:117] "RemoveContainer" containerID="c55bef8deb98e06ab5958af5282d59c249f20cb8a6a232c04ea687c00b392995" Jul 12 08:22:02.289837 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:02.289817 2566 scope.go:117] "RemoveContainer" containerID="18aa076c4b23d23244d435411d5bff8063d00bec01156f9e00e96ed40324dbc5" Jul 12 08:22:02.289989 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:22:02.289967 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-k2ltl_openshift-console-operator(bc3f8e97-c9ab-4370-b437-8a47514c869c)\"" pod="openshift-console-operator/console-operator-7866497494-k2ltl" podUID="bc3f8e97-c9ab-4370-b437-8a47514c869c" Jul 12 08:22:02.428992 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:02.428955 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:02.460400 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:02.460377 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6l8ww"] Jul 12 08:22:02.473192 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:02.473172 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6l8ww" Jul 12 08:22:02.475087 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:02.475066 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6l8ww"] Jul 12 08:22:02.475473 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:02.475451 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 12 08:22:02.475737 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:02.475718 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 12 08:22:02.475826 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:02.475721 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-tkj94\"" Jul 12 08:22:02.512031 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:02.512004 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-98qmc\" (UID: \"636c08ec-7cba-4f5e-a5b5-4db9f35be60a\") " pod="openshift-insights/insights-runtime-extractor-98qmc" Jul 12 08:22:02.512131 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:02.512122 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-98qmc\" (UID: \"636c08ec-7cba-4f5e-a5b5-4db9f35be60a\") " pod="openshift-insights/insights-runtime-extractor-98qmc" Jul 12 08:22:02.512170 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:22:02.512140 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 08:22:02.512215 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:02.512193 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlxdl\" (UniqueName: \"kubernetes.io/projected/55cb2ce1-ea8b-4fc7-9a14-7280c499cf67-kube-api-access-nlxdl\") pod \"migrator-7f59b54f6c-6l8ww\" (UID: \"55cb2ce1-ea8b-4fc7-9a14-7280c499cf67\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6l8ww" Jul 12 08:22:02.512215 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:22:02.512205 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-insights-runtime-extractor-tls podName:636c08ec-7cba-4f5e-a5b5-4db9f35be60a nodeName:}" failed. No retries permitted until 2026-07-12 08:22:03.512178437 +0000 UTC m=+47.172974078 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-98qmc" (UID: "636c08ec-7cba-4f5e-a5b5-4db9f35be60a") : secret "insights-runtime-extractor-tls" not found Jul 12 08:22:02.512300 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:22:02.512234 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 08:22:02.512300 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:22:02.512292 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-kube-rbac-proxy-cm podName:636c08ec-7cba-4f5e-a5b5-4db9f35be60a nodeName:}" failed. No retries permitted until 2026-07-12 08:22:03.512262775 +0000 UTC m=+47.173058415 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-98qmc" (UID: "636c08ec-7cba-4f5e-a5b5-4db9f35be60a") : configmap "kube-rbac-proxy" not found Jul 12 08:22:02.525249 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:02.525223 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:02.612934 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:02.612845 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlxdl\" (UniqueName: \"kubernetes.io/projected/55cb2ce1-ea8b-4fc7-9a14-7280c499cf67-kube-api-access-nlxdl\") pod \"migrator-7f59b54f6c-6l8ww\" (UID: \"55cb2ce1-ea8b-4fc7-9a14-7280c499cf67\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6l8ww" Jul 12 08:22:02.620335 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:02.620310 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlxdl\" (UniqueName: \"kubernetes.io/projected/55cb2ce1-ea8b-4fc7-9a14-7280c499cf67-kube-api-access-nlxdl\") pod \"migrator-7f59b54f6c-6l8ww\" (UID: \"55cb2ce1-ea8b-4fc7-9a14-7280c499cf67\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6l8ww" Jul 12 08:22:02.625992 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:02.625965 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:02.783752 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:02.783720 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6l8ww" Jul 12 08:22:02.922413 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:02.922384 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:02.930781 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:02.930760 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6l8ww"] Jul 12 08:22:02.933032 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:22:02.933005 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55cb2ce1_ea8b_4fc7_9a14_7280c499cf67.slice/crio-e11a4206cec5fa0956a8928ceeb405d75cb30f51351ab2b7b3bbf860eda27e7a WatchSource:0}: Error finding container e11a4206cec5fa0956a8928ceeb405d75cb30f51351ab2b7b3bbf860eda27e7a: Status 404 returned error can't find the container with id e11a4206cec5fa0956a8928ceeb405d75cb30f51351ab2b7b3bbf860eda27e7a Jul 12 08:22:03.023608 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:03.023579 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:03.060308 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:03.060286 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lwln2_ccef3090-87ec-4e72-a24c-5dd65f19c55b/dns-node-resolver/0.log" Jul 12 08:22:03.192217 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:03.192133 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:03.289427 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:03.289393 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:03.294252 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:03.294232 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-k2ltl_bc3f8e97-c9ab-4370-b437-8a47514c869c/console-operator/1.log" Jul 12 08:22:03.294588 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:03.294571 2566 scope.go:117] "RemoveContainer" containerID="18aa076c4b23d23244d435411d5bff8063d00bec01156f9e00e96ed40324dbc5" Jul 12 08:22:03.294778 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:22:03.294758 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-k2ltl_openshift-console-operator(bc3f8e97-c9ab-4370-b437-8a47514c869c)\"" pod="openshift-console-operator/console-operator-7866497494-k2ltl" podUID="bc3f8e97-c9ab-4370-b437-8a47514c869c" Jul 12 08:22:03.297994 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:03.297970 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xvhw4" event={"ID":"a69a78ba-31f4-488d-a7a5-0d2a197ef775","Type":"ContainerStarted","Data":"73d815f1ac55b86cbc00fe4f830b161bb3a6a45c57ee927ba4bdc627f087c8af"} Jul 12 08:22:03.299084 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:03.299064 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6l8ww" event={"ID":"55cb2ce1-ea8b-4fc7-9a14-7280c499cf67","Type":"ContainerStarted","Data":"e11a4206cec5fa0956a8928ceeb405d75cb30f51351ab2b7b3bbf860eda27e7a"} Jul 12 08:22:03.331357 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:03.331314 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xvhw4" podStartSLOduration=10.065223391 podStartE2EDuration="46.331296439s" totalImagesPullingTime="36.266073048s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 08:21:17 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:22:03.329913053 +0000 UTC m=+46.990708736" watchObservedRunningTime="2026-07-12 08:22:03.331296439 +0000 UTC m=+46.992092100" Jul 12 08:22:03.492531 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:03.492496 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:03.521892 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:03.521859 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-98qmc\" (UID: \"636c08ec-7cba-4f5e-a5b5-4db9f35be60a\") " pod="openshift-insights/insights-runtime-extractor-98qmc" Jul 12 08:22:03.522052 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:03.521946 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-98qmc\" (UID: \"636c08ec-7cba-4f5e-a5b5-4db9f35be60a\") " pod="openshift-insights/insights-runtime-extractor-98qmc" Jul 12 08:22:03.522052 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:22:03.521963 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 08:22:03.522136 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:22:03.522040 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-kube-rbac-proxy-cm podName:636c08ec-7cba-4f5e-a5b5-4db9f35be60a nodeName:}" failed. No retries permitted until 2026-07-12 08:22:05.522018451 +0000 UTC m=+49.182814093 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-98qmc" (UID: "636c08ec-7cba-4f5e-a5b5-4db9f35be60a") : configmap "kube-rbac-proxy" not found Jul 12 08:22:03.522136 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:22:03.522097 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 08:22:03.522259 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:22:03.522145 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-insights-runtime-extractor-tls podName:636c08ec-7cba-4f5e-a5b5-4db9f35be60a nodeName:}" failed. No retries permitted until 2026-07-12 08:22:05.522133728 +0000 UTC m=+49.182929371 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-98qmc" (UID: "636c08ec-7cba-4f5e-a5b5-4db9f35be60a") : secret "insights-runtime-extractor-tls" not found Jul 12 08:22:03.590188 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:03.590153 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:04.044650 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:04.044621 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pzgtb_6cd77bb0-5bbe-4d49-9e22-05dad8c650af/node-ca/0.log" Jul 12 08:22:04.643903 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:04.643870 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4p4w4_f5e63ffc-280a-49f6-8861-6b5bcf16ac29/serve-healthcheck-canary/0.log" Jul 12 08:22:04.786048 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:04.786004 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-txd5m"] Jul 12 08:22:04.802644 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:04.802622 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-txd5m"] Jul 12 08:22:04.802774 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:04.802747 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-txd5m" Jul 12 08:22:04.804978 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:04.804958 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 12 08:22:04.805118 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:04.805096 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 12 08:22:04.805872 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:04.805856 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 12 08:22:04.805952 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:04.805898 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 12 08:22:04.806055 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:04.806036 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-zn7cm\"" Jul 12 08:22:04.819508 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:04.819465 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:04.835405 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:04.835381 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnbfq\" (UniqueName: \"kubernetes.io/projected/fa7af77a-e2c9-44c2-984f-ad2ff21cf31c-kube-api-access-wnbfq\") pod \"service-ca-79f68b9f4f-txd5m\" (UID: \"fa7af77a-e2c9-44c2-984f-ad2ff21cf31c\") " pod="openshift-service-ca/service-ca-79f68b9f4f-txd5m" Jul 12 08:22:04.835493 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:04.835411 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/fa7af77a-e2c9-44c2-984f-ad2ff21cf31c-signing-key\") pod \"service-ca-79f68b9f4f-txd5m\" (UID: \"fa7af77a-e2c9-44c2-984f-ad2ff21cf31c\") " pod="openshift-service-ca/service-ca-79f68b9f4f-txd5m" Jul 12 08:22:04.835493 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:04.835443 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/fa7af77a-e2c9-44c2-984f-ad2ff21cf31c-signing-cabundle\") pod \"service-ca-79f68b9f4f-txd5m\" (UID: \"fa7af77a-e2c9-44c2-984f-ad2ff21cf31c\") " pod="openshift-service-ca/service-ca-79f68b9f4f-txd5m" Jul 12 08:22:04.919593 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:04.919566 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:04.936205 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:04.936174 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnbfq\" (UniqueName: \"kubernetes.io/projected/fa7af77a-e2c9-44c2-984f-ad2ff21cf31c-kube-api-access-wnbfq\") pod \"service-ca-79f68b9f4f-txd5m\" (UID: \"fa7af77a-e2c9-44c2-984f-ad2ff21cf31c\") " pod="openshift-service-ca/service-ca-79f68b9f4f-txd5m" Jul 12 08:22:04.936313 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:04.936219 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/fa7af77a-e2c9-44c2-984f-ad2ff21cf31c-signing-key\") pod \"service-ca-79f68b9f4f-txd5m\" (UID: \"fa7af77a-e2c9-44c2-984f-ad2ff21cf31c\") " pod="openshift-service-ca/service-ca-79f68b9f4f-txd5m" Jul 12 08:22:04.936313 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:04.936268 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/fa7af77a-e2c9-44c2-984f-ad2ff21cf31c-signing-cabundle\") pod \"service-ca-79f68b9f4f-txd5m\" (UID: \"fa7af77a-e2c9-44c2-984f-ad2ff21cf31c\") " pod="openshift-service-ca/service-ca-79f68b9f4f-txd5m" Jul 12 08:22:04.936995 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:04.936974 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/fa7af77a-e2c9-44c2-984f-ad2ff21cf31c-signing-cabundle\") pod \"service-ca-79f68b9f4f-txd5m\" (UID: \"fa7af77a-e2c9-44c2-984f-ad2ff21cf31c\") " pod="openshift-service-ca/service-ca-79f68b9f4f-txd5m" Jul 12 08:22:04.938434 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:04.938408 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/fa7af77a-e2c9-44c2-984f-ad2ff21cf31c-signing-key\") pod \"service-ca-79f68b9f4f-txd5m\" (UID: \"fa7af77a-e2c9-44c2-984f-ad2ff21cf31c\") " pod="openshift-service-ca/service-ca-79f68b9f4f-txd5m" Jul 12 08:22:04.944735 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:04.944715 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnbfq\" (UniqueName: \"kubernetes.io/projected/fa7af77a-e2c9-44c2-984f-ad2ff21cf31c-kube-api-access-wnbfq\") pod \"service-ca-79f68b9f4f-txd5m\" (UID: \"fa7af77a-e2c9-44c2-984f-ad2ff21cf31c\") " pod="openshift-service-ca/service-ca-79f68b9f4f-txd5m" Jul 12 08:22:05.112504 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:05.112469 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-txd5m" Jul 12 08:22:05.245958 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:05.245861 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-txd5m"] Jul 12 08:22:05.248174 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:22:05.248151 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa7af77a_e2c9_44c2_984f_ad2ff21cf31c.slice/crio-6089bcf1a0632806df99a90904e62e93c48e8460dc41a8f8fceaeac15ae8ac89 WatchSource:0}: Error finding container 6089bcf1a0632806df99a90904e62e93c48e8460dc41a8f8fceaeac15ae8ac89: Status 404 returned error can't find the container with id 6089bcf1a0632806df99a90904e62e93c48e8460dc41a8f8fceaeac15ae8ac89 Jul 12 08:22:05.306103 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:05.306077 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6l8ww" event={"ID":"55cb2ce1-ea8b-4fc7-9a14-7280c499cf67","Type":"ContainerStarted","Data":"e5e8a87b3395f0780ad84d54edd4bc39f851574d02d98551411d26f44fa69e16"} Jul 12 08:22:05.306195 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:05.306136 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6l8ww" event={"ID":"55cb2ce1-ea8b-4fc7-9a14-7280c499cf67","Type":"ContainerStarted","Data":"7318e37a49296f484cf6ea777c1a99a0333817d02c9d1e811602ca8363210ea2"} Jul 12 08:22:05.307176 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:05.307154 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-txd5m" event={"ID":"fa7af77a-e2c9-44c2-984f-ad2ff21cf31c","Type":"ContainerStarted","Data":"6089bcf1a0632806df99a90904e62e93c48e8460dc41a8f8fceaeac15ae8ac89"} Jul 12 08:22:05.323787 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:05.323739 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6l8ww" podStartSLOduration=1.589119111 podStartE2EDuration="3.323719799s" totalImagesPullingTime="1.734600688s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:22:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:22:05.323587044 +0000 UTC m=+48.984382725" watchObservedRunningTime="2026-07-12 08:22:05.323719799 +0000 UTC m=+48.984515460" Jul 12 08:22:05.540885 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:05.540851 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-98qmc\" (UID: \"636c08ec-7cba-4f5e-a5b5-4db9f35be60a\") " pod="openshift-insights/insights-runtime-extractor-98qmc" Jul 12 08:22:05.541051 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:22:05.541001 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 08:22:05.541051 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:05.541006 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-98qmc\" (UID: \"636c08ec-7cba-4f5e-a5b5-4db9f35be60a\") " pod="openshift-insights/insights-runtime-extractor-98qmc" Jul 12 08:22:05.541144 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:22:05.541063 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-insights-runtime-extractor-tls podName:636c08ec-7cba-4f5e-a5b5-4db9f35be60a nodeName:}" failed. No retries permitted until 2026-07-12 08:22:09.541045451 +0000 UTC m=+53.201841100 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-98qmc" (UID: "636c08ec-7cba-4f5e-a5b5-4db9f35be60a") : secret "insights-runtime-extractor-tls" not found Jul 12 08:22:05.541144 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:22:05.541078 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 08:22:05.541144 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:22:05.541116 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-kube-rbac-proxy-cm podName:636c08ec-7cba-4f5e-a5b5-4db9f35be60a nodeName:}" failed. No retries permitted until 2026-07-12 08:22:09.541109117 +0000 UTC m=+53.201904756 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-98qmc" (UID: "636c08ec-7cba-4f5e-a5b5-4db9f35be60a") : configmap "kube-rbac-proxy" not found Jul 12 08:22:05.653635 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:05.653601 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:06.236580 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:06.236532 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:06.324059 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:06.324009 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-txd5m" event={"ID":"fa7af77a-e2c9-44c2-984f-ad2ff21cf31c","Type":"ContainerStarted","Data":"6c4c89a75ffc4be8a39e83561c998e13ff1721de99c325397d3e1b89712f435d"} Jul 12 08:22:06.340809 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:06.340745 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-txd5m" podStartSLOduration=2.340727516 podStartE2EDuration="2.340727516s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:22:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:22:06.339482742 +0000 UTC m=+50.000278404" watchObservedRunningTime="2026-07-12 08:22:06.340727516 +0000 UTC m=+50.001523177" Jul 12 08:22:06.403352 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:06.403320 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:06.661159 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:06.661077 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:06.763202 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:06.763165 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:06.824600 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:06.824565 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:06.930164 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:06.930075 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:07.057308 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:07.057275 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/56a89c85-753b-4d5c-b012-a89e921d6400-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-94b94\" (UID: \"56a89c85-753b-4d5c-b012-a89e921d6400\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94b94" Jul 12 08:22:07.057455 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:22:07.057418 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 12 08:22:07.057496 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:07.057460 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-registry-tls\") pod \"image-registry-5bd7886769-cpjl8\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:22:07.057496 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:22:07.057475 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56a89c85-753b-4d5c-b012-a89e921d6400-samples-operator-tls podName:56a89c85-753b-4d5c-b012-a89e921d6400 nodeName:}" failed. No retries permitted until 2026-07-12 08:22:23.057457642 +0000 UTC m=+66.718253282 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/56a89c85-753b-4d5c-b012-a89e921d6400-samples-operator-tls") pod "cluster-samples-operator-65db858894-94b94" (UID: "56a89c85-753b-4d5c-b012-a89e921d6400") : secret "samples-operator-tls" not found Jul 12 08:22:07.057596 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:22:07.057540 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 08:22:07.057596 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:22:07.057569 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bd7886769-cpjl8: secret "image-registry-tls" not found Jul 12 08:22:07.057596 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:07.057577 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f8037f69-5405-41e4-9473-f2e22adbbb17-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-kxpkk\" (UID: \"f8037f69-5405-41e4-9473-f2e22adbbb17\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kxpkk" Jul 12 08:22:07.057681 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:22:07.057608 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-registry-tls podName:f98d0302-b975-4e38-a0d2-1f37eb5679f3 nodeName:}" failed. No retries permitted until 2026-07-12 08:22:23.057597906 +0000 UTC m=+66.718393545 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-registry-tls") pod "image-registry-5bd7886769-cpjl8" (UID: "f98d0302-b975-4e38-a0d2-1f37eb5679f3") : secret "image-registry-tls" not found Jul 12 08:22:07.057681 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:22:07.057648 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 12 08:22:07.057681 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:22:07.057679 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8037f69-5405-41e4-9473-f2e22adbbb17-cluster-monitoring-operator-tls podName:f8037f69-5405-41e4-9473-f2e22adbbb17 nodeName:}" failed. No retries permitted until 2026-07-12 08:22:23.057670995 +0000 UTC m=+66.718466634 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f8037f69-5405-41e4-9473-f2e22adbbb17-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-kxpkk" (UID: "f8037f69-5405-41e4-9473-f2e22adbbb17") : secret "cluster-monitoring-operator-tls" not found Jul 12 08:22:07.158064 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:07.158029 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bd2626f-14a5-450e-ae9a-6968c5aa9055-metrics-certs\") pod \"router-default-b8989f8c6-v5dgq\" (UID: \"1bd2626f-14a5-450e-ae9a-6968c5aa9055\") " pod="openshift-ingress/router-default-b8989f8c6-v5dgq" Jul 12 08:22:07.158227 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:07.158073 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/088c2a49-9c89-44f2-b75a-68dbeda985a6-metrics-tls\") pod \"dns-default-b756k\" (UID: \"088c2a49-9c89-44f2-b75a-68dbeda985a6\") " pod="openshift-dns/dns-default-b756k" Jul 12 08:22:07.158227 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:07.158122 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/96c69912-dd41-420f-9c38-a7b64b14d50f-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8t6pv\" (UID: \"96c69912-dd41-420f-9c38-a7b64b14d50f\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8t6pv" Jul 12 08:22:07.158227 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:22:07.158185 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 12 08:22:07.158227 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:07.158195 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bd2626f-14a5-450e-ae9a-6968c5aa9055-service-ca-bundle\") pod \"router-default-b8989f8c6-v5dgq\" (UID: \"1bd2626f-14a5-450e-ae9a-6968c5aa9055\") " pod="openshift-ingress/router-default-b8989f8c6-v5dgq" Jul 12 08:22:07.158403 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:22:07.158246 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bd2626f-14a5-450e-ae9a-6968c5aa9055-metrics-certs podName:1bd2626f-14a5-450e-ae9a-6968c5aa9055 nodeName:}" failed. No retries permitted until 2026-07-12 08:22:23.158228867 +0000 UTC m=+66.819024525 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bd2626f-14a5-450e-ae9a-6968c5aa9055-metrics-certs") pod "router-default-b8989f8c6-v5dgq" (UID: "1bd2626f-14a5-450e-ae9a-6968c5aa9055") : secret "router-metrics-certs-default" not found Jul 12 08:22:07.158403 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:22:07.158188 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 08:22:07.158403 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:22:07.158271 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 08:22:07.158403 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:22:07.158291 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1bd2626f-14a5-450e-ae9a-6968c5aa9055-service-ca-bundle podName:1bd2626f-14a5-450e-ae9a-6968c5aa9055 nodeName:}" failed. No retries permitted until 2026-07-12 08:22:23.158281467 +0000 UTC m=+66.819077122 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1bd2626f-14a5-450e-ae9a-6968c5aa9055-service-ca-bundle") pod "router-default-b8989f8c6-v5dgq" (UID: "1bd2626f-14a5-450e-ae9a-6968c5aa9055") : configmap references non-existent config key: service-ca.crt Jul 12 08:22:07.158403 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:22:07.158315 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/088c2a49-9c89-44f2-b75a-68dbeda985a6-metrics-tls podName:088c2a49-9c89-44f2-b75a-68dbeda985a6 nodeName:}" failed. No retries permitted until 2026-07-12 08:22:23.15829827 +0000 UTC m=+66.819093913 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/088c2a49-9c89-44f2-b75a-68dbeda985a6-metrics-tls") pod "dns-default-b756k" (UID: "088c2a49-9c89-44f2-b75a-68dbeda985a6") : secret "dns-default-metrics-tls" not found Jul 12 08:22:07.158403 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:22:07.158335 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96c69912-dd41-420f-9c38-a7b64b14d50f-networking-console-plugin-cert podName:96c69912-dd41-420f-9c38-a7b64b14d50f nodeName:}" failed. No retries permitted until 2026-07-12 08:22:23.158326183 +0000 UTC m=+66.819121826 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/96c69912-dd41-420f-9c38-a7b64b14d50f-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-8t6pv" (UID: "96c69912-dd41-420f-9c38-a7b64b14d50f") : secret "networking-console-plugin-cert" not found Jul 12 08:22:07.844601 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:07.844564 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:07.956072 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:07.956035 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:08.129182 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:08.129102 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:08.325757 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:08.325723 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:08.524642 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:08.524597 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:08.730765 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:08.730738 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:08.924537 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:08.924435 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:09.580515 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:09.580482 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-98qmc\" (UID: \"636c08ec-7cba-4f5e-a5b5-4db9f35be60a\") " pod="openshift-insights/insights-runtime-extractor-98qmc" Jul 12 08:22:09.580737 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:09.580568 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-98qmc\" (UID: \"636c08ec-7cba-4f5e-a5b5-4db9f35be60a\") " pod="openshift-insights/insights-runtime-extractor-98qmc" Jul 12 08:22:09.580737 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:22:09.580691 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 08:22:09.580861 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:22:09.580752 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-insights-runtime-extractor-tls podName:636c08ec-7cba-4f5e-a5b5-4db9f35be60a nodeName:}" failed. No retries permitted until 2026-07-12 08:22:17.580739643 +0000 UTC m=+61.241535282 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-98qmc" (UID: "636c08ec-7cba-4f5e-a5b5-4db9f35be60a") : secret "insights-runtime-extractor-tls" not found Jul 12 08:22:09.581096 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:09.581072 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-98qmc\" (UID: \"636c08ec-7cba-4f5e-a5b5-4db9f35be60a\") " pod="openshift-insights/insights-runtime-extractor-98qmc" Jul 12 08:22:09.921926 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:09.921846 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:09.973652 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:09.973604 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:10.119257 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:10.119222 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:10.324248 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:10.324211 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:10.530468 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:10.530435 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:10.722008 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:10.721913 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:10.771145 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:10.771110 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:10.922413 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:10.922379 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:11.550038 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:11.550000 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-k2ltl" Jul 12 08:22:11.550388 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:11.550084 2566 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-k2ltl" Jul 12 08:22:11.550442 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:11.550389 2566 scope.go:117] "RemoveContainer" containerID="18aa076c4b23d23244d435411d5bff8063d00bec01156f9e00e96ed40324dbc5" Jul 12 08:22:11.923391 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:11.923291 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:12.275970 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:12.275936 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:12.345235 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:12.345214 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-k2ltl_bc3f8e97-c9ab-4370-b437-8a47514c869c/console-operator/1.log" Jul 12 08:22:12.345397 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:12.345277 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-k2ltl" event={"ID":"bc3f8e97-c9ab-4370-b437-8a47514c869c","Type":"ContainerStarted","Data":"d02e40ef91d8b2bc9c750eabff0b32fa4a87f54959a743df4f28cafab847aec5"} Jul 12 08:22:12.345565 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:12.345528 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-k2ltl" Jul 12 08:22:12.363615 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:12.363574 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-k2ltl" podStartSLOduration=45.804792938 podStartE2EDuration="54.363539299s" totalImagesPullingTime="8.558746361s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:21:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:22:12.362792466 +0000 UTC m=+56.023588138" watchObservedRunningTime="2026-07-12 08:22:12.363539299 +0000 UTC m=+56.024335007" Jul 12 08:22:12.366533 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:12.366511 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:12.394231 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:12.394210 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-k2ltl" Jul 12 08:22:12.519691 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:12.519657 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:12.621071 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:12.620980 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:12.704097 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:12.704065 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:12.754556 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:12.754515 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:12.872107 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:12.872011 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:12.972885 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:12.972853 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:13.244853 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:13.244813 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:13.345745 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:13.345712 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:13.649643 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:13.649561 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:13.708375 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:13.708327 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:13.762121 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:13.762088 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:13.926541 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:13.926454 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:14.056536 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:14.056508 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:14.202175 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:14.202092 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:14.302969 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:14.302937 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:14.355352 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:14.355323 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:14.878379 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:14.878341 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:14.979722 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:14.979688 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:15.121305 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:15.121254 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:15.522592 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:15.522539 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:15.726447 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:15.726410 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:15.923998 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:15.923921 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:16.320847 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:16.320814 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:16.927025 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:16.926989 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:17.119604 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:17.119572 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:17.199104 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:17.199036 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lfbsq" Jul 12 08:22:17.331816 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:17.331784 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:17.653863 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:17.653831 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-98qmc\" (UID: \"636c08ec-7cba-4f5e-a5b5-4db9f35be60a\") " pod="openshift-insights/insights-runtime-extractor-98qmc" Jul 12 08:22:17.656242 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:17.656208 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/636c08ec-7cba-4f5e-a5b5-4db9f35be60a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-98qmc\" (UID: \"636c08ec-7cba-4f5e-a5b5-4db9f35be60a\") " pod="openshift-insights/insights-runtime-extractor-98qmc" Jul 12 08:22:17.791651 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:17.791620 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qlrjx\"" Jul 12 08:22:17.800408 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:17.800381 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-98qmc" Jul 12 08:22:17.918830 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:17.918760 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-98qmc"] Jul 12 08:22:17.922488 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:22:17.922455 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod636c08ec_7cba_4f5e_a5b5_4db9f35be60a.slice/crio-a18a91c96fb3627d86148d0a87eb8b210b92348ae9c62c1d0dfbe496bddda8d9 WatchSource:0}: Error finding container a18a91c96fb3627d86148d0a87eb8b210b92348ae9c62c1d0dfbe496bddda8d9: Status 404 returned error can't find the container with id a18a91c96fb3627d86148d0a87eb8b210b92348ae9c62c1d0dfbe496bddda8d9 Jul 12 08:22:18.365334 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:18.365292 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-98qmc" event={"ID":"636c08ec-7cba-4f5e-a5b5-4db9f35be60a","Type":"ContainerStarted","Data":"a18a91c96fb3627d86148d0a87eb8b210b92348ae9c62c1d0dfbe496bddda8d9"} Jul 12 08:22:22.278337 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:22.278290 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:22.333561 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:22.333506 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:22.379781 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:22.379705 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-98qmc" event={"ID":"636c08ec-7cba-4f5e-a5b5-4db9f35be60a","Type":"ContainerStarted","Data":"a7a1a8ae8ee7c8e6d401cb9c8acfca8b68ba75645c9ce9c187d2f47afaeb74c6"} Jul 12 08:22:22.379781 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:22.379759 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-98qmc" event={"ID":"636c08ec-7cba-4f5e-a5b5-4db9f35be60a","Type":"ContainerStarted","Data":"99737962b726b1688bf6ba7f0e8722f3dfbebbcbf1f4ba5d23612643100d25a1"} Jul 12 08:22:22.497206 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:22.497115 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f993375-bf43-40a2-b8c6-558a97dcba75-original-pull-secret\") pod \"global-pull-secret-syncer-w4cfw\" (UID: \"0f993375-bf43-40a2-b8c6-558a97dcba75\") " pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:22:22.499421 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:22.499406 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 12 08:22:22.510247 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:22.510220 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f993375-bf43-40a2-b8c6-558a97dcba75-original-pull-secret\") pod \"global-pull-secret-syncer-w4cfw\" (UID: \"0f993375-bf43-40a2-b8c6-558a97dcba75\") " pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:22:22.598538 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:22.598506 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/508b3df5-2cbe-4cd6-a8fd-e6cb29188184-metrics-certs\") pod \"network-metrics-daemon-9ms85\" (UID: \"508b3df5-2cbe-4cd6-a8fd-e6cb29188184\") " pod="openshift-multus/network-metrics-daemon-9ms85" Jul 12 08:22:22.600942 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:22.600924 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 12 08:22:22.612199 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:22.612171 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/508b3df5-2cbe-4cd6-a8fd-e6cb29188184-metrics-certs\") pod \"network-metrics-daemon-9ms85\" (UID: \"508b3df5-2cbe-4cd6-a8fd-e6cb29188184\") " pod="openshift-multus/network-metrics-daemon-9ms85" Jul 12 08:22:22.635499 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:22.635431 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8cc5r\"" Jul 12 08:22:22.644406 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:22.644387 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ms85" Jul 12 08:22:22.646988 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:22.646970 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4cfw" Jul 12 08:22:22.700496 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:22.700459 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr7dk\" (UniqueName: \"kubernetes.io/projected/f66d4032-1ab2-4d52-874b-a7d0296525ee-kube-api-access-sr7dk\") pod \"network-check-target-hr58s\" (UID: \"f66d4032-1ab2-4d52-874b-a7d0296525ee\") " pod="openshift-network-diagnostics/network-check-target-hr58s" Jul 12 08:22:22.704012 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:22.703828 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sr7dk\" (UniqueName: \"kubernetes.io/projected/f66d4032-1ab2-4d52-874b-a7d0296525ee-kube-api-access-sr7dk\") pod \"network-check-target-hr58s\" (UID: \"f66d4032-1ab2-4d52-874b-a7d0296525ee\") " pod="openshift-network-diagnostics/network-check-target-hr58s" Jul 12 08:22:22.790156 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:22.790127 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-w4cfw"] Jul 12 08:22:22.792738 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:22:22.792713 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f993375_bf43_40a2_b8c6_558a97dcba75.slice/crio-83ba2e7e9d4a39de4400086d9701d695bb97705bb80070a16bf35c98c491ff7d WatchSource:0}: Error finding container 83ba2e7e9d4a39de4400086d9701d695bb97705bb80070a16bf35c98c491ff7d: Status 404 returned error can't find the container with id 83ba2e7e9d4a39de4400086d9701d695bb97705bb80070a16bf35c98c491ff7d Jul 12 08:22:22.806069 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:22.806044 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9ms85"] Jul 12 08:22:22.808612 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:22:22.808587 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod508b3df5_2cbe_4cd6_a8fd_e6cb29188184.slice/crio-f4676f5b032e81036540f4781db253e1bc6343001e83d2a8425d28dc45ee6855 WatchSource:0}: Error finding container f4676f5b032e81036540f4781db253e1bc6343001e83d2a8425d28dc45ee6855: Status 404 returned error can't find the container with id f4676f5b032e81036540f4781db253e1bc6343001e83d2a8425d28dc45ee6855 Jul 12 08:22:22.943293 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:22.943219 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-d6mvs\"" Jul 12 08:22:22.952050 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:22.952026 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hr58s" Jul 12 08:22:23.103987 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.103953 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/56a89c85-753b-4d5c-b012-a89e921d6400-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-94b94\" (UID: \"56a89c85-753b-4d5c-b012-a89e921d6400\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94b94" Jul 12 08:22:23.104273 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.104051 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-registry-tls\") pod \"image-registry-5bd7886769-cpjl8\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:22:23.104273 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.104115 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f8037f69-5405-41e4-9473-f2e22adbbb17-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-kxpkk\" (UID: \"f8037f69-5405-41e4-9473-f2e22adbbb17\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kxpkk" Jul 12 08:22:23.106588 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.106559 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/56a89c85-753b-4d5c-b012-a89e921d6400-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-94b94\" (UID: \"56a89c85-753b-4d5c-b012-a89e921d6400\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94b94" Jul 12 08:22:23.106909 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.106887 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f8037f69-5405-41e4-9473-f2e22adbbb17-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-kxpkk\" (UID: \"f8037f69-5405-41e4-9473-f2e22adbbb17\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kxpkk" Jul 12 08:22:23.106978 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.106887 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-registry-tls\") pod \"image-registry-5bd7886769-cpjl8\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:22:23.114465 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.114433 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hr58s"] Jul 12 08:22:23.117645 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:22:23.117605 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf66d4032_1ab2_4d52_874b_a7d0296525ee.slice/crio-59ef722f290a25ac98ad4f5adf30ec557774e83f6673b5e66da0b9700a1b3719 WatchSource:0}: Error finding container 59ef722f290a25ac98ad4f5adf30ec557774e83f6673b5e66da0b9700a1b3719: Status 404 returned error can't find the container with id 59ef722f290a25ac98ad4f5adf30ec557774e83f6673b5e66da0b9700a1b3719 Jul 12 08:22:23.205409 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.205332 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/96c69912-dd41-420f-9c38-a7b64b14d50f-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8t6pv\" (UID: \"96c69912-dd41-420f-9c38-a7b64b14d50f\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8t6pv" Jul 12 08:22:23.205581 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.205408 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bd2626f-14a5-450e-ae9a-6968c5aa9055-service-ca-bundle\") pod \"router-default-b8989f8c6-v5dgq\" (UID: \"1bd2626f-14a5-450e-ae9a-6968c5aa9055\") " pod="openshift-ingress/router-default-b8989f8c6-v5dgq" Jul 12 08:22:23.205581 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.205467 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bd2626f-14a5-450e-ae9a-6968c5aa9055-metrics-certs\") pod \"router-default-b8989f8c6-v5dgq\" (UID: \"1bd2626f-14a5-450e-ae9a-6968c5aa9055\") " pod="openshift-ingress/router-default-b8989f8c6-v5dgq" Jul 12 08:22:23.205581 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.205501 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/088c2a49-9c89-44f2-b75a-68dbeda985a6-metrics-tls\") pod \"dns-default-b756k\" (UID: \"088c2a49-9c89-44f2-b75a-68dbeda985a6\") " pod="openshift-dns/dns-default-b756k" Jul 12 08:22:23.206122 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.206095 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bd2626f-14a5-450e-ae9a-6968c5aa9055-service-ca-bundle\") pod \"router-default-b8989f8c6-v5dgq\" (UID: \"1bd2626f-14a5-450e-ae9a-6968c5aa9055\") " pod="openshift-ingress/router-default-b8989f8c6-v5dgq" Jul 12 08:22:23.208175 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.208153 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bd2626f-14a5-450e-ae9a-6968c5aa9055-metrics-certs\") pod \"router-default-b8989f8c6-v5dgq\" (UID: \"1bd2626f-14a5-450e-ae9a-6968c5aa9055\") " pod="openshift-ingress/router-default-b8989f8c6-v5dgq" Jul 12 08:22:23.208283 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.208192 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/088c2a49-9c89-44f2-b75a-68dbeda985a6-metrics-tls\") pod \"dns-default-b756k\" (UID: \"088c2a49-9c89-44f2-b75a-68dbeda985a6\") " pod="openshift-dns/dns-default-b756k" Jul 12 08:22:23.208283 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.208263 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/96c69912-dd41-420f-9c38-a7b64b14d50f-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8t6pv\" (UID: \"96c69912-dd41-420f-9c38-a7b64b14d50f\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8t6pv" Jul 12 08:22:23.319837 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.319804 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-wckvx\"" Jul 12 08:22:23.328630 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.328601 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:22:23.333462 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.333442 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-79zxt\"" Jul 12 08:22:23.342330 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.342297 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kxpkk" Jul 12 08:22:23.358560 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.358522 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-g5dtw\"" Jul 12 08:22:23.366502 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.366462 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94b94" Jul 12 08:22:23.378443 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.378219 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-bl4p7\"" Jul 12 08:22:23.387119 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.386493 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-qfn84\"" Jul 12 08:22:23.387119 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.386897 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-8t6pv" Jul 12 08:22:23.396085 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.391673 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-w4cfw" event={"ID":"0f993375-bf43-40a2-b8c6-558a97dcba75","Type":"ContainerStarted","Data":"83ba2e7e9d4a39de4400086d9701d695bb97705bb80070a16bf35c98c491ff7d"} Jul 12 08:22:23.396085 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.392692 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9lnkq\"" Jul 12 08:22:23.396085 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.394703 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-b8989f8c6-v5dgq" Jul 12 08:22:23.396747 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.396629 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hr58s" event={"ID":"f66d4032-1ab2-4d52-874b-a7d0296525ee","Type":"ContainerStarted","Data":"a56b4bfe1c07753c930ffc0e88706fb0446be3aa70f85ddb0e8f85ee1632c9c6"} Jul 12 08:22:23.396747 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.396677 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hr58s" event={"ID":"f66d4032-1ab2-4d52-874b-a7d0296525ee","Type":"ContainerStarted","Data":"59ef722f290a25ac98ad4f5adf30ec557774e83f6673b5e66da0b9700a1b3719"} Jul 12 08:22:23.396747 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.396719 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-hr58s" Jul 12 08:22:23.403305 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.402688 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b756k" Jul 12 08:22:23.410089 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.410002 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9ms85" event={"ID":"508b3df5-2cbe-4cd6-a8fd-e6cb29188184","Type":"ContainerStarted","Data":"f4676f5b032e81036540f4781db253e1bc6343001e83d2a8425d28dc45ee6855"} Jul 12 08:22:23.423174 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.418593 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-hr58s" podStartSLOduration=66.418572347 podStartE2EDuration="1m6.418572347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:21:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:22:23.417009565 +0000 UTC m=+67.077805226" watchObservedRunningTime="2026-07-12 08:22:23.418572347 +0000 UTC m=+67.079368009" Jul 12 08:22:23.531433 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.531394 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5bd7886769-cpjl8"] Jul 12 08:22:23.582165 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.582134 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kxpkk"] Jul 12 08:22:23.597526 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.593318 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94b94"] Jul 12 08:22:23.598205 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:22:23.598151 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8037f69_5405_41e4_9473_f2e22adbbb17.slice/crio-8cd5d2df506c9294a09ec857cc3c13858c0fcf0ebbc5fbf4de601dc643f15b08 WatchSource:0}: Error finding container 8cd5d2df506c9294a09ec857cc3c13858c0fcf0ebbc5fbf4de601dc643f15b08: Status 404 returned error can't find the container with id 8cd5d2df506c9294a09ec857cc3c13858c0fcf0ebbc5fbf4de601dc643f15b08 Jul 12 08:22:23.650070 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.650029 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-8t6pv"] Jul 12 08:22:23.656280 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:22:23.655725 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96c69912_dd41_420f_9c38_a7b64b14d50f.slice/crio-1621fb919c73116dc9642020cb98b4c21ad6dda5c7a85993f34c493d998874b1 WatchSource:0}: Error finding container 1621fb919c73116dc9642020cb98b4c21ad6dda5c7a85993f34c493d998874b1: Status 404 returned error can't find the container with id 1621fb919c73116dc9642020cb98b4c21ad6dda5c7a85993f34c493d998874b1 Jul 12 08:22:23.679731 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.679667 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-b8989f8c6-v5dgq"] Jul 12 08:22:23.682289 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:22:23.682263 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bd2626f_14a5_450e_ae9a_6968c5aa9055.slice/crio-e65af0f90073e6b0395a5266678f8b160cb4cac7a62a88d6205401b22309295d WatchSource:0}: Error finding container e65af0f90073e6b0395a5266678f8b160cb4cac7a62a88d6205401b22309295d: Status 404 returned error can't find the container with id e65af0f90073e6b0395a5266678f8b160cb4cac7a62a88d6205401b22309295d Jul 12 08:22:23.702781 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:23.702720 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b756k"] Jul 12 08:22:23.706103 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:22:23.706077 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod088c2a49_9c89_44f2_b75a_68dbeda985a6.slice/crio-04ec73db826d1ba3593c30f61591b6afd893fbd72f0f5d679d46ef9a208c3a9e WatchSource:0}: Error finding container 04ec73db826d1ba3593c30f61591b6afd893fbd72f0f5d679d46ef9a208c3a9e: Status 404 returned error can't find the container with id 04ec73db826d1ba3593c30f61591b6afd893fbd72f0f5d679d46ef9a208c3a9e Jul 12 08:22:24.417763 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:24.417725 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94b94" event={"ID":"56a89c85-753b-4d5c-b012-a89e921d6400","Type":"ContainerStarted","Data":"880d8b2b5817be69e29a7554edd4f497852e83a305bd13f5c6ba62f513b8de3c"} Jul 12 08:22:24.420410 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:24.420376 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" event={"ID":"f98d0302-b975-4e38-a0d2-1f37eb5679f3","Type":"ContainerStarted","Data":"0e11bb0f270a124a73cd85575038ced2613d51bccdf798b200e9de8015a35cf1"} Jul 12 08:22:24.420536 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:24.420430 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" event={"ID":"f98d0302-b975-4e38-a0d2-1f37eb5679f3","Type":"ContainerStarted","Data":"842081e511cb25b6319a142807a1ac86d7f4b08f960d51edd16a7b138484d1f5"} Jul 12 08:22:24.420622 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:24.420598 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:22:24.424024 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:24.423973 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-b8989f8c6-v5dgq" event={"ID":"1bd2626f-14a5-450e-ae9a-6968c5aa9055","Type":"ContainerStarted","Data":"8deda3643400eb09d870134db754c3c4dd7add2a9996bc0e6fe5e68593cbe6ec"} Jul 12 08:22:24.424024 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:24.424015 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-b8989f8c6-v5dgq" event={"ID":"1bd2626f-14a5-450e-ae9a-6968c5aa9055","Type":"ContainerStarted","Data":"e65af0f90073e6b0395a5266678f8b160cb4cac7a62a88d6205401b22309295d"} Jul 12 08:22:24.427508 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:24.427485 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kxpkk" event={"ID":"f8037f69-5405-41e4-9473-f2e22adbbb17","Type":"ContainerStarted","Data":"8cd5d2df506c9294a09ec857cc3c13858c0fcf0ebbc5fbf4de601dc643f15b08"} Jul 12 08:22:24.428830 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:24.428807 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b756k" event={"ID":"088c2a49-9c89-44f2-b75a-68dbeda985a6","Type":"ContainerStarted","Data":"04ec73db826d1ba3593c30f61591b6afd893fbd72f0f5d679d46ef9a208c3a9e"} Jul 12 08:22:24.430978 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:24.430955 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-8t6pv" event={"ID":"96c69912-dd41-420f-9c38-a7b64b14d50f","Type":"ContainerStarted","Data":"1621fb919c73116dc9642020cb98b4c21ad6dda5c7a85993f34c493d998874b1"} Jul 12 08:22:24.440280 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:24.440223 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" podStartSLOduration=67.4402069 podStartE2EDuration="1m7.4402069s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:21:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:22:24.438156094 +0000 UTC m=+68.098951757" watchObservedRunningTime="2026-07-12 08:22:24.4402069 +0000 UTC m=+68.101002563" Jul 12 08:22:24.456480 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:24.456219 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-b8989f8c6-v5dgq" podStartSLOduration=66.456199876 podStartE2EDuration="1m6.456199876s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:21:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:22:24.453987287 +0000 UTC m=+68.114782964" watchObservedRunningTime="2026-07-12 08:22:24.456199876 +0000 UTC m=+68.116995538" Jul 12 08:22:25.395864 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:25.395816 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-b8989f8c6-v5dgq" Jul 12 08:22:25.398964 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:25.398940 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-b8989f8c6-v5dgq" Jul 12 08:22:25.434941 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:25.434911 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-b8989f8c6-v5dgq" Jul 12 08:22:25.436158 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:25.436135 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-b8989f8c6-v5dgq" Jul 12 08:22:27.423819 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.423784 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58f6fb9448-hkkq5"] Jul 12 08:22:27.426294 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.426271 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-67946754cf-m6cn7"] Jul 12 08:22:27.426476 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.426436 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58f6fb9448-hkkq5" Jul 12 08:22:27.428449 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.428417 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9d87467-9c875"] Jul 12 08:22:27.430433 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.429931 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67946754cf-m6cn7" Jul 12 08:22:27.433866 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.432337 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 12 08:22:27.433866 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.432601 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 12 08:22:27.433866 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.432787 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-qgnvk\"" Jul 12 08:22:27.433866 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.433337 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 12 08:22:27.433866 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.433360 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 12 08:22:27.433866 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.433598 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 12 08:22:27.436471 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.436450 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9d87467-9c875" Jul 12 08:22:27.439980 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.439428 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 12 08:22:27.440880 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.440109 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 12 08:22:27.440880 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.440213 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 12 08:22:27.440880 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.440330 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 12 08:22:27.442966 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.442793 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-67946754cf-m6cn7"] Jul 12 08:22:27.447511 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.445926 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58f6fb9448-hkkq5"] Jul 12 08:22:27.448760 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.448736 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9d87467-9c875"] Jul 12 08:22:27.530584 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.530533 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:27.536964 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.536933 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-nslt6"] Jul 12 08:22:27.539467 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.539442 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-nslt6" Jul 12 08:22:27.539933 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.539911 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-657bf5f5cd-fgk22"] Jul 12 08:22:27.541836 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.541815 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 12 08:22:27.542003 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.541981 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-ndllx\"" Jul 12 08:22:27.542092 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.542033 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-657bf5f5cd-fgk22" Jul 12 08:22:27.542216 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.542198 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 12 08:22:27.544573 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.544539 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c8eca2b8-6881-46de-ba9d-69aec404a493-klusterlet-config\") pod \"klusterlet-addon-workmgr-67946754cf-m6cn7\" (UID: \"c8eca2b8-6881-46de-ba9d-69aec404a493\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67946754cf-m6cn7" Jul 12 08:22:27.544671 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.544595 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 12 08:22:27.544671 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.544620 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7d2jg\" (UniqueName: \"kubernetes.io/projected/c8eca2b8-6881-46de-ba9d-69aec404a493-kube-api-access-7d2jg\") pod \"klusterlet-addon-workmgr-67946754cf-m6cn7\" (UID: \"c8eca2b8-6881-46de-ba9d-69aec404a493\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67946754cf-m6cn7" Jul 12 08:22:27.544671 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.544649 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a0505338-1fd8-459b-9838-f258eb7c952f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-59c9d87467-9c875\" (UID: \"a0505338-1fd8-459b-9838-f258eb7c952f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9d87467-9c875" Jul 12 08:22:27.544671 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.544666 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv59w\" (UniqueName: \"kubernetes.io/projected/51c32c39-0016-44f2-90d6-f523a5e29dbc-kube-api-access-hv59w\") pod \"managed-serviceaccount-addon-agent-58f6fb9448-hkkq5\" (UID: \"51c32c39-0016-44f2-90d6-f523a5e29dbc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58f6fb9448-hkkq5" Jul 12 08:22:27.544901 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.544693 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a0505338-1fd8-459b-9838-f258eb7c952f-hub\") pod \"cluster-proxy-proxy-agent-59c9d87467-9c875\" (UID: \"a0505338-1fd8-459b-9838-f258eb7c952f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9d87467-9c875" Jul 12 08:22:27.544901 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.544730 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a0505338-1fd8-459b-9838-f258eb7c952f-ca\") pod \"cluster-proxy-proxy-agent-59c9d87467-9c875\" (UID: \"a0505338-1fd8-459b-9838-f258eb7c952f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9d87467-9c875" Jul 12 08:22:27.544901 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.544755 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a0505338-1fd8-459b-9838-f258eb7c952f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-59c9d87467-9c875\" (UID: \"a0505338-1fd8-459b-9838-f258eb7c952f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9d87467-9c875" Jul 12 08:22:27.544901 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.544866 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-65qxs\"" Jul 12 08:22:27.544901 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.544879 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/51c32c39-0016-44f2-90d6-f523a5e29dbc-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-58f6fb9448-hkkq5\" (UID: \"51c32c39-0016-44f2-90d6-f523a5e29dbc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58f6fb9448-hkkq5" Jul 12 08:22:27.545130 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.544922 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 12 08:22:27.545130 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.545066 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0505338-1fd8-459b-9838-f258eb7c952f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-59c9d87467-9c875\" (UID: \"a0505338-1fd8-459b-9838-f258eb7c952f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9d87467-9c875" Jul 12 08:22:27.545130 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.545069 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 12 08:22:27.545130 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.545103 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w27l6\" (UniqueName: \"kubernetes.io/projected/a0505338-1fd8-459b-9838-f258eb7c952f-kube-api-access-w27l6\") pod \"cluster-proxy-proxy-agent-59c9d87467-9c875\" (UID: \"a0505338-1fd8-459b-9838-f258eb7c952f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9d87467-9c875" Jul 12 08:22:27.545130 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.545124 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8eca2b8-6881-46de-ba9d-69aec404a493-tmp\") pod \"klusterlet-addon-workmgr-67946754cf-m6cn7\" (UID: \"c8eca2b8-6881-46de-ba9d-69aec404a493\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67946754cf-m6cn7" Jul 12 08:22:27.545356 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.545147 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 12 08:22:27.545356 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.545265 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 12 08:22:27.558463 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.558134 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-nslt6"] Jul 12 08:22:27.569522 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.569496 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-657bf5f5cd-fgk22"] Jul 12 08:22:27.646250 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.646214 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-console-config\") pod \"console-657bf5f5cd-fgk22\" (UID: \"9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683\") " pod="openshift-console/console-657bf5f5cd-fgk22" Jul 12 08:22:27.646442 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.646268 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c8eca2b8-6881-46de-ba9d-69aec404a493-klusterlet-config\") pod \"klusterlet-addon-workmgr-67946754cf-m6cn7\" (UID: \"c8eca2b8-6881-46de-ba9d-69aec404a493\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67946754cf-m6cn7" Jul 12 08:22:27.646442 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.646305 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7d2jg\" (UniqueName: \"kubernetes.io/projected/c8eca2b8-6881-46de-ba9d-69aec404a493-kube-api-access-7d2jg\") pod \"klusterlet-addon-workmgr-67946754cf-m6cn7\" (UID: \"c8eca2b8-6881-46de-ba9d-69aec404a493\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67946754cf-m6cn7" Jul 12 08:22:27.646442 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.646327 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a0505338-1fd8-459b-9838-f258eb7c952f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-59c9d87467-9c875\" (UID: \"a0505338-1fd8-459b-9838-f258eb7c952f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9d87467-9c875" Jul 12 08:22:27.646442 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.646351 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv59w\" (UniqueName: \"kubernetes.io/projected/51c32c39-0016-44f2-90d6-f523a5e29dbc-kube-api-access-hv59w\") pod \"managed-serviceaccount-addon-agent-58f6fb9448-hkkq5\" (UID: \"51c32c39-0016-44f2-90d6-f523a5e29dbc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58f6fb9448-hkkq5" Jul 12 08:22:27.646671 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.646607 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-console-serving-cert\") pod \"console-657bf5f5cd-fgk22\" (UID: \"9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683\") " pod="openshift-console/console-657bf5f5cd-fgk22" Jul 12 08:22:27.646756 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.646665 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a0505338-1fd8-459b-9838-f258eb7c952f-hub\") pod \"cluster-proxy-proxy-agent-59c9d87467-9c875\" (UID: \"a0505338-1fd8-459b-9838-f258eb7c952f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9d87467-9c875" Jul 12 08:22:27.646756 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.646714 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a0505338-1fd8-459b-9838-f258eb7c952f-ca\") pod \"cluster-proxy-proxy-agent-59c9d87467-9c875\" (UID: \"a0505338-1fd8-459b-9838-f258eb7c952f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9d87467-9c875" Jul 12 08:22:27.646756 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.646744 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a0505338-1fd8-459b-9838-f258eb7c952f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-59c9d87467-9c875\" (UID: \"a0505338-1fd8-459b-9838-f258eb7c952f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9d87467-9c875" Jul 12 08:22:27.646987 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.646965 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/51c32c39-0016-44f2-90d6-f523a5e29dbc-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-58f6fb9448-hkkq5\" (UID: \"51c32c39-0016-44f2-90d6-f523a5e29dbc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58f6fb9448-hkkq5" Jul 12 08:22:27.647046 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.647009 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjm44\" (UniqueName: \"kubernetes.io/projected/360e4c5b-afe0-4bb4-88d7-5e419497559d-kube-api-access-fjm44\") pod \"downloads-68c9cbb497-nslt6\" (UID: \"360e4c5b-afe0-4bb4-88d7-5e419497559d\") " pod="openshift-console/downloads-68c9cbb497-nslt6" Jul 12 08:22:27.647093 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.647060 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0505338-1fd8-459b-9838-f258eb7c952f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-59c9d87467-9c875\" (UID: \"a0505338-1fd8-459b-9838-f258eb7c952f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9d87467-9c875" Jul 12 08:22:27.647152 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.647098 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qvhz\" (UniqueName: \"kubernetes.io/projected/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-kube-api-access-8qvhz\") pod \"console-657bf5f5cd-fgk22\" (UID: \"9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683\") " pod="openshift-console/console-657bf5f5cd-fgk22" Jul 12 08:22:27.647152 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.647128 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w27l6\" (UniqueName: \"kubernetes.io/projected/a0505338-1fd8-459b-9838-f258eb7c952f-kube-api-access-w27l6\") pod \"cluster-proxy-proxy-agent-59c9d87467-9c875\" (UID: \"a0505338-1fd8-459b-9838-f258eb7c952f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9d87467-9c875" Jul 12 08:22:27.647253 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.647158 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-service-ca\") pod \"console-657bf5f5cd-fgk22\" (UID: \"9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683\") " pod="openshift-console/console-657bf5f5cd-fgk22" Jul 12 08:22:27.647253 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.647182 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-oauth-serving-cert\") pod \"console-657bf5f5cd-fgk22\" (UID: \"9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683\") " pod="openshift-console/console-657bf5f5cd-fgk22" Jul 12 08:22:27.647253 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.647208 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8eca2b8-6881-46de-ba9d-69aec404a493-tmp\") pod \"klusterlet-addon-workmgr-67946754cf-m6cn7\" (UID: \"c8eca2b8-6881-46de-ba9d-69aec404a493\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67946754cf-m6cn7" Jul 12 08:22:27.647253 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.647240 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-console-oauth-config\") pod \"console-657bf5f5cd-fgk22\" (UID: \"9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683\") " pod="openshift-console/console-657bf5f5cd-fgk22" Jul 12 08:22:27.647856 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.647801 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8eca2b8-6881-46de-ba9d-69aec404a493-tmp\") pod \"klusterlet-addon-workmgr-67946754cf-m6cn7\" (UID: \"c8eca2b8-6881-46de-ba9d-69aec404a493\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67946754cf-m6cn7" Jul 12 08:22:27.647967 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.647868 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:27.648045 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.648017 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0505338-1fd8-459b-9838-f258eb7c952f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-59c9d87467-9c875\" (UID: \"a0505338-1fd8-459b-9838-f258eb7c952f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9d87467-9c875" Jul 12 08:22:27.650030 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.650010 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a0505338-1fd8-459b-9838-f258eb7c952f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-59c9d87467-9c875\" (UID: \"a0505338-1fd8-459b-9838-f258eb7c952f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9d87467-9c875" Jul 12 08:22:27.650584 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.650513 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a0505338-1fd8-459b-9838-f258eb7c952f-ca\") pod \"cluster-proxy-proxy-agent-59c9d87467-9c875\" (UID: \"a0505338-1fd8-459b-9838-f258eb7c952f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9d87467-9c875" Jul 12 08:22:27.650666 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.650594 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a0505338-1fd8-459b-9838-f258eb7c952f-hub\") pod \"cluster-proxy-proxy-agent-59c9d87467-9c875\" (UID: \"a0505338-1fd8-459b-9838-f258eb7c952f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9d87467-9c875" Jul 12 08:22:27.650666 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.650597 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c8eca2b8-6881-46de-ba9d-69aec404a493-klusterlet-config\") pod \"klusterlet-addon-workmgr-67946754cf-m6cn7\" (UID: \"c8eca2b8-6881-46de-ba9d-69aec404a493\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67946754cf-m6cn7" Jul 12 08:22:27.651002 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.650968 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a0505338-1fd8-459b-9838-f258eb7c952f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-59c9d87467-9c875\" (UID: \"a0505338-1fd8-459b-9838-f258eb7c952f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9d87467-9c875" Jul 12 08:22:27.651446 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.651423 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/51c32c39-0016-44f2-90d6-f523a5e29dbc-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-58f6fb9448-hkkq5\" (UID: \"51c32c39-0016-44f2-90d6-f523a5e29dbc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58f6fb9448-hkkq5" Jul 12 08:22:27.658898 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.658876 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv59w\" (UniqueName: \"kubernetes.io/projected/51c32c39-0016-44f2-90d6-f523a5e29dbc-kube-api-access-hv59w\") pod \"managed-serviceaccount-addon-agent-58f6fb9448-hkkq5\" (UID: \"51c32c39-0016-44f2-90d6-f523a5e29dbc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58f6fb9448-hkkq5" Jul 12 08:22:27.659166 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.659144 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7d2jg\" (UniqueName: \"kubernetes.io/projected/c8eca2b8-6881-46de-ba9d-69aec404a493-kube-api-access-7d2jg\") pod \"klusterlet-addon-workmgr-67946754cf-m6cn7\" (UID: \"c8eca2b8-6881-46de-ba9d-69aec404a493\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67946754cf-m6cn7" Jul 12 08:22:27.659355 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.659334 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w27l6\" (UniqueName: \"kubernetes.io/projected/a0505338-1fd8-459b-9838-f258eb7c952f-kube-api-access-w27l6\") pod \"cluster-proxy-proxy-agent-59c9d87467-9c875\" (UID: \"a0505338-1fd8-459b-9838-f258eb7c952f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9d87467-9c875" Jul 12 08:22:27.738916 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.738879 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:27.748628 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.748599 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qvhz\" (UniqueName: \"kubernetes.io/projected/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-kube-api-access-8qvhz\") pod \"console-657bf5f5cd-fgk22\" (UID: \"9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683\") " pod="openshift-console/console-657bf5f5cd-fgk22" Jul 12 08:22:27.748766 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.748633 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-service-ca\") pod \"console-657bf5f5cd-fgk22\" (UID: \"9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683\") " pod="openshift-console/console-657bf5f5cd-fgk22" Jul 12 08:22:27.748766 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.748650 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-oauth-serving-cert\") pod \"console-657bf5f5cd-fgk22\" (UID: \"9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683\") " pod="openshift-console/console-657bf5f5cd-fgk22" Jul 12 08:22:27.748766 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.748673 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-console-oauth-config\") pod \"console-657bf5f5cd-fgk22\" (UID: \"9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683\") " pod="openshift-console/console-657bf5f5cd-fgk22" Jul 12 08:22:27.748766 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.748700 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-console-config\") pod \"console-657bf5f5cd-fgk22\" (UID: \"9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683\") " pod="openshift-console/console-657bf5f5cd-fgk22" Jul 12 08:22:27.748766 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.748739 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-console-serving-cert\") pod \"console-657bf5f5cd-fgk22\" (UID: \"9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683\") " pod="openshift-console/console-657bf5f5cd-fgk22" Jul 12 08:22:27.749001 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.748795 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjm44\" (UniqueName: \"kubernetes.io/projected/360e4c5b-afe0-4bb4-88d7-5e419497559d-kube-api-access-fjm44\") pod \"downloads-68c9cbb497-nslt6\" (UID: \"360e4c5b-afe0-4bb4-88d7-5e419497559d\") " pod="openshift-console/downloads-68c9cbb497-nslt6" Jul 12 08:22:27.749444 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.749396 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-service-ca\") pod \"console-657bf5f5cd-fgk22\" (UID: \"9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683\") " pod="openshift-console/console-657bf5f5cd-fgk22" Jul 12 08:22:27.749582 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.749455 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-oauth-serving-cert\") pod \"console-657bf5f5cd-fgk22\" (UID: \"9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683\") " pod="openshift-console/console-657bf5f5cd-fgk22" Jul 12 08:22:27.749656 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.749579 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-console-config\") pod \"console-657bf5f5cd-fgk22\" (UID: \"9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683\") " pod="openshift-console/console-657bf5f5cd-fgk22" Jul 12 08:22:27.751494 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.751472 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-console-oauth-config\") pod \"console-657bf5f5cd-fgk22\" (UID: \"9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683\") " pod="openshift-console/console-657bf5f5cd-fgk22" Jul 12 08:22:27.751617 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.751590 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-console-serving-cert\") pod \"console-657bf5f5cd-fgk22\" (UID: \"9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683\") " pod="openshift-console/console-657bf5f5cd-fgk22" Jul 12 08:22:27.762936 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.762915 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjm44\" (UniqueName: \"kubernetes.io/projected/360e4c5b-afe0-4bb4-88d7-5e419497559d-kube-api-access-fjm44\") pod \"downloads-68c9cbb497-nslt6\" (UID: \"360e4c5b-afe0-4bb4-88d7-5e419497559d\") " pod="openshift-console/downloads-68c9cbb497-nslt6" Jul 12 08:22:27.763130 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.763109 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qvhz\" (UniqueName: \"kubernetes.io/projected/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-kube-api-access-8qvhz\") pod \"console-657bf5f5cd-fgk22\" (UID: \"9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683\") " pod="openshift-console/console-657bf5f5cd-fgk22" Jul 12 08:22:27.763744 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.763723 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58f6fb9448-hkkq5" Jul 12 08:22:27.774269 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.774251 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67946754cf-m6cn7" Jul 12 08:22:27.778192 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.778170 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9d87467-9c875" Jul 12 08:22:27.860318 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.860286 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:27.866138 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.866048 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-nslt6" Jul 12 08:22:27.871256 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.871233 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-657bf5f5cd-fgk22" Jul 12 08:22:27.955399 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:27.955370 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:28.008523 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:28.008447 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:28.898469 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:28.898393 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:28.998717 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:28.998685 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:29.207249 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:29.207173 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:29.306825 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:29.306794 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:29.356349 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:29.356321 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:29.596151 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:29.596062 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-657bf5f5cd-fgk22"] Jul 12 08:22:29.627402 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:22:29.617578 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ef22f9b_9b2b_4d3b_9b82_0b56ed2d7683.slice/crio-c38ea892d0e7dc3d39b58a1d7d5e81283da9c7aafc2bc1aee6a1ddf2384d8a84 WatchSource:0}: Error finding container c38ea892d0e7dc3d39b58a1d7d5e81283da9c7aafc2bc1aee6a1ddf2384d8a84: Status 404 returned error can't find the container with id c38ea892d0e7dc3d39b58a1d7d5e81283da9c7aafc2bc1aee6a1ddf2384d8a84 Jul 12 08:22:29.632242 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:29.631242 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-nslt6"] Jul 12 08:22:29.658338 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:29.658281 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9d87467-9c875"] Jul 12 08:22:29.659438 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:29.659390 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58f6fb9448-hkkq5"] Jul 12 08:22:29.665158 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:22:29.665133 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda0505338_1fd8_459b_9838_f258eb7c952f.slice/crio-bc912dc397b3c74d0002fcf5526b44058acc72437a43d8fcb660eca817fb1fb4 WatchSource:0}: Error finding container bc912dc397b3c74d0002fcf5526b44058acc72437a43d8fcb660eca817fb1fb4: Status 404 returned error can't find the container with id bc912dc397b3c74d0002fcf5526b44058acc72437a43d8fcb660eca817fb1fb4 Jul 12 08:22:29.666619 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:22:29.666595 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod51c32c39_0016_44f2_90d6_f523a5e29dbc.slice/crio-bea4cef90f3cce188b16b67fd0a2c0597af652d1af0e1576138a43fd2d4fe357 WatchSource:0}: Error finding container bea4cef90f3cce188b16b67fd0a2c0597af652d1af0e1576138a43fd2d4fe357: Status 404 returned error can't find the container with id bea4cef90f3cce188b16b67fd0a2c0597af652d1af0e1576138a43fd2d4fe357 Jul 12 08:22:29.681489 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:29.681463 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-67946754cf-m6cn7"] Jul 12 08:22:29.687277 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:22:29.687188 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8eca2b8_6881_46de_ba9d_69aec404a493.slice/crio-c78ba38f8deff88f23551858e8f8327c84fc081ef1147cad539b448679b15dd3 WatchSource:0}: Error finding container c78ba38f8deff88f23551858e8f8327c84fc081ef1147cad539b448679b15dd3: Status 404 returned error can't find the container with id c78ba38f8deff88f23551858e8f8327c84fc081ef1147cad539b448679b15dd3 Jul 12 08:22:30.033866 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.033774 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:30.134485 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.134449 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:30.202247 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.198992 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bxvgb"] Jul 12 08:22:30.211674 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.211637 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:30.218850 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.218673 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bxvgb"] Jul 12 08:22:30.218850 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.218783 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bxvgb" Jul 12 08:22:30.221801 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.221671 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-4kfv5\"" Jul 12 08:22:30.221801 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.221680 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 12 08:22:30.304788 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.304701 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:30.378122 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.375037 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5e9e1ae0-3d7c-4a0f-bdf1-694141807cc5-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-bxvgb\" (UID: \"5e9e1ae0-3d7c-4a0f-bdf1-694141807cc5\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bxvgb" Jul 12 08:22:30.471353 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.471263 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67946754cf-m6cn7" event={"ID":"c8eca2b8-6881-46de-ba9d-69aec404a493","Type":"ContainerStarted","Data":"c78ba38f8deff88f23551858e8f8327c84fc081ef1147cad539b448679b15dd3"} Jul 12 08:22:30.473663 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.473596 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-nslt6" event={"ID":"360e4c5b-afe0-4bb4-88d7-5e419497559d","Type":"ContainerStarted","Data":"ced93f4fc32305a3ec575523c44e843d05a4f0e7a5e2d8250566465d06dd4029"} Jul 12 08:22:30.476471 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.476440 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5e9e1ae0-3d7c-4a0f-bdf1-694141807cc5-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-bxvgb\" (UID: \"5e9e1ae0-3d7c-4a0f-bdf1-694141807cc5\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bxvgb" Jul 12 08:22:30.478572 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.478290 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-98qmc" event={"ID":"636c08ec-7cba-4f5e-a5b5-4db9f35be60a","Type":"ContainerStarted","Data":"f3d0b4f1e01e1b1657ebe4c8a9543691a22f391dd19e82337a0e9950fc12809e"} Jul 12 08:22:30.480724 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.480660 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-657bf5f5cd-fgk22" event={"ID":"9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683","Type":"ContainerStarted","Data":"c38ea892d0e7dc3d39b58a1d7d5e81283da9c7aafc2bc1aee6a1ddf2384d8a84"} Jul 12 08:22:30.483732 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.483222 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94b94" event={"ID":"56a89c85-753b-4d5c-b012-a89e921d6400","Type":"ContainerStarted","Data":"5d69e3284799bfc94679e1f501ff56b5647171050feea3eec874717c104c6482"} Jul 12 08:22:30.483732 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.483266 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94b94" event={"ID":"56a89c85-753b-4d5c-b012-a89e921d6400","Type":"ContainerStarted","Data":"de2f03b6812bc39698cb23cd084680439cba21725c1d8815f41d0aa9122ece40"} Jul 12 08:22:30.487189 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.487144 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5e9e1ae0-3d7c-4a0f-bdf1-694141807cc5-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-bxvgb\" (UID: \"5e9e1ae0-3d7c-4a0f-bdf1-694141807cc5\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bxvgb" Jul 12 08:22:30.488708 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.488138 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9ms85" event={"ID":"508b3df5-2cbe-4cd6-a8fd-e6cb29188184","Type":"ContainerStarted","Data":"529c701b8e277d9b9920e8fef8fa89ad98f7cd3a526772b7c5afe24fc998f961"} Jul 12 08:22:30.488708 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.488186 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9ms85" event={"ID":"508b3df5-2cbe-4cd6-a8fd-e6cb29188184","Type":"ContainerStarted","Data":"54bdbe96ce1a338c9016aaca8c4c85d8f7cfeb586b50c025afcebea1a4b9e14d"} Jul 12 08:22:30.491095 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.490646 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-w4cfw" event={"ID":"0f993375-bf43-40a2-b8c6-558a97dcba75","Type":"ContainerStarted","Data":"dc51dc8258fa6de04a13a0519c19716ccef48e82fd8791830fd88d8e33da5675"} Jul 12 08:22:30.493593 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.493183 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kxpkk" event={"ID":"f8037f69-5405-41e4-9473-f2e22adbbb17","Type":"ContainerStarted","Data":"4a3179289c167053610121021b4c56fcb1b815475d632f7ed15a91033a6ee9bf"} Jul 12 08:22:30.499134 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.498655 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-98qmc" podStartSLOduration=21.965088986 podStartE2EDuration="29.498637287s" totalImagesPullingTime="7.533548301s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 08:22:01 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:22:30.496405717 +0000 UTC m=+74.157201379" watchObservedRunningTime="2026-07-12 08:22:30.498637287 +0000 UTC m=+74.159432929" Jul 12 08:22:30.501256 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.501202 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58f6fb9448-hkkq5" event={"ID":"51c32c39-0016-44f2-90d6-f523a5e29dbc","Type":"ContainerStarted","Data":"bea4cef90f3cce188b16b67fd0a2c0597af652d1af0e1576138a43fd2d4fe357"} Jul 12 08:22:30.503204 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.503105 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9d87467-9c875" event={"ID":"a0505338-1fd8-459b-9838-f258eb7c952f","Type":"ContainerStarted","Data":"bc912dc397b3c74d0002fcf5526b44058acc72437a43d8fcb660eca817fb1fb4"} Jul 12 08:22:30.508821 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.508286 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b756k" event={"ID":"088c2a49-9c89-44f2-b75a-68dbeda985a6","Type":"ContainerStarted","Data":"f533a6d503815424311ad52195e5d4ea06174a21447f8cf87b597bef200739e2"} Jul 12 08:22:30.508821 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.508362 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b756k" event={"ID":"088c2a49-9c89-44f2-b75a-68dbeda985a6","Type":"ContainerStarted","Data":"87e83478f9e675ede1cda899954c0191554a10459215fce355a86be62491714a"} Jul 12 08:22:30.509293 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.509224 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-b756k" Jul 12 08:22:30.511184 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.511159 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-8t6pv" event={"ID":"96c69912-dd41-420f-9c38-a7b64b14d50f","Type":"ContainerStarted","Data":"186e099376c56f27ab9673381382ae127d56dd3d020745e1b3e41d10d0201cb4"} Jul 12 08:22:30.514642 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.514572 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kxpkk" podStartSLOduration=67.743182312 podStartE2EDuration="1m13.514540756s" totalImagesPullingTime="5.771358444s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:22:30.514209446 +0000 UTC m=+74.175005110" watchObservedRunningTime="2026-07-12 08:22:30.514540756 +0000 UTC m=+74.175336419" Jul 12 08:22:30.531082 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.531052 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bxvgb" Jul 12 08:22:30.541781 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.540782 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-94b94" podStartSLOduration=66.848246821 podStartE2EDuration="1m12.540764466s" totalImagesPullingTime="5.692517645s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:21:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:22:30.540597061 +0000 UTC m=+74.201392723" watchObservedRunningTime="2026-07-12 08:22:30.540764466 +0000 UTC m=+74.201560122" Jul 12 08:22:30.557970 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.557883 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-w4cfw" podStartSLOduration=66.975339634 podStartE2EDuration="1m13.557863721s" totalImagesPullingTime="6.582524087s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 08:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:22:30.556764437 +0000 UTC m=+74.217560100" watchObservedRunningTime="2026-07-12 08:22:30.557863721 +0000 UTC m=+74.218659379" Jul 12 08:22:30.575193 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.574974 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9ms85" podStartSLOduration=70.833216885 podStartE2EDuration="1m13.574955562s" totalImagesPullingTime="2.741738677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:22:30.573218621 +0000 UTC m=+74.234014282" watchObservedRunningTime="2026-07-12 08:22:30.574955562 +0000 UTC m=+74.235751224" Jul 12 08:22:30.591980 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.591022 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-b756k" podStartSLOduration=33.658910978 podStartE2EDuration="39.591005512s" totalImagesPullingTime="5.932094534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:21:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:22:30.588918844 +0000 UTC m=+74.249714529" watchObservedRunningTime="2026-07-12 08:22:30.591005512 +0000 UTC m=+74.251801175" Jul 12 08:22:30.606814 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.606109 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-8t6pv" podStartSLOduration=39.894805437 podStartE2EDuration="45.606090547s" totalImagesPullingTime="5.71128511s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:21:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:22:30.604434561 +0000 UTC m=+74.265230220" watchObservedRunningTime="2026-07-12 08:22:30.606090547 +0000 UTC m=+74.266886207" Jul 12 08:22:30.697708 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.697534 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:30.802889 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.802796 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:30.809446 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:30.809338 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bxvgb"] Jul 12 08:22:30.813688 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:22:30.813657 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e9e1ae0_3d7c_4a0f_bdf1_694141807cc5.slice/crio-4167feb77a260469871bceafc463630a0257c08a717be29746afc65e144fba08 WatchSource:0}: Error finding container 4167feb77a260469871bceafc463630a0257c08a717be29746afc65e144fba08: Status 404 returned error can't find the container with id 4167feb77a260469871bceafc463630a0257c08a717be29746afc65e144fba08 Jul 12 08:22:31.467784 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:31.467699 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:31.530594 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:31.528959 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bxvgb" event={"ID":"5e9e1ae0-3d7c-4a0f-bdf1-694141807cc5","Type":"ContainerStarted","Data":"4167feb77a260469871bceafc463630a0257c08a717be29746afc65e144fba08"} Jul 12 08:22:31.686482 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:31.686445 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:31.739954 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:31.739416 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:31.861833 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:31.861684 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:31.919206 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:31.917857 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:32.080867 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:32.080779 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-85b8c4bc8d-nsmwp"] Jul 12 08:22:32.104945 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:32.104914 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85b8c4bc8d-nsmwp"] Jul 12 08:22:32.105116 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:32.105084 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85b8c4bc8d-nsmwp" Jul 12 08:22:32.115047 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:32.115014 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 12 08:22:32.195394 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:32.194997 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4h5rg\" (UniqueName: \"kubernetes.io/projected/bd26a038-0655-4265-9834-97c2d8de330a-kube-api-access-4h5rg\") pod \"console-85b8c4bc8d-nsmwp\" (UID: \"bd26a038-0655-4265-9834-97c2d8de330a\") " pod="openshift-console/console-85b8c4bc8d-nsmwp" Jul 12 08:22:32.195394 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:32.195073 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd26a038-0655-4265-9834-97c2d8de330a-service-ca\") pod \"console-85b8c4bc8d-nsmwp\" (UID: \"bd26a038-0655-4265-9834-97c2d8de330a\") " pod="openshift-console/console-85b8c4bc8d-nsmwp" Jul 12 08:22:32.195394 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:32.195113 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd26a038-0655-4265-9834-97c2d8de330a-console-serving-cert\") pod \"console-85b8c4bc8d-nsmwp\" (UID: \"bd26a038-0655-4265-9834-97c2d8de330a\") " pod="openshift-console/console-85b8c4bc8d-nsmwp" Jul 12 08:22:32.195394 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:32.195152 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd26a038-0655-4265-9834-97c2d8de330a-oauth-serving-cert\") pod \"console-85b8c4bc8d-nsmwp\" (UID: \"bd26a038-0655-4265-9834-97c2d8de330a\") " pod="openshift-console/console-85b8c4bc8d-nsmwp" Jul 12 08:22:32.195394 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:32.195183 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd26a038-0655-4265-9834-97c2d8de330a-trusted-ca-bundle\") pod \"console-85b8c4bc8d-nsmwp\" (UID: \"bd26a038-0655-4265-9834-97c2d8de330a\") " pod="openshift-console/console-85b8c4bc8d-nsmwp" Jul 12 08:22:32.195394 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:32.195223 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd26a038-0655-4265-9834-97c2d8de330a-console-oauth-config\") pod \"console-85b8c4bc8d-nsmwp\" (UID: \"bd26a038-0655-4265-9834-97c2d8de330a\") " pod="openshift-console/console-85b8c4bc8d-nsmwp" Jul 12 08:22:32.195394 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:32.195260 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd26a038-0655-4265-9834-97c2d8de330a-console-config\") pod \"console-85b8c4bc8d-nsmwp\" (UID: \"bd26a038-0655-4265-9834-97c2d8de330a\") " pod="openshift-console/console-85b8c4bc8d-nsmwp" Jul 12 08:22:32.296732 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:32.296692 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd26a038-0655-4265-9834-97c2d8de330a-console-serving-cert\") pod \"console-85b8c4bc8d-nsmwp\" (UID: \"bd26a038-0655-4265-9834-97c2d8de330a\") " pod="openshift-console/console-85b8c4bc8d-nsmwp" Jul 12 08:22:32.296906 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:32.296757 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd26a038-0655-4265-9834-97c2d8de330a-oauth-serving-cert\") pod \"console-85b8c4bc8d-nsmwp\" (UID: \"bd26a038-0655-4265-9834-97c2d8de330a\") " pod="openshift-console/console-85b8c4bc8d-nsmwp" Jul 12 08:22:32.296906 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:32.296797 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd26a038-0655-4265-9834-97c2d8de330a-trusted-ca-bundle\") pod \"console-85b8c4bc8d-nsmwp\" (UID: \"bd26a038-0655-4265-9834-97c2d8de330a\") " pod="openshift-console/console-85b8c4bc8d-nsmwp" Jul 12 08:22:32.296906 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:32.296837 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd26a038-0655-4265-9834-97c2d8de330a-console-oauth-config\") pod \"console-85b8c4bc8d-nsmwp\" (UID: \"bd26a038-0655-4265-9834-97c2d8de330a\") " pod="openshift-console/console-85b8c4bc8d-nsmwp" Jul 12 08:22:32.296906 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:32.296874 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd26a038-0655-4265-9834-97c2d8de330a-console-config\") pod \"console-85b8c4bc8d-nsmwp\" (UID: \"bd26a038-0655-4265-9834-97c2d8de330a\") " pod="openshift-console/console-85b8c4bc8d-nsmwp" Jul 12 08:22:32.297121 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:32.296909 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4h5rg\" (UniqueName: \"kubernetes.io/projected/bd26a038-0655-4265-9834-97c2d8de330a-kube-api-access-4h5rg\") pod \"console-85b8c4bc8d-nsmwp\" (UID: \"bd26a038-0655-4265-9834-97c2d8de330a\") " pod="openshift-console/console-85b8c4bc8d-nsmwp" Jul 12 08:22:32.297121 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:32.296962 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd26a038-0655-4265-9834-97c2d8de330a-service-ca\") pod \"console-85b8c4bc8d-nsmwp\" (UID: \"bd26a038-0655-4265-9834-97c2d8de330a\") " pod="openshift-console/console-85b8c4bc8d-nsmwp" Jul 12 08:22:32.297722 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:32.297699 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd26a038-0655-4265-9834-97c2d8de330a-service-ca\") pod \"console-85b8c4bc8d-nsmwp\" (UID: \"bd26a038-0655-4265-9834-97c2d8de330a\") " pod="openshift-console/console-85b8c4bc8d-nsmwp" Jul 12 08:22:32.300135 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:32.298892 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd26a038-0655-4265-9834-97c2d8de330a-oauth-serving-cert\") pod \"console-85b8c4bc8d-nsmwp\" (UID: \"bd26a038-0655-4265-9834-97c2d8de330a\") " pod="openshift-console/console-85b8c4bc8d-nsmwp" Jul 12 08:22:32.300135 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:32.299409 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd26a038-0655-4265-9834-97c2d8de330a-console-config\") pod \"console-85b8c4bc8d-nsmwp\" (UID: \"bd26a038-0655-4265-9834-97c2d8de330a\") " pod="openshift-console/console-85b8c4bc8d-nsmwp" Jul 12 08:22:32.300599 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:32.300577 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd26a038-0655-4265-9834-97c2d8de330a-trusted-ca-bundle\") pod \"console-85b8c4bc8d-nsmwp\" (UID: \"bd26a038-0655-4265-9834-97c2d8de330a\") " pod="openshift-console/console-85b8c4bc8d-nsmwp" Jul 12 08:22:32.302764 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:32.302723 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd26a038-0655-4265-9834-97c2d8de330a-console-oauth-config\") pod \"console-85b8c4bc8d-nsmwp\" (UID: \"bd26a038-0655-4265-9834-97c2d8de330a\") " pod="openshift-console/console-85b8c4bc8d-nsmwp" Jul 12 08:22:32.304227 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:32.304189 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd26a038-0655-4265-9834-97c2d8de330a-console-serving-cert\") pod \"console-85b8c4bc8d-nsmwp\" (UID: \"bd26a038-0655-4265-9834-97c2d8de330a\") " pod="openshift-console/console-85b8c4bc8d-nsmwp" Jul 12 08:22:32.328544 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:32.328486 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4h5rg\" (UniqueName: \"kubernetes.io/projected/bd26a038-0655-4265-9834-97c2d8de330a-kube-api-access-4h5rg\") pod \"console-85b8c4bc8d-nsmwp\" (UID: \"bd26a038-0655-4265-9834-97c2d8de330a\") " pod="openshift-console/console-85b8c4bc8d-nsmwp" Jul 12 08:22:32.423716 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:32.423638 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85b8c4bc8d-nsmwp" Jul 12 08:22:32.696980 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:32.696842 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:32.749861 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:32.749777 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:33.607447 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:33.607409 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:33.658499 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:33.658404 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:33.710177 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:33.710129 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:33.865324 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:33.865249 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:33.917062 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:33.917022 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:34.116144 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:34.116051 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:34.215408 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:34.215369 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:38.358149 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:38.358056 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:38.412770 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:38.412731 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:39.001086 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:39.000904 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85b8c4bc8d-nsmwp"] Jul 12 08:22:39.005263 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:22:39.004990 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd26a038_0655_4265_9834_97c2d8de330a.slice/crio-25f250358803e149121878a35768676b6a3fb564adb8c583c7fd84b9f4315338 WatchSource:0}: Error finding container 25f250358803e149121878a35768676b6a3fb564adb8c583c7fd84b9f4315338: Status 404 returned error can't find the container with id 25f250358803e149121878a35768676b6a3fb564adb8c583c7fd84b9f4315338 Jul 12 08:22:39.571117 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:39.571081 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85b8c4bc8d-nsmwp" event={"ID":"bd26a038-0655-4265-9834-97c2d8de330a","Type":"ContainerStarted","Data":"89f85aa563dae37e7811543e6c656eda5ab2a3bc129c8a5a8a4e01e13daebee3"} Jul 12 08:22:39.571523 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:39.571140 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85b8c4bc8d-nsmwp" event={"ID":"bd26a038-0655-4265-9834-97c2d8de330a","Type":"ContainerStarted","Data":"25f250358803e149121878a35768676b6a3fb564adb8c583c7fd84b9f4315338"} Jul 12 08:22:39.572736 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:39.572706 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-657bf5f5cd-fgk22" event={"ID":"9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683","Type":"ContainerStarted","Data":"89e4e0063ab821c9c8a1d3dc922763b701e2157a0bcb0c2cc11536b903f79849"} Jul 12 08:22:39.574084 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:39.574061 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58f6fb9448-hkkq5" event={"ID":"51c32c39-0016-44f2-90d6-f523a5e29dbc","Type":"ContainerStarted","Data":"ef4c63ccb225fed017f3c6d42a3831c3aa52427db3ba04875502604855ce836d"} Jul 12 08:22:39.575898 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:39.575879 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9d87467-9c875" event={"ID":"a0505338-1fd8-459b-9838-f258eb7c952f","Type":"ContainerStarted","Data":"735a99a0950fa65686d23b6d11c2b9bec2af9e62e1c5ddeb27eee50ae1b7fa76"} Jul 12 08:22:39.576024 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:39.575903 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9d87467-9c875" event={"ID":"a0505338-1fd8-459b-9838-f258eb7c952f","Type":"ContainerStarted","Data":"71f291956adc278019c41fcea1ba85bce8fe6fb444c37ed3cc400ca3831e8147"} Jul 12 08:22:39.576024 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:39.575916 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9d87467-9c875" event={"ID":"a0505338-1fd8-459b-9838-f258eb7c952f","Type":"ContainerStarted","Data":"77a568fd3e48c2495a1c275c07f087ee7cc1c3ae7869a75699e44e6a43e35ee1"} Jul 12 08:22:39.577228 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:39.577210 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67946754cf-m6cn7" event={"ID":"c8eca2b8-6881-46de-ba9d-69aec404a493","Type":"ContainerStarted","Data":"c167cfb44b4f479925ca9469e655546946d08ee6ffbe9609d8520d19a24185ef"} Jul 12 08:22:39.577315 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:39.577247 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67946754cf-m6cn7" Jul 12 08:22:39.578801 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:39.578780 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bxvgb" event={"ID":"5e9e1ae0-3d7c-4a0f-bdf1-694141807cc5","Type":"ContainerStarted","Data":"6b4c4b598d5e83ead64f1f0ba58e1ef7b348bee65f136ac8bcfa2d45d5e1d9b8"} Jul 12 08:22:39.578975 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:39.578950 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bxvgb" Jul 12 08:22:39.579120 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:39.579106 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67946754cf-m6cn7" Jul 12 08:22:39.582976 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:39.582963 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bxvgb" Jul 12 08:22:39.588429 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:39.588377 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-85b8c4bc8d-nsmwp" podStartSLOduration=7.588360964 podStartE2EDuration="7.588360964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:22:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:22:39.586770107 +0000 UTC m=+83.247565768" watchObservedRunningTime="2026-07-12 08:22:39.588360964 +0000 UTC m=+83.249156606" Jul 12 08:22:39.608940 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:39.608900 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9d87467-9c875" podStartSLOduration=3.460201393 podStartE2EDuration="12.608888682s" totalImagesPullingTime="9.148687289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:22:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:22:39.607261826 +0000 UTC m=+83.268057488" watchObservedRunningTime="2026-07-12 08:22:39.608888682 +0000 UTC m=+83.269684357" Jul 12 08:22:39.643496 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:39.643409 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58f6fb9448-hkkq5" podStartSLOduration=3.4331141450000002 podStartE2EDuration="12.643389648s" totalImagesPullingTime="9.210275503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:22:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:22:39.64221584 +0000 UTC m=+83.303011502" watchObservedRunningTime="2026-07-12 08:22:39.643389648 +0000 UTC m=+83.304185310" Jul 12 08:22:39.643971 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:39.643932 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-657bf5f5cd-fgk22" podStartSLOduration=3.405873171 podStartE2EDuration="12.643920884s" totalImagesPullingTime="9.238047713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:22:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:22:39.626588684 +0000 UTC m=+83.287384341" watchObservedRunningTime="2026-07-12 08:22:39.643920884 +0000 UTC m=+83.304716607" Jul 12 08:22:39.646578 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:39.646526 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:39.668801 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:39.668741 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67946754cf-m6cn7" podStartSLOduration=3.455179771 podStartE2EDuration="12.668723909s" totalImagesPullingTime="9.213544138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:22:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:22:39.667003919 +0000 UTC m=+83.327799603" watchObservedRunningTime="2026-07-12 08:22:39.668723909 +0000 UTC m=+83.329519571" Jul 12 08:22:39.682627 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:39.682489 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bxvgb" podStartSLOduration=1.635820014 podStartE2EDuration="9.68247276s" totalImagesPullingTime="8.046652746s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:22:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:22:39.681904255 +0000 UTC m=+83.342699943" watchObservedRunningTime="2026-07-12 08:22:39.68247276 +0000 UTC m=+83.343268422" Jul 12 08:22:39.710303 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:39.710246 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:40.221774 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:40.221737 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:40.300039 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:40.300003 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:40.484256 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:40.484216 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:40.531740 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:40.531712 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-b756k" Jul 12 08:22:42.424506 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:42.424469 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-85b8c4bc8d-nsmwp" Jul 12 08:22:42.424973 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:42.424533 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-85b8c4bc8d-nsmwp" Jul 12 08:22:42.426031 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:42.425992 2566 patch_prober.go:28] interesting pod/console-85b8c4bc8d-nsmwp container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.28:8443/health\": dial tcp 10.132.0.28:8443: connect: connection refused" start-of-body= Jul 12 08:22:42.426166 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:42.426052 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-85b8c4bc8d-nsmwp" podUID="bd26a038-0655-4265-9834-97c2d8de330a" containerName="console" probeResult="failure" output="Get \"https://10.132.0.28:8443/health\": dial tcp 10.132.0.28:8443: connect: connection refused" Jul 12 08:22:43.334206 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:43.334164 2566 patch_prober.go:28] interesting pod/image-registry-5bd7886769-cpjl8 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 12 08:22:43.334374 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:43.334227 2566 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" podUID="f98d0302-b975-4e38-a0d2-1f37eb5679f3" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 12 08:22:43.662589 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:43.662476 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:44.207224 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:44.207126 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:44.257027 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:44.256993 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:45.228626 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:45.228509 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:45.342520 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:45.342472 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:45.440023 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:45.439991 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:22:45.632656 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:45.632621 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5gtms"] Jul 12 08:22:45.635952 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:45.635922 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:45.708862 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:45.708825 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:45.773025 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:45.772966 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:45.831224 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:45.830071 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5gtms" Jul 12 08:22:45.832911 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:45.832857 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-nhgpz\"" Jul 12 08:22:45.834263 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:45.833711 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 12 08:22:45.835325 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:45.834759 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 12 08:22:45.835325 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:45.834986 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 12 08:22:45.835325 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:45.835194 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 12 08:22:45.854088 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:45.854063 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:45.911016 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:45.910936 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:45.924644 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:45.924603 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbfh8\" (UniqueName: \"kubernetes.io/projected/f2a2abd2-70f9-42e1-b68d-7026bc1be2d2-kube-api-access-zbfh8\") pod \"node-exporter-5gtms\" (UID: \"f2a2abd2-70f9-42e1-b68d-7026bc1be2d2\") " pod="openshift-monitoring/node-exporter-5gtms" Jul 12 08:22:45.924779 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:45.924651 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f2a2abd2-70f9-42e1-b68d-7026bc1be2d2-metrics-client-ca\") pod \"node-exporter-5gtms\" (UID: \"f2a2abd2-70f9-42e1-b68d-7026bc1be2d2\") " pod="openshift-monitoring/node-exporter-5gtms" Jul 12 08:22:45.924779 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:45.924715 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f2a2abd2-70f9-42e1-b68d-7026bc1be2d2-node-exporter-accelerators-collector-config\") pod \"node-exporter-5gtms\" (UID: \"f2a2abd2-70f9-42e1-b68d-7026bc1be2d2\") " pod="openshift-monitoring/node-exporter-5gtms" Jul 12 08:22:45.924884 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:45.924792 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f2a2abd2-70f9-42e1-b68d-7026bc1be2d2-node-exporter-textfile\") pod \"node-exporter-5gtms\" (UID: \"f2a2abd2-70f9-42e1-b68d-7026bc1be2d2\") " pod="openshift-monitoring/node-exporter-5gtms" Jul 12 08:22:45.924884 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:45.924827 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f2a2abd2-70f9-42e1-b68d-7026bc1be2d2-node-exporter-tls\") pod \"node-exporter-5gtms\" (UID: \"f2a2abd2-70f9-42e1-b68d-7026bc1be2d2\") " pod="openshift-monitoring/node-exporter-5gtms" Jul 12 08:22:45.924884 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:45.924856 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f2a2abd2-70f9-42e1-b68d-7026bc1be2d2-root\") pod \"node-exporter-5gtms\" (UID: \"f2a2abd2-70f9-42e1-b68d-7026bc1be2d2\") " pod="openshift-monitoring/node-exporter-5gtms" Jul 12 08:22:45.925057 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:45.924905 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2a2abd2-70f9-42e1-b68d-7026bc1be2d2-sys\") pod \"node-exporter-5gtms\" (UID: \"f2a2abd2-70f9-42e1-b68d-7026bc1be2d2\") " pod="openshift-monitoring/node-exporter-5gtms" Jul 12 08:22:45.925057 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:45.924938 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f2a2abd2-70f9-42e1-b68d-7026bc1be2d2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5gtms\" (UID: \"f2a2abd2-70f9-42e1-b68d-7026bc1be2d2\") " pod="openshift-monitoring/node-exporter-5gtms" Jul 12 08:22:45.925057 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:45.924973 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f2a2abd2-70f9-42e1-b68d-7026bc1be2d2-node-exporter-wtmp\") pod \"node-exporter-5gtms\" (UID: \"f2a2abd2-70f9-42e1-b68d-7026bc1be2d2\") " pod="openshift-monitoring/node-exporter-5gtms" Jul 12 08:22:46.011137 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:46.011083 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:46.026101 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:46.026075 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbfh8\" (UniqueName: \"kubernetes.io/projected/f2a2abd2-70f9-42e1-b68d-7026bc1be2d2-kube-api-access-zbfh8\") pod \"node-exporter-5gtms\" (UID: \"f2a2abd2-70f9-42e1-b68d-7026bc1be2d2\") " pod="openshift-monitoring/node-exporter-5gtms" Jul 12 08:22:46.026249 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:46.026112 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f2a2abd2-70f9-42e1-b68d-7026bc1be2d2-metrics-client-ca\") pod \"node-exporter-5gtms\" (UID: \"f2a2abd2-70f9-42e1-b68d-7026bc1be2d2\") " pod="openshift-monitoring/node-exporter-5gtms" Jul 12 08:22:46.026249 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:46.026154 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f2a2abd2-70f9-42e1-b68d-7026bc1be2d2-node-exporter-accelerators-collector-config\") pod \"node-exporter-5gtms\" (UID: \"f2a2abd2-70f9-42e1-b68d-7026bc1be2d2\") " pod="openshift-monitoring/node-exporter-5gtms" Jul 12 08:22:46.026249 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:46.026196 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f2a2abd2-70f9-42e1-b68d-7026bc1be2d2-node-exporter-textfile\") pod \"node-exporter-5gtms\" (UID: \"f2a2abd2-70f9-42e1-b68d-7026bc1be2d2\") " pod="openshift-monitoring/node-exporter-5gtms" Jul 12 08:22:46.026249 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:46.026220 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f2a2abd2-70f9-42e1-b68d-7026bc1be2d2-node-exporter-tls\") pod \"node-exporter-5gtms\" (UID: \"f2a2abd2-70f9-42e1-b68d-7026bc1be2d2\") " pod="openshift-monitoring/node-exporter-5gtms" Jul 12 08:22:46.026249 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:46.026242 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f2a2abd2-70f9-42e1-b68d-7026bc1be2d2-root\") pod \"node-exporter-5gtms\" (UID: \"f2a2abd2-70f9-42e1-b68d-7026bc1be2d2\") " pod="openshift-monitoring/node-exporter-5gtms" Jul 12 08:22:46.026494 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:46.026285 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2a2abd2-70f9-42e1-b68d-7026bc1be2d2-sys\") pod \"node-exporter-5gtms\" (UID: \"f2a2abd2-70f9-42e1-b68d-7026bc1be2d2\") " pod="openshift-monitoring/node-exporter-5gtms" Jul 12 08:22:46.026494 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:46.026315 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f2a2abd2-70f9-42e1-b68d-7026bc1be2d2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5gtms\" (UID: \"f2a2abd2-70f9-42e1-b68d-7026bc1be2d2\") " pod="openshift-monitoring/node-exporter-5gtms" Jul 12 08:22:46.026494 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:46.026348 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f2a2abd2-70f9-42e1-b68d-7026bc1be2d2-node-exporter-wtmp\") pod \"node-exporter-5gtms\" (UID: \"f2a2abd2-70f9-42e1-b68d-7026bc1be2d2\") " pod="openshift-monitoring/node-exporter-5gtms" Jul 12 08:22:46.026672 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:46.026510 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f2a2abd2-70f9-42e1-b68d-7026bc1be2d2-node-exporter-wtmp\") pod \"node-exporter-5gtms\" (UID: \"f2a2abd2-70f9-42e1-b68d-7026bc1be2d2\") " pod="openshift-monitoring/node-exporter-5gtms" Jul 12 08:22:46.026672 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:46.026595 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f2a2abd2-70f9-42e1-b68d-7026bc1be2d2-root\") pod \"node-exporter-5gtms\" (UID: \"f2a2abd2-70f9-42e1-b68d-7026bc1be2d2\") " pod="openshift-monitoring/node-exporter-5gtms" Jul 12 08:22:46.026672 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:46.026650 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2a2abd2-70f9-42e1-b68d-7026bc1be2d2-sys\") pod \"node-exporter-5gtms\" (UID: \"f2a2abd2-70f9-42e1-b68d-7026bc1be2d2\") " pod="openshift-monitoring/node-exporter-5gtms" Jul 12 08:22:46.026819 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:46.026782 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f2a2abd2-70f9-42e1-b68d-7026bc1be2d2-metrics-client-ca\") pod \"node-exporter-5gtms\" (UID: \"f2a2abd2-70f9-42e1-b68d-7026bc1be2d2\") " pod="openshift-monitoring/node-exporter-5gtms" Jul 12 08:22:46.026924 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:46.026905 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f2a2abd2-70f9-42e1-b68d-7026bc1be2d2-node-exporter-textfile\") pod \"node-exporter-5gtms\" (UID: \"f2a2abd2-70f9-42e1-b68d-7026bc1be2d2\") " pod="openshift-monitoring/node-exporter-5gtms" Jul 12 08:22:46.027093 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:46.027068 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f2a2abd2-70f9-42e1-b68d-7026bc1be2d2-node-exporter-accelerators-collector-config\") pod \"node-exporter-5gtms\" (UID: \"f2a2abd2-70f9-42e1-b68d-7026bc1be2d2\") " pod="openshift-monitoring/node-exporter-5gtms" Jul 12 08:22:46.029148 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:46.029125 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f2a2abd2-70f9-42e1-b68d-7026bc1be2d2-node-exporter-tls\") pod \"node-exporter-5gtms\" (UID: \"f2a2abd2-70f9-42e1-b68d-7026bc1be2d2\") " pod="openshift-monitoring/node-exporter-5gtms" Jul 12 08:22:46.029458 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:46.029436 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f2a2abd2-70f9-42e1-b68d-7026bc1be2d2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5gtms\" (UID: \"f2a2abd2-70f9-42e1-b68d-7026bc1be2d2\") " pod="openshift-monitoring/node-exporter-5gtms" Jul 12 08:22:46.033800 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:46.033778 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbfh8\" (UniqueName: \"kubernetes.io/projected/f2a2abd2-70f9-42e1-b68d-7026bc1be2d2-kube-api-access-zbfh8\") pod \"node-exporter-5gtms\" (UID: \"f2a2abd2-70f9-42e1-b68d-7026bc1be2d2\") " pod="openshift-monitoring/node-exporter-5gtms" Jul 12 08:22:46.146983 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:46.146948 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5gtms" Jul 12 08:22:46.190355 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:46.190321 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:46.420409 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:46.420280 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:46.594265 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:46.594218 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:46.654014 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:46.653953 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:46.724658 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:46.724579 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:47.871677 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:47.871644 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-657bf5f5cd-fgk22" Jul 12 08:22:47.872095 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:47.871714 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-657bf5f5cd-fgk22" Jul 12 08:22:47.877059 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:47.877033 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-657bf5f5cd-fgk22" Jul 12 08:22:48.064921 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:48.064863 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:48.548848 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:48.548801 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:48.629619 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:48.629323 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-657bf5f5cd-fgk22" Jul 12 08:22:48.649228 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:48.649194 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:48.677396 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:22:48.677354 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2a2abd2_70f9_42e1_b68d_7026bc1be2d2.slice/crio-e7288612174bb2defea056a5472624c7fb2465ed3a6cd2e58b2865c0996928af WatchSource:0}: Error finding container e7288612174bb2defea056a5472624c7fb2465ed3a6cd2e58b2865c0996928af: Status 404 returned error can't find the container with id e7288612174bb2defea056a5472624c7fb2465ed3a6cd2e58b2865c0996928af Jul 12 08:22:48.758645 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:48.758606 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:48.859066 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:48.858974 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:49.093864 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:49.093825 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:49.630701 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:49.630659 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-nslt6" event={"ID":"360e4c5b-afe0-4bb4-88d7-5e419497559d","Type":"ContainerStarted","Data":"4f27a8454030ef82b62608cea659de146d462c5074ed5941dac058dea824145b"} Jul 12 08:22:49.631992 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:49.631659 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-nslt6" Jul 12 08:22:49.633969 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:49.633938 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5gtms" event={"ID":"f2a2abd2-70f9-42e1-b68d-7026bc1be2d2","Type":"ContainerStarted","Data":"e7288612174bb2defea056a5472624c7fb2465ed3a6cd2e58b2865c0996928af"} Jul 12 08:22:49.645291 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:49.645267 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-nslt6" Jul 12 08:22:49.647451 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:49.647404 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-nslt6" podStartSLOduration=3.244565725 podStartE2EDuration="22.647387741s" totalImagesPullingTime="19.402822016s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:22:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:22:49.646288558 +0000 UTC m=+93.307084243" watchObservedRunningTime="2026-07-12 08:22:49.647387741 +0000 UTC m=+93.308183403" Jul 12 08:22:49.649814 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:49.649789 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:49.715303 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:49.715266 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:50.230361 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:50.230317 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:50.332719 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:50.332687 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:50.399148 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:50.399112 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:50.455346 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:50.455312 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:50.641523 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:50.641439 2566 generic.go:356] "Generic (PLEG): container finished" podID="f2a2abd2-70f9-42e1-b68d-7026bc1be2d2" containerID="81b598fc8d489514368f2bc8e1a2ff906fa1a4d264c8ee80975e37c6005e055b" exitCode=0 Jul 12 08:22:50.641701 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:50.641529 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5gtms" event={"ID":"f2a2abd2-70f9-42e1-b68d-7026bc1be2d2","Type":"ContainerDied","Data":"81b598fc8d489514368f2bc8e1a2ff906fa1a4d264c8ee80975e37c6005e055b"} Jul 12 08:22:50.855352 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:50.855313 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:50.954420 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:50.954385 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:51.263677 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:51.263642 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85b8c4bc8d-nsmwp"] Jul 12 08:22:51.649675 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:51.649592 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5gtms" event={"ID":"f2a2abd2-70f9-42e1-b68d-7026bc1be2d2","Type":"ContainerStarted","Data":"f89eae0cf38c26dc7f9190a41bb29a8c2dd4b2aeb558b9fa16cbb9fb2c1ce0a0"} Jul 12 08:22:51.649675 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:51.649659 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5gtms" event={"ID":"f2a2abd2-70f9-42e1-b68d-7026bc1be2d2","Type":"ContainerStarted","Data":"44b292a96a309cab837fa993210133b987cc86fd5be7558a832060fb46aaaadc"} Jul 12 08:22:51.667471 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:51.667415 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5gtms" podStartSLOduration=5.198820814 podStartE2EDuration="6.667395658s" totalImagesPullingTime="1.367266898s" totalInitContainerRuntime="101.307946ms" isStatefulPod=true podCreationTimestamp="2026-07-12 08:22:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:22:51.66640039 +0000 UTC m=+95.327196075" watchObservedRunningTime="2026-07-12 08:22:51.667395658 +0000 UTC m=+95.328191319" Jul 12 08:22:51.841622 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:51.841586 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:52.922771 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:52.922728 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:53.023657 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:53.023614 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:54.434457 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:54.434426 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-hr58s" Jul 12 08:22:55.109230 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:55.109191 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:55.210411 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:55.210374 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:56.864575 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:56.862597 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5bd7886769-cpjl8"] Jul 12 08:22:56.943968 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:56.943923 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:57.046607 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:57.046571 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:58.021941 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:58.021882 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:58.121503 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:58.121452 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:58.222528 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:58.222499 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:59.208746 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:59.208714 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:59.308896 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:59.308867 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:22:59.949396 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:22:59.949352 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:23:00.048309 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:00.048274 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:23:01.781021 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:01.780988 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-657bf5f5cd-fgk22"] Jul 12 08:23:01.891650 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:01.891605 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:23:01.994822 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:01.994790 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:23:02.094061 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:02.093970 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:23:06.718993 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:06.718910 2566 generic.go:356] "Generic (PLEG): container finished" podID="40be23cf-4432-4ea2-9979-3ae525ba3ef9" containerID="2a3c25fa32c316b5b1ee68883eb403fafbf9b65f19067370b0a7c19ef4a7f2df" exitCode=0 Jul 12 08:23:06.719485 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:06.718989 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4z9qg" event={"ID":"40be23cf-4432-4ea2-9979-3ae525ba3ef9","Type":"ContainerDied","Data":"2a3c25fa32c316b5b1ee68883eb403fafbf9b65f19067370b0a7c19ef4a7f2df"} Jul 12 08:23:06.719485 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:06.719380 2566 scope.go:117] "RemoveContainer" containerID="2a3c25fa32c316b5b1ee68883eb403fafbf9b65f19067370b0a7c19ef4a7f2df" Jul 12 08:23:07.725063 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:07.725027 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-4z9qg" event={"ID":"40be23cf-4432-4ea2-9979-3ae525ba3ef9","Type":"ContainerStarted","Data":"4597b763786d016a8802de801a03fed467643df9068842af8c7bbbb0f00173f3"} Jul 12 08:23:08.943424 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:08.943380 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:23:14.200830 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:14.200791 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:23:14.261813 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:14.261776 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:23:16.288125 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:16.288088 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-85b8c4bc8d-nsmwp" podUID="bd26a038-0655-4265-9834-97c2d8de330a" containerName="console" containerID="cri-o://89f85aa563dae37e7811543e6c656eda5ab2a3bc129c8a5a8a4e01e13daebee3" gracePeriod=15 Jul 12 08:23:16.574728 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:16.574696 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85b8c4bc8d-nsmwp" Jul 12 08:23:16.690354 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:16.690271 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd26a038-0655-4265-9834-97c2d8de330a-trusted-ca-bundle\") pod \"bd26a038-0655-4265-9834-97c2d8de330a\" (UID: \"bd26a038-0655-4265-9834-97c2d8de330a\") " Jul 12 08:23:16.690354 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:16.690325 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd26a038-0655-4265-9834-97c2d8de330a-service-ca\") pod \"bd26a038-0655-4265-9834-97c2d8de330a\" (UID: \"bd26a038-0655-4265-9834-97c2d8de330a\") " Jul 12 08:23:16.690354 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:16.690355 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4h5rg\" (UniqueName: \"kubernetes.io/projected/bd26a038-0655-4265-9834-97c2d8de330a-kube-api-access-4h5rg\") pod \"bd26a038-0655-4265-9834-97c2d8de330a\" (UID: \"bd26a038-0655-4265-9834-97c2d8de330a\") " Jul 12 08:23:16.690664 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:16.690386 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd26a038-0655-4265-9834-97c2d8de330a-console-serving-cert\") pod \"bd26a038-0655-4265-9834-97c2d8de330a\" (UID: \"bd26a038-0655-4265-9834-97c2d8de330a\") " Jul 12 08:23:16.690664 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:16.690420 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd26a038-0655-4265-9834-97c2d8de330a-oauth-serving-cert\") pod \"bd26a038-0655-4265-9834-97c2d8de330a\" (UID: \"bd26a038-0655-4265-9834-97c2d8de330a\") " Jul 12 08:23:16.690664 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:16.690440 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd26a038-0655-4265-9834-97c2d8de330a-console-oauth-config\") pod \"bd26a038-0655-4265-9834-97c2d8de330a\" (UID: \"bd26a038-0655-4265-9834-97c2d8de330a\") " Jul 12 08:23:16.690664 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:16.690461 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd26a038-0655-4265-9834-97c2d8de330a-console-config\") pod \"bd26a038-0655-4265-9834-97c2d8de330a\" (UID: \"bd26a038-0655-4265-9834-97c2d8de330a\") " Jul 12 08:23:16.690863 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:16.690761 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd26a038-0655-4265-9834-97c2d8de330a-service-ca" (OuterVolumeSpecName: "service-ca") pod "bd26a038-0655-4265-9834-97c2d8de330a" (UID: "bd26a038-0655-4265-9834-97c2d8de330a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 08:23:16.690936 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:16.690896 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd26a038-0655-4265-9834-97c2d8de330a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bd26a038-0655-4265-9834-97c2d8de330a" (UID: "bd26a038-0655-4265-9834-97c2d8de330a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 08:23:16.690936 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:16.690921 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd26a038-0655-4265-9834-97c2d8de330a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "bd26a038-0655-4265-9834-97c2d8de330a" (UID: "bd26a038-0655-4265-9834-97c2d8de330a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 08:23:16.691028 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:16.690992 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd26a038-0655-4265-9834-97c2d8de330a-console-config" (OuterVolumeSpecName: "console-config") pod "bd26a038-0655-4265-9834-97c2d8de330a" (UID: "bd26a038-0655-4265-9834-97c2d8de330a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 08:23:16.692720 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:16.692696 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd26a038-0655-4265-9834-97c2d8de330a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bd26a038-0655-4265-9834-97c2d8de330a" (UID: "bd26a038-0655-4265-9834-97c2d8de330a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 08:23:16.692720 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:16.692703 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd26a038-0655-4265-9834-97c2d8de330a-kube-api-access-4h5rg" (OuterVolumeSpecName: "kube-api-access-4h5rg") pod "bd26a038-0655-4265-9834-97c2d8de330a" (UID: "bd26a038-0655-4265-9834-97c2d8de330a"). InnerVolumeSpecName "kube-api-access-4h5rg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:23:16.692874 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:16.692843 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd26a038-0655-4265-9834-97c2d8de330a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bd26a038-0655-4265-9834-97c2d8de330a" (UID: "bd26a038-0655-4265-9834-97c2d8de330a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 08:23:16.762142 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:16.762110 2566 generic.go:356] "Generic (PLEG): container finished" podID="bd26a038-0655-4265-9834-97c2d8de330a" containerID="89f85aa563dae37e7811543e6c656eda5ab2a3bc129c8a5a8a4e01e13daebee3" exitCode=0 Jul 12 08:23:16.762286 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:16.762162 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85b8c4bc8d-nsmwp" Jul 12 08:23:16.762286 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:16.762188 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85b8c4bc8d-nsmwp" event={"ID":"bd26a038-0655-4265-9834-97c2d8de330a","Type":"ContainerDied","Data":"89f85aa563dae37e7811543e6c656eda5ab2a3bc129c8a5a8a4e01e13daebee3"} Jul 12 08:23:16.762286 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:16.762257 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85b8c4bc8d-nsmwp" event={"ID":"bd26a038-0655-4265-9834-97c2d8de330a","Type":"ContainerDied","Data":"25f250358803e149121878a35768676b6a3fb564adb8c583c7fd84b9f4315338"} Jul 12 08:23:16.762286 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:16.762278 2566 scope.go:117] "RemoveContainer" containerID="89f85aa563dae37e7811543e6c656eda5ab2a3bc129c8a5a8a4e01e13daebee3" Jul 12 08:23:16.763763 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:16.763744 2566 generic.go:356] "Generic (PLEG): container finished" podID="e142390b-f773-488a-92de-6883d5f4ad70" containerID="7fc22fdfb89d8d9b8b484653cdf945a00ecc41aefa91320d277109ef560b7a52" exitCode=0 Jul 12 08:23:16.763845 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:16.763774 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-rvkmd" event={"ID":"e142390b-f773-488a-92de-6883d5f4ad70","Type":"ContainerDied","Data":"7fc22fdfb89d8d9b8b484653cdf945a00ecc41aefa91320d277109ef560b7a52"} Jul 12 08:23:16.764051 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:16.764040 2566 scope.go:117] "RemoveContainer" containerID="7fc22fdfb89d8d9b8b484653cdf945a00ecc41aefa91320d277109ef560b7a52" Jul 12 08:23:16.771136 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:16.771120 2566 scope.go:117] "RemoveContainer" containerID="89f85aa563dae37e7811543e6c656eda5ab2a3bc129c8a5a8a4e01e13daebee3" Jul 12 08:23:16.771380 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:23:16.771360 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89f85aa563dae37e7811543e6c656eda5ab2a3bc129c8a5a8a4e01e13daebee3\": container with ID starting with 89f85aa563dae37e7811543e6c656eda5ab2a3bc129c8a5a8a4e01e13daebee3 not found: ID does not exist" containerID="89f85aa563dae37e7811543e6c656eda5ab2a3bc129c8a5a8a4e01e13daebee3" Jul 12 08:23:16.771435 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:16.771388 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89f85aa563dae37e7811543e6c656eda5ab2a3bc129c8a5a8a4e01e13daebee3"} err="failed to get container status \"89f85aa563dae37e7811543e6c656eda5ab2a3bc129c8a5a8a4e01e13daebee3\": rpc error: code = NotFound desc = could not find container \"89f85aa563dae37e7811543e6c656eda5ab2a3bc129c8a5a8a4e01e13daebee3\": container with ID starting with 89f85aa563dae37e7811543e6c656eda5ab2a3bc129c8a5a8a4e01e13daebee3 not found: ID does not exist" Jul 12 08:23:16.791149 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:16.791128 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd26a038-0655-4265-9834-97c2d8de330a-trusted-ca-bundle\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:23:16.791149 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:16.791150 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd26a038-0655-4265-9834-97c2d8de330a-service-ca\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:23:16.791149 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:16.791161 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4h5rg\" (UniqueName: \"kubernetes.io/projected/bd26a038-0655-4265-9834-97c2d8de330a-kube-api-access-4h5rg\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:23:16.791322 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:16.791170 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd26a038-0655-4265-9834-97c2d8de330a-console-serving-cert\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:23:16.791322 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:16.791179 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd26a038-0655-4265-9834-97c2d8de330a-oauth-serving-cert\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:23:16.791322 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:16.791207 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd26a038-0655-4265-9834-97c2d8de330a-console-oauth-config\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:23:16.791322 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:16.791216 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd26a038-0655-4265-9834-97c2d8de330a-console-config\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:23:16.801969 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:16.801946 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85b8c4bc8d-nsmwp"] Jul 12 08:23:16.807984 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:16.807960 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-85b8c4bc8d-nsmwp"] Jul 12 08:23:17.026574 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:17.026528 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd26a038-0655-4265-9834-97c2d8de330a" path="/var/lib/kubelet/pods/bd26a038-0655-4265-9834-97c2d8de330a/volumes" Jul 12 08:23:17.307765 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:17.307694 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-b8989f8c6-v5dgq_1bd2626f-14a5-450e-ae9a-6968c5aa9055/router/0.log" Jul 12 08:23:17.318651 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:17.318619 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4p4w4_f5e63ffc-280a-49f6-8861-6b5bcf16ac29/serve-healthcheck-canary/0.log" Jul 12 08:23:17.769919 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:17.769882 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-rvkmd" event={"ID":"e142390b-f773-488a-92de-6883d5f4ad70","Type":"ContainerStarted","Data":"8ad536b15c9e44e1528d108081190965d0cbf8b4ab852c9f73aa283c87df46c4"} Jul 12 08:23:21.789355 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:21.789324 2566 generic.go:356] "Generic (PLEG): container finished" podID="5cf8c8a8-825a-404d-b968-dfb4f4f466df" containerID="487de75e0d2c54697dc41019aeca0c18b1e8356bd42097b45853a1be779ae041" exitCode=0 Jul 12 08:23:21.789826 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:21.789407 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n9s6q" event={"ID":"5cf8c8a8-825a-404d-b968-dfb4f4f466df","Type":"ContainerDied","Data":"487de75e0d2c54697dc41019aeca0c18b1e8356bd42097b45853a1be779ae041"} Jul 12 08:23:21.789826 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:21.789729 2566 scope.go:117] "RemoveContainer" containerID="487de75e0d2c54697dc41019aeca0c18b1e8356bd42097b45853a1be779ae041" Jul 12 08:23:21.893956 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:21.893918 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" podUID="f98d0302-b975-4e38-a0d2-1f37eb5679f3" containerName="registry" containerID="cri-o://0e11bb0f270a124a73cd85575038ced2613d51bccdf798b200e9de8015a35cf1" gracePeriod=30 Jul 12 08:23:22.161222 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:22.161198 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:23:22.341385 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:22.341312 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rh858\" (UniqueName: \"kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-kube-api-access-rh858\") pod \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " Jul 12 08:23:22.341385 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:22.341347 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f98d0302-b975-4e38-a0d2-1f37eb5679f3-installation-pull-secrets\") pod \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " Jul 12 08:23:22.341613 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:22.341386 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f98d0302-b975-4e38-a0d2-1f37eb5679f3-trusted-ca\") pod \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " Jul 12 08:23:22.341613 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:22.341501 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-bound-sa-token\") pod \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " Jul 12 08:23:22.341613 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:22.341533 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-registry-tls\") pod \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " Jul 12 08:23:22.341613 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:22.341580 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f98d0302-b975-4e38-a0d2-1f37eb5679f3-ca-trust-extracted\") pod \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " Jul 12 08:23:22.341814 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:22.341619 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f98d0302-b975-4e38-a0d2-1f37eb5679f3-image-registry-private-configuration\") pod \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " Jul 12 08:23:22.341814 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:22.341663 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f98d0302-b975-4e38-a0d2-1f37eb5679f3-registry-certificates\") pod \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\" (UID: \"f98d0302-b975-4e38-a0d2-1f37eb5679f3\") " Jul 12 08:23:22.341939 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:22.341910 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f98d0302-b975-4e38-a0d2-1f37eb5679f3-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f98d0302-b975-4e38-a0d2-1f37eb5679f3" (UID: "f98d0302-b975-4e38-a0d2-1f37eb5679f3"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 08:23:22.342243 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:22.342217 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f98d0302-b975-4e38-a0d2-1f37eb5679f3-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f98d0302-b975-4e38-a0d2-1f37eb5679f3" (UID: "f98d0302-b975-4e38-a0d2-1f37eb5679f3"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 08:23:22.344256 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:22.344216 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f98d0302-b975-4e38-a0d2-1f37eb5679f3" (UID: "f98d0302-b975-4e38-a0d2-1f37eb5679f3"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:23:22.344377 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:22.344323 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-kube-api-access-rh858" (OuterVolumeSpecName: "kube-api-access-rh858") pod "f98d0302-b975-4e38-a0d2-1f37eb5679f3" (UID: "f98d0302-b975-4e38-a0d2-1f37eb5679f3"). InnerVolumeSpecName "kube-api-access-rh858". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:23:22.344539 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:22.344514 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f98d0302-b975-4e38-a0d2-1f37eb5679f3" (UID: "f98d0302-b975-4e38-a0d2-1f37eb5679f3"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:23:22.344647 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:22.344533 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f98d0302-b975-4e38-a0d2-1f37eb5679f3-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f98d0302-b975-4e38-a0d2-1f37eb5679f3" (UID: "f98d0302-b975-4e38-a0d2-1f37eb5679f3"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 08:23:22.344847 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:22.344829 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f98d0302-b975-4e38-a0d2-1f37eb5679f3-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f98d0302-b975-4e38-a0d2-1f37eb5679f3" (UID: "f98d0302-b975-4e38-a0d2-1f37eb5679f3"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 08:23:22.352858 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:22.352825 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f98d0302-b975-4e38-a0d2-1f37eb5679f3-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f98d0302-b975-4e38-a0d2-1f37eb5679f3" (UID: "f98d0302-b975-4e38-a0d2-1f37eb5679f3"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:23:22.442638 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:22.442606 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rh858\" (UniqueName: \"kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-kube-api-access-rh858\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:23:22.442638 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:22.442632 2566 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f98d0302-b975-4e38-a0d2-1f37eb5679f3-installation-pull-secrets\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:23:22.442638 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:22.442641 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f98d0302-b975-4e38-a0d2-1f37eb5679f3-trusted-ca\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:23:22.442818 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:22.442650 2566 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-bound-sa-token\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:23:22.442818 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:22.442658 2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f98d0302-b975-4e38-a0d2-1f37eb5679f3-registry-tls\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:23:22.442818 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:22.442666 2566 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f98d0302-b975-4e38-a0d2-1f37eb5679f3-ca-trust-extracted\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:23:22.442818 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:22.442676 2566 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f98d0302-b975-4e38-a0d2-1f37eb5679f3-image-registry-private-configuration\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:23:22.442818 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:22.442686 2566 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f98d0302-b975-4e38-a0d2-1f37eb5679f3-registry-certificates\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:23:22.795386 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:22.795350 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n9s6q" event={"ID":"5cf8c8a8-825a-404d-b968-dfb4f4f466df","Type":"ContainerStarted","Data":"2bdbb8d9680951a943d97a2b4764140ad9d4fda848ee2aec1062eb259abe4705"} Jul 12 08:23:22.796515 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:22.796495 2566 generic.go:356] "Generic (PLEG): container finished" podID="f98d0302-b975-4e38-a0d2-1f37eb5679f3" containerID="0e11bb0f270a124a73cd85575038ced2613d51bccdf798b200e9de8015a35cf1" exitCode=0 Jul 12 08:23:22.796606 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:22.796522 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" event={"ID":"f98d0302-b975-4e38-a0d2-1f37eb5679f3","Type":"ContainerDied","Data":"0e11bb0f270a124a73cd85575038ced2613d51bccdf798b200e9de8015a35cf1"} Jul 12 08:23:22.796606 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:22.796538 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" Jul 12 08:23:22.796606 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:22.796585 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bd7886769-cpjl8" event={"ID":"f98d0302-b975-4e38-a0d2-1f37eb5679f3","Type":"ContainerDied","Data":"842081e511cb25b6319a142807a1ac86d7f4b08f960d51edd16a7b138484d1f5"} Jul 12 08:23:22.796606 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:22.796606 2566 scope.go:117] "RemoveContainer" containerID="0e11bb0f270a124a73cd85575038ced2613d51bccdf798b200e9de8015a35cf1" Jul 12 08:23:22.805333 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:22.805315 2566 scope.go:117] "RemoveContainer" containerID="0e11bb0f270a124a73cd85575038ced2613d51bccdf798b200e9de8015a35cf1" Jul 12 08:23:22.805586 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:23:22.805568 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e11bb0f270a124a73cd85575038ced2613d51bccdf798b200e9de8015a35cf1\": container with ID starting with 0e11bb0f270a124a73cd85575038ced2613d51bccdf798b200e9de8015a35cf1 not found: ID does not exist" containerID="0e11bb0f270a124a73cd85575038ced2613d51bccdf798b200e9de8015a35cf1" Jul 12 08:23:22.805634 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:22.805595 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e11bb0f270a124a73cd85575038ced2613d51bccdf798b200e9de8015a35cf1"} err="failed to get container status \"0e11bb0f270a124a73cd85575038ced2613d51bccdf798b200e9de8015a35cf1\": rpc error: code = NotFound desc = could not find container \"0e11bb0f270a124a73cd85575038ced2613d51bccdf798b200e9de8015a35cf1\": container with ID starting with 0e11bb0f270a124a73cd85575038ced2613d51bccdf798b200e9de8015a35cf1 not found: ID does not exist" Jul 12 08:23:22.825347 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:22.825320 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5bd7886769-cpjl8"] Jul 12 08:23:22.832806 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:22.832784 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5bd7886769-cpjl8"] Jul 12 08:23:23.026080 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:23.026046 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f98d0302-b975-4e38-a0d2-1f37eb5679f3" path="/var/lib/kubelet/pods/f98d0302-b975-4e38-a0d2-1f37eb5679f3/volumes" Jul 12 08:23:26.804150 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:26.804103 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-657bf5f5cd-fgk22" podUID="9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683" containerName="console" containerID="cri-o://89e4e0063ab821c9c8a1d3dc922763b701e2157a0bcb0c2cc11536b903f79849" gracePeriod=15 Jul 12 08:23:27.077719 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:27.077664 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-657bf5f5cd-fgk22" Jul 12 08:23:27.178159 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:27.178124 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-service-ca\") pod \"9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683\" (UID: \"9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683\") " Jul 12 08:23:27.178328 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:27.178179 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-oauth-serving-cert\") pod \"9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683\" (UID: \"9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683\") " Jul 12 08:23:27.178328 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:27.178210 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-console-serving-cert\") pod \"9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683\" (UID: \"9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683\") " Jul 12 08:23:27.178328 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:27.178234 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-console-config\") pod \"9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683\" (UID: \"9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683\") " Jul 12 08:23:27.178543 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:27.178356 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-console-oauth-config\") pod \"9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683\" (UID: \"9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683\") " Jul 12 08:23:27.178543 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:27.178410 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8qvhz\" (UniqueName: \"kubernetes.io/projected/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-kube-api-access-8qvhz\") pod \"9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683\" (UID: \"9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683\") " Jul 12 08:23:27.178677 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:27.178640 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-service-ca" (OuterVolumeSpecName: "service-ca") pod "9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683" (UID: "9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 08:23:27.178677 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:27.178658 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683" (UID: "9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 08:23:27.178781 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:27.178676 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-console-config" (OuterVolumeSpecName: "console-config") pod "9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683" (UID: "9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 08:23:27.180584 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:27.180542 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683" (UID: "9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 08:23:27.180673 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:27.180605 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-kube-api-access-8qvhz" (OuterVolumeSpecName: "kube-api-access-8qvhz") pod "9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683" (UID: "9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683"). InnerVolumeSpecName "kube-api-access-8qvhz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:23:27.180673 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:27.180629 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683" (UID: "9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 08:23:27.279848 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:27.279812 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-console-serving-cert\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:23:27.279848 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:27.279845 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-console-config\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:23:27.280039 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:27.279859 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-console-oauth-config\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:23:27.280039 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:27.279872 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qvhz\" (UniqueName: \"kubernetes.io/projected/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-kube-api-access-8qvhz\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:23:27.280039 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:27.279885 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-service-ca\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:23:27.280039 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:27.279895 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683-oauth-serving-cert\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:23:27.819258 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:27.819216 2566 generic.go:356] "Generic (PLEG): container finished" podID="9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683" containerID="89e4e0063ab821c9c8a1d3dc922763b701e2157a0bcb0c2cc11536b903f79849" exitCode=0 Jul 12 08:23:27.819780 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:27.819348 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-657bf5f5cd-fgk22" event={"ID":"9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683","Type":"ContainerDied","Data":"89e4e0063ab821c9c8a1d3dc922763b701e2157a0bcb0c2cc11536b903f79849"} Jul 12 08:23:27.819780 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:27.819395 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-657bf5f5cd-fgk22" event={"ID":"9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683","Type":"ContainerDied","Data":"c38ea892d0e7dc3d39b58a1d7d5e81283da9c7aafc2bc1aee6a1ddf2384d8a84"} Jul 12 08:23:27.819780 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:27.819423 2566 scope.go:117] "RemoveContainer" containerID="89e4e0063ab821c9c8a1d3dc922763b701e2157a0bcb0c2cc11536b903f79849" Jul 12 08:23:27.819780 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:27.819483 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-657bf5f5cd-fgk22" Jul 12 08:23:27.829249 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:27.829229 2566 scope.go:117] "RemoveContainer" containerID="89e4e0063ab821c9c8a1d3dc922763b701e2157a0bcb0c2cc11536b903f79849" Jul 12 08:23:27.830024 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:23:27.829688 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89e4e0063ab821c9c8a1d3dc922763b701e2157a0bcb0c2cc11536b903f79849\": container with ID starting with 89e4e0063ab821c9c8a1d3dc922763b701e2157a0bcb0c2cc11536b903f79849 not found: ID does not exist" containerID="89e4e0063ab821c9c8a1d3dc922763b701e2157a0bcb0c2cc11536b903f79849" Jul 12 08:23:27.830024 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:27.829717 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89e4e0063ab821c9c8a1d3dc922763b701e2157a0bcb0c2cc11536b903f79849"} err="failed to get container status \"89e4e0063ab821c9c8a1d3dc922763b701e2157a0bcb0c2cc11536b903f79849\": rpc error: code = NotFound desc = could not find container \"89e4e0063ab821c9c8a1d3dc922763b701e2157a0bcb0c2cc11536b903f79849\": container with ID starting with 89e4e0063ab821c9c8a1d3dc922763b701e2157a0bcb0c2cc11536b903f79849 not found: ID does not exist" Jul 12 08:23:27.842157 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:27.842128 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-657bf5f5cd-fgk22"] Jul 12 08:23:27.844015 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:27.843990 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-657bf5f5cd-fgk22"] Jul 12 08:23:28.837599 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:28.837535 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:23:28.931163 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:28.931117 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:23:29.029150 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:29.028751 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683" path="/var/lib/kubelet/pods/9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683/volumes" Jul 12 08:23:32.384324 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:32.384285 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:23:32.485323 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:32.485287 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:23:32.537362 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:23:32.537328 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:24:04.821797 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:24:04.821756 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:24:05.059201 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:24:05.059167 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:24:10.008090 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:24:10.008007 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:24:14.780239 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:24:14.780200 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:24:14.836860 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:24:14.836826 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:24:20.041936 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:24:20.041900 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:24:20.091573 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:24:20.091515 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:24:23.018521 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:24:23.018478 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:24:23.104191 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:24:23.104156 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:24:23.547110 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:24:23.547069 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:24:23.598412 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:24:23.598375 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:24:32.028138 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:24:32.028095 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:24:32.129836 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:24:32.129800 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:24:33.250751 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:24:33.250719 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:24:33.409402 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:24:33.409361 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:24:33.535191 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:24:33.535077 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:24:44.222358 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:24:44.222315 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:24:44.324000 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:24:44.323971 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:24:47.417699 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:24:47.417662 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:24:47.473834 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:24:47.473791 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:24:54.060368 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:24:54.060331 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:25:55.246148 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:25:55.246116 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:25:55.342800 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:25:55.342766 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:26:16.727051 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:16.727020 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-k2ltl_bc3f8e97-c9ab-4370-b437-8a47514c869c/console-operator/1.log" Jul 12 08:26:16.730577 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:16.730532 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-k2ltl_bc3f8e97-c9ab-4370-b437-8a47514c869c/console-operator/1.log" Jul 12 08:26:16.732957 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:16.732937 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lfbsq_abe468eb-f6d3-4af4-bf5e-a3c029a404ab/ovn-acl-logging/0.log" Jul 12 08:26:16.736762 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:16.736736 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lfbsq_abe468eb-f6d3-4af4-bf5e-a3c029a404ab/ovn-acl-logging/0.log" Jul 12 08:26:16.738239 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:16.738223 2566 kubelet.go:1648] "Image garbage collection succeeded" Jul 12 08:26:16.992000 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:16.991966 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:26:17.842956 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:17.842921 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:26:20.805477 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:20.805436 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:26:22.065808 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:22.065770 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:26:22.589156 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:22.589123 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:26:22.678743 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:22.678703 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:26:24.788152 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:24.788115 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:26:25.147347 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:25.147264 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:26:27.594510 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:27.594476 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:26:27.695057 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:27.695027 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:26:31.012534 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:31.012491 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:26:31.111113 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:31.111078 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:26:32.751153 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:32.751107 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:26:36.190670 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:36.190629 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:26:36.291472 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:36.291423 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:26:36.870400 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:36.870365 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:26:36.973384 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:36.973350 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:26:38.241658 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:38.241598 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:26:53.447229 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:53.447178 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:26:53.540196 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:53.540159 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:26:54.021705 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.021669 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:26:54.356944 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.356858 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:26:54.452136 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.452098 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:26:54.517726 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.517696 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4"] Jul 12 08:26:54.518034 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.518023 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f98d0302-b975-4e38-a0d2-1f37eb5679f3" containerName="registry" Jul 12 08:26:54.518080 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.518036 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f98d0302-b975-4e38-a0d2-1f37eb5679f3" containerName="registry" Jul 12 08:26:54.518080 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.518044 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683" containerName="console" Jul 12 08:26:54.518080 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.518049 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683" containerName="console" Jul 12 08:26:54.518080 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.518057 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd26a038-0655-4265-9834-97c2d8de330a" containerName="console" Jul 12 08:26:54.518080 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.518062 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd26a038-0655-4265-9834-97c2d8de330a" containerName="console" Jul 12 08:26:54.518221 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.518108 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd26a038-0655-4265-9834-97c2d8de330a" containerName="console" Jul 12 08:26:54.518221 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.518116 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f98d0302-b975-4e38-a0d2-1f37eb5679f3" containerName="registry" Jul 12 08:26:54.518221 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.518124 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ef22f9b-9b2b-4d3b-9b82-0b56ed2d7683" containerName="console" Jul 12 08:26:54.520074 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.520058 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4" Jul 12 08:26:54.522474 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.522452 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 12 08:26:54.522578 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.522541 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-xhbr4\"" Jul 12 08:26:54.522803 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.522782 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 12 08:26:54.522915 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.522826 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 12 08:26:54.522964 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.522947 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 12 08:26:54.543924 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.543902 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4"] Jul 12 08:26:54.639936 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.639850 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9942x\" (UniqueName: \"kubernetes.io/projected/19bc5347-542e-4e2d-8de3-1a2c1cb298c0-kube-api-access-9942x\") pod \"opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4\" (UID: \"19bc5347-542e-4e2d-8de3-1a2c1cb298c0\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4" Jul 12 08:26:54.639936 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.639891 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/19bc5347-542e-4e2d-8de3-1a2c1cb298c0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4\" (UID: \"19bc5347-542e-4e2d-8de3-1a2c1cb298c0\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4" Jul 12 08:26:54.639936 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.639918 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/19bc5347-542e-4e2d-8de3-1a2c1cb298c0-tmp\") pod \"opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4\" (UID: \"19bc5347-542e-4e2d-8de3-1a2c1cb298c0\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4" Jul 12 08:26:54.640154 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.640008 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/19bc5347-542e-4e2d-8de3-1a2c1cb298c0-manifests\") pod \"opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4\" (UID: \"19bc5347-542e-4e2d-8de3-1a2c1cb298c0\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4" Jul 12 08:26:54.640154 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.640038 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/19bc5347-542e-4e2d-8de3-1a2c1cb298c0-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4\" (UID: \"19bc5347-542e-4e2d-8de3-1a2c1cb298c0\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4" Jul 12 08:26:54.650722 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.650691 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:26:54.740872 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.740838 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9942x\" (UniqueName: \"kubernetes.io/projected/19bc5347-542e-4e2d-8de3-1a2c1cb298c0-kube-api-access-9942x\") pod \"opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4\" (UID: \"19bc5347-542e-4e2d-8de3-1a2c1cb298c0\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4" Jul 12 08:26:54.741013 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.740880 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/19bc5347-542e-4e2d-8de3-1a2c1cb298c0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4\" (UID: \"19bc5347-542e-4e2d-8de3-1a2c1cb298c0\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4" Jul 12 08:26:54.741013 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.740903 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/19bc5347-542e-4e2d-8de3-1a2c1cb298c0-tmp\") pod \"opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4\" (UID: \"19bc5347-542e-4e2d-8de3-1a2c1cb298c0\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4" Jul 12 08:26:54.741013 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.740944 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/19bc5347-542e-4e2d-8de3-1a2c1cb298c0-manifests\") pod \"opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4\" (UID: \"19bc5347-542e-4e2d-8de3-1a2c1cb298c0\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4" Jul 12 08:26:54.741013 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.740962 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/19bc5347-542e-4e2d-8de3-1a2c1cb298c0-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4\" (UID: \"19bc5347-542e-4e2d-8de3-1a2c1cb298c0\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4" Jul 12 08:26:54.741404 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.741375 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/19bc5347-542e-4e2d-8de3-1a2c1cb298c0-manifests\") pod \"opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4\" (UID: \"19bc5347-542e-4e2d-8de3-1a2c1cb298c0\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4" Jul 12 08:26:54.743187 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.743165 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/19bc5347-542e-4e2d-8de3-1a2c1cb298c0-tmp\") pod \"opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4\" (UID: \"19bc5347-542e-4e2d-8de3-1a2c1cb298c0\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4" Jul 12 08:26:54.743333 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.743316 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/19bc5347-542e-4e2d-8de3-1a2c1cb298c0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4\" (UID: \"19bc5347-542e-4e2d-8de3-1a2c1cb298c0\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4" Jul 12 08:26:54.743391 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.743374 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/19bc5347-542e-4e2d-8de3-1a2c1cb298c0-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4\" (UID: \"19bc5347-542e-4e2d-8de3-1a2c1cb298c0\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4" Jul 12 08:26:54.745815 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.745783 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:26:54.749051 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.749030 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9942x\" (UniqueName: \"kubernetes.io/projected/19bc5347-542e-4e2d-8de3-1a2c1cb298c0-kube-api-access-9942x\") pod \"opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4\" (UID: \"19bc5347-542e-4e2d-8de3-1a2c1cb298c0\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4" Jul 12 08:26:54.829958 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.829936 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4" Jul 12 08:26:54.847091 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.847059 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:26:54.947233 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.947189 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:26:54.975481 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.975404 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4"] Jul 12 08:26:54.977853 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:26:54.977829 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19bc5347_542e_4e2d_8de3_1a2c1cb298c0.slice/crio-c3f959b6e6c2a74b98bcdc4f675194a94fcc9c93d8fead5c374be7c8b98865fc WatchSource:0}: Error finding container c3f959b6e6c2a74b98bcdc4f675194a94fcc9c93d8fead5c374be7c8b98865fc: Status 404 returned error can't find the container with id c3f959b6e6c2a74b98bcdc4f675194a94fcc9c93d8fead5c374be7c8b98865fc Jul 12 08:26:54.980204 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:54.980189 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 08:26:55.047462 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:55.047427 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:26:55.665903 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:55.665869 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4" event={"ID":"19bc5347-542e-4e2d-8de3-1a2c1cb298c0","Type":"ContainerStarted","Data":"c3f959b6e6c2a74b98bcdc4f675194a94fcc9c93d8fead5c374be7c8b98865fc"} Jul 12 08:26:58.680702 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:58.680665 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4" event={"ID":"19bc5347-542e-4e2d-8de3-1a2c1cb298c0","Type":"ContainerStarted","Data":"e0c5bd9f4c7ecc28417fa89741959dba5c5692d20d4160fd34c71ea566f4e4b4"} Jul 12 08:26:59.686530 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:59.686499 2566 generic.go:356] "Generic (PLEG): container finished" podID="19bc5347-542e-4e2d-8de3-1a2c1cb298c0" containerID="e0c5bd9f4c7ecc28417fa89741959dba5c5692d20d4160fd34c71ea566f4e4b4" exitCode=0 Jul 12 08:26:59.686912 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:26:59.686585 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4" event={"ID":"19bc5347-542e-4e2d-8de3-1a2c1cb298c0","Type":"ContainerDied","Data":"e0c5bd9f4c7ecc28417fa89741959dba5c5692d20d4160fd34c71ea566f4e4b4"} Jul 12 08:27:00.532653 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:00.532610 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:27:00.631791 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:00.631759 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:27:00.697768 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:00.697731 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4" event={"ID":"19bc5347-542e-4e2d-8de3-1a2c1cb298c0","Type":"ContainerStarted","Data":"425977da9a75c18a6f9370f0f848e3f197c7c4c0bd34bab3e23285e9d8fd7960"} Jul 12 08:27:00.698121 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:00.697861 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4" Jul 12 08:27:00.722654 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:00.722611 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4" podStartSLOduration=2.448542656 podStartE2EDuration="6.722598063s" totalImagesPullingTime="3.179230649s" totalInitContainerRuntime="1.094824758s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:26:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:27:00.720462296 +0000 UTC m=+344.381257974" watchObservedRunningTime="2026-07-12 08:27:00.722598063 +0000 UTC m=+344.383393727" Jul 12 08:27:02.831822 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:02.831786 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:27:02.895121 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:02.895081 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:27:04.596892 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:04.596838 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:27:04.664986 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:04.664948 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:27:11.703835 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:11.703747 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4" Jul 12 08:27:13.338507 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:13.338473 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:27:13.507704 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:13.507668 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:27:13.747242 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:13.747204 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:27:14.338536 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:14.338500 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:27:14.439515 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:14.439476 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:27:14.548226 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:14.548190 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:27:14.663129 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:14.663041 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:27:15.238541 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:15.238499 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:27:15.325497 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:15.325436 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:27:16.784006 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:16.783973 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:27:19.582681 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:19.582646 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:27:19.678757 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:19.678722 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:27:20.270765 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:20.270730 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:27:20.385086 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:20.385051 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:27:20.689169 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:20.689089 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:27:21.112687 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:21.112650 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7cb89b9bb7-b7r6r"] Jul 12 08:27:21.119107 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:21.119081 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7cb89b9bb7-b7r6r" Jul 12 08:27:21.125699 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:21.125675 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 12 08:27:21.125832 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:21.125746 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 12 08:27:21.126593 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:21.126576 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 12 08:27:21.126693 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:21.126598 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 12 08:27:21.126914 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:21.126881 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-xrctc\"" Jul 12 08:27:21.127029 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:21.127015 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 12 08:27:21.136458 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:21.136438 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mw5bn\" (UniqueName: \"kubernetes.io/projected/7a86bf2f-527f-4468-a37b-3cd9bcd91c6e-kube-api-access-mw5bn\") pod \"lws-controller-manager-7cb89b9bb7-b7r6r\" (UID: \"7a86bf2f-527f-4468-a37b-3cd9bcd91c6e\") " pod="openshift-lws-operator/lws-controller-manager-7cb89b9bb7-b7r6r" Jul 12 08:27:21.136574 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:21.136477 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7a86bf2f-527f-4468-a37b-3cd9bcd91c6e-metrics-cert\") pod \"lws-controller-manager-7cb89b9bb7-b7r6r\" (UID: \"7a86bf2f-527f-4468-a37b-3cd9bcd91c6e\") " pod="openshift-lws-operator/lws-controller-manager-7cb89b9bb7-b7r6r" Jul 12 08:27:21.136574 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:21.136543 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7a86bf2f-527f-4468-a37b-3cd9bcd91c6e-manager-config\") pod \"lws-controller-manager-7cb89b9bb7-b7r6r\" (UID: \"7a86bf2f-527f-4468-a37b-3cd9bcd91c6e\") " pod="openshift-lws-operator/lws-controller-manager-7cb89b9bb7-b7r6r" Jul 12 08:27:21.136710 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:21.136585 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a86bf2f-527f-4468-a37b-3cd9bcd91c6e-cert\") pod \"lws-controller-manager-7cb89b9bb7-b7r6r\" (UID: \"7a86bf2f-527f-4468-a37b-3cd9bcd91c6e\") " pod="openshift-lws-operator/lws-controller-manager-7cb89b9bb7-b7r6r" Jul 12 08:27:21.144959 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:21.144942 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7cb89b9bb7-b7r6r"] Jul 12 08:27:21.237854 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:21.237818 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw5bn\" (UniqueName: \"kubernetes.io/projected/7a86bf2f-527f-4468-a37b-3cd9bcd91c6e-kube-api-access-mw5bn\") pod \"lws-controller-manager-7cb89b9bb7-b7r6r\" (UID: \"7a86bf2f-527f-4468-a37b-3cd9bcd91c6e\") " pod="openshift-lws-operator/lws-controller-manager-7cb89b9bb7-b7r6r" Jul 12 08:27:21.237854 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:21.237857 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7a86bf2f-527f-4468-a37b-3cd9bcd91c6e-metrics-cert\") pod \"lws-controller-manager-7cb89b9bb7-b7r6r\" (UID: \"7a86bf2f-527f-4468-a37b-3cd9bcd91c6e\") " pod="openshift-lws-operator/lws-controller-manager-7cb89b9bb7-b7r6r" Jul 12 08:27:21.238097 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:21.237882 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7a86bf2f-527f-4468-a37b-3cd9bcd91c6e-manager-config\") pod \"lws-controller-manager-7cb89b9bb7-b7r6r\" (UID: \"7a86bf2f-527f-4468-a37b-3cd9bcd91c6e\") " pod="openshift-lws-operator/lws-controller-manager-7cb89b9bb7-b7r6r" Jul 12 08:27:21.238097 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:21.237903 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a86bf2f-527f-4468-a37b-3cd9bcd91c6e-cert\") pod \"lws-controller-manager-7cb89b9bb7-b7r6r\" (UID: \"7a86bf2f-527f-4468-a37b-3cd9bcd91c6e\") " pod="openshift-lws-operator/lws-controller-manager-7cb89b9bb7-b7r6r" Jul 12 08:27:21.238620 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:21.238597 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7a86bf2f-527f-4468-a37b-3cd9bcd91c6e-manager-config\") pod \"lws-controller-manager-7cb89b9bb7-b7r6r\" (UID: \"7a86bf2f-527f-4468-a37b-3cd9bcd91c6e\") " pod="openshift-lws-operator/lws-controller-manager-7cb89b9bb7-b7r6r" Jul 12 08:27:21.240794 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:21.240765 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a86bf2f-527f-4468-a37b-3cd9bcd91c6e-cert\") pod \"lws-controller-manager-7cb89b9bb7-b7r6r\" (UID: \"7a86bf2f-527f-4468-a37b-3cd9bcd91c6e\") " pod="openshift-lws-operator/lws-controller-manager-7cb89b9bb7-b7r6r" Jul 12 08:27:21.240794 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:21.240793 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7a86bf2f-527f-4468-a37b-3cd9bcd91c6e-metrics-cert\") pod \"lws-controller-manager-7cb89b9bb7-b7r6r\" (UID: \"7a86bf2f-527f-4468-a37b-3cd9bcd91c6e\") " pod="openshift-lws-operator/lws-controller-manager-7cb89b9bb7-b7r6r" Jul 12 08:27:21.249733 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:21.249712 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw5bn\" (UniqueName: \"kubernetes.io/projected/7a86bf2f-527f-4468-a37b-3cd9bcd91c6e-kube-api-access-mw5bn\") pod \"lws-controller-manager-7cb89b9bb7-b7r6r\" (UID: \"7a86bf2f-527f-4468-a37b-3cd9bcd91c6e\") " pod="openshift-lws-operator/lws-controller-manager-7cb89b9bb7-b7r6r" Jul 12 08:27:21.427840 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:21.427767 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7cb89b9bb7-b7r6r" Jul 12 08:27:21.552244 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:21.552218 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7cb89b9bb7-b7r6r"] Jul 12 08:27:21.554818 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:27:21.554793 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a86bf2f_527f_4468_a37b_3cd9bcd91c6e.slice/crio-bd1b4f22c24c2788f7f09e3de10af618b94b2f907a080d34495f36a6655565b7 WatchSource:0}: Error finding container bd1b4f22c24c2788f7f09e3de10af618b94b2f907a080d34495f36a6655565b7: Status 404 returned error can't find the container with id bd1b4f22c24c2788f7f09e3de10af618b94b2f907a080d34495f36a6655565b7 Jul 12 08:27:21.788836 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:21.788802 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7cb89b9bb7-b7r6r" event={"ID":"7a86bf2f-527f-4468-a37b-3cd9bcd91c6e","Type":"ContainerStarted","Data":"bd1b4f22c24c2788f7f09e3de10af618b94b2f907a080d34495f36a6655565b7"} Jul 12 08:27:24.806522 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:24.806486 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7cb89b9bb7-b7r6r" event={"ID":"7a86bf2f-527f-4468-a37b-3cd9bcd91c6e","Type":"ContainerStarted","Data":"86f159eefc85c8d3b2f7c70d442fe3832455dba05051c5b2199e91449be0973e"} Jul 12 08:27:24.806990 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:24.806620 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7cb89b9bb7-b7r6r" Jul 12 08:27:24.829803 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:24.829760 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7cb89b9bb7-b7r6r" podStartSLOduration=1.470388571 podStartE2EDuration="3.829747107s" totalImagesPullingTime="2.359358536s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:27:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:27:24.829263419 +0000 UTC m=+368.490059082" watchObservedRunningTime="2026-07-12 08:27:24.829747107 +0000 UTC m=+368.490542769" Jul 12 08:27:25.386468 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:25.386429 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:27:25.503211 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:25.503179 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:27:25.635294 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:25.635255 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:27:25.735291 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:25.735254 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:27:26.685010 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:26.684966 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:27:26.801220 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:26.801182 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:27:35.812695 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:35.812667 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7cb89b9bb7-b7r6r" Jul 12 08:27:42.373213 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:27:42.373172 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:28:14.999231 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:14.999196 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:28:15.098024 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:15.097981 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:28:15.160602 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:15.160570 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2zrwd"] Jul 12 08:28:15.162726 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:15.162709 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2zrwd" Jul 12 08:28:15.165158 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:15.165135 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 12 08:28:15.165871 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:15.165844 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 12 08:28:15.165871 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:15.165865 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-xhpsj\"" Jul 12 08:28:15.173045 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:15.173018 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2zrwd"] Jul 12 08:28:15.210267 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:15.210240 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:28:15.266417 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:15.266360 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8r27\" (UniqueName: \"kubernetes.io/projected/a2d0ba70-ac08-412e-b654-e786d4320d2d-kube-api-access-v8r27\") pod \"kuadrant-operator-catalog-2zrwd\" (UID: \"a2d0ba70-ac08-412e-b654-e786d4320d2d\") " pod="kuadrant-system/kuadrant-operator-catalog-2zrwd" Jul 12 08:28:15.367708 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:15.367678 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8r27\" (UniqueName: \"kubernetes.io/projected/a2d0ba70-ac08-412e-b654-e786d4320d2d-kube-api-access-v8r27\") pod \"kuadrant-operator-catalog-2zrwd\" (UID: \"a2d0ba70-ac08-412e-b654-e786d4320d2d\") " pod="kuadrant-system/kuadrant-operator-catalog-2zrwd" Jul 12 08:28:15.375409 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:15.375387 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8r27\" (UniqueName: \"kubernetes.io/projected/a2d0ba70-ac08-412e-b654-e786d4320d2d-kube-api-access-v8r27\") pod \"kuadrant-operator-catalog-2zrwd\" (UID: \"a2d0ba70-ac08-412e-b654-e786d4320d2d\") " pod="kuadrant-system/kuadrant-operator-catalog-2zrwd" Jul 12 08:28:15.475883 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:15.475863 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2zrwd" Jul 12 08:28:15.532314 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:15.532279 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2zrwd"] Jul 12 08:28:15.601979 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:15.601951 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2zrwd"] Jul 12 08:28:15.604476 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:28:15.604451 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2d0ba70_ac08_412e_b654_e786d4320d2d.slice/crio-b9214f7515db65d224e57240cba175dd3f4691185b0b4e59519c91febc7a12af WatchSource:0}: Error finding container b9214f7515db65d224e57240cba175dd3f4691185b0b4e59519c91febc7a12af: Status 404 returned error can't find the container with id b9214f7515db65d224e57240cba175dd3f4691185b0b4e59519c91febc7a12af Jul 12 08:28:15.742943 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:15.742912 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4nvqv"] Jul 12 08:28:15.745592 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:15.745574 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4nvqv" Jul 12 08:28:15.753731 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:15.753709 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4nvqv"] Jul 12 08:28:15.771756 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:15.771733 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwfcq\" (UniqueName: \"kubernetes.io/projected/3898a1d5-9053-4ddf-9a3f-b1c27dafc611-kube-api-access-hwfcq\") pod \"kuadrant-operator-catalog-4nvqv\" (UID: \"3898a1d5-9053-4ddf-9a3f-b1c27dafc611\") " pod="kuadrant-system/kuadrant-operator-catalog-4nvqv" Jul 12 08:28:15.872241 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:15.872187 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwfcq\" (UniqueName: \"kubernetes.io/projected/3898a1d5-9053-4ddf-9a3f-b1c27dafc611-kube-api-access-hwfcq\") pod \"kuadrant-operator-catalog-4nvqv\" (UID: \"3898a1d5-9053-4ddf-9a3f-b1c27dafc611\") " pod="kuadrant-system/kuadrant-operator-catalog-4nvqv" Jul 12 08:28:15.880065 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:15.880039 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwfcq\" (UniqueName: \"kubernetes.io/projected/3898a1d5-9053-4ddf-9a3f-b1c27dafc611-kube-api-access-hwfcq\") pod \"kuadrant-operator-catalog-4nvqv\" (UID: \"3898a1d5-9053-4ddf-9a3f-b1c27dafc611\") " pod="kuadrant-system/kuadrant-operator-catalog-4nvqv" Jul 12 08:28:16.018114 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:16.018087 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2zrwd" event={"ID":"a2d0ba70-ac08-412e-b654-e786d4320d2d","Type":"ContainerStarted","Data":"b9214f7515db65d224e57240cba175dd3f4691185b0b4e59519c91febc7a12af"} Jul 12 08:28:16.055647 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:16.055623 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4nvqv" Jul 12 08:28:16.180072 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:16.180049 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4nvqv"] Jul 12 08:28:16.182426 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:28:16.182404 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3898a1d5_9053_4ddf_9a3f_b1c27dafc611.slice/crio-de22b7d8a882aa0623ede5b7236e443a5d08b03bb4a49edb5a31954ea06da8b1 WatchSource:0}: Error finding container de22b7d8a882aa0623ede5b7236e443a5d08b03bb4a49edb5a31954ea06da8b1: Status 404 returned error can't find the container with id de22b7d8a882aa0623ede5b7236e443a5d08b03bb4a49edb5a31954ea06da8b1 Jul 12 08:28:17.032974 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:17.032942 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4nvqv" event={"ID":"3898a1d5-9053-4ddf-9a3f-b1c27dafc611","Type":"ContainerStarted","Data":"de22b7d8a882aa0623ede5b7236e443a5d08b03bb4a49edb5a31954ea06da8b1"} Jul 12 08:28:19.036695 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:19.036660 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4nvqv" event={"ID":"3898a1d5-9053-4ddf-9a3f-b1c27dafc611","Type":"ContainerStarted","Data":"ef7e2fe846d0b564adf2874e3db54bb4c921293cc74ea023b6f58b759843f0d0"} Jul 12 08:28:19.037958 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:19.037939 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2zrwd" event={"ID":"a2d0ba70-ac08-412e-b654-e786d4320d2d","Type":"ContainerStarted","Data":"429091932a65f6647ccfade0edbd90b651c47d4c481024f7b6437a7598e75110"} Jul 12 08:28:19.038038 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:19.037993 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-2zrwd" podUID="a2d0ba70-ac08-412e-b654-e786d4320d2d" containerName="registry-server" containerID="cri-o://429091932a65f6647ccfade0edbd90b651c47d4c481024f7b6437a7598e75110" gracePeriod=2 Jul 12 08:28:19.053232 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:19.053193 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-4nvqv" podStartSLOduration=1.851571826 podStartE2EDuration="4.053180222s" totalImagesPullingTime="2.201608396s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:28:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:28:19.052403391 +0000 UTC m=+422.713199077" watchObservedRunningTime="2026-07-12 08:28:19.053180222 +0000 UTC m=+422.713975861" Jul 12 08:28:19.072508 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:19.072468 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-2zrwd" podStartSLOduration=1.375455508 podStartE2EDuration="4.072455051s" totalImagesPullingTime="2.696999543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:28:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:28:19.071306292 +0000 UTC m=+422.732101973" watchObservedRunningTime="2026-07-12 08:28:19.072455051 +0000 UTC m=+422.733250711" Jul 12 08:28:19.278737 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:19.278715 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2zrwd" Jul 12 08:28:19.301789 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:19.301737 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8r27\" (UniqueName: \"kubernetes.io/projected/a2d0ba70-ac08-412e-b654-e786d4320d2d-kube-api-access-v8r27\") pod \"a2d0ba70-ac08-412e-b654-e786d4320d2d\" (UID: \"a2d0ba70-ac08-412e-b654-e786d4320d2d\") " Jul 12 08:28:19.303757 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:19.303732 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2d0ba70-ac08-412e-b654-e786d4320d2d-kube-api-access-v8r27" (OuterVolumeSpecName: "kube-api-access-v8r27") pod "a2d0ba70-ac08-412e-b654-e786d4320d2d" (UID: "a2d0ba70-ac08-412e-b654-e786d4320d2d"). InnerVolumeSpecName "kube-api-access-v8r27". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:28:19.402949 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:19.402920 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8r27\" (UniqueName: \"kubernetes.io/projected/a2d0ba70-ac08-412e-b654-e786d4320d2d-kube-api-access-v8r27\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:28:20.043857 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:20.043825 2566 generic.go:356] "Generic (PLEG): container finished" podID="a2d0ba70-ac08-412e-b654-e786d4320d2d" containerID="429091932a65f6647ccfade0edbd90b651c47d4c481024f7b6437a7598e75110" exitCode=0 Jul 12 08:28:20.044320 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:20.043869 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2zrwd" Jul 12 08:28:20.044320 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:20.043873 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2zrwd" event={"ID":"a2d0ba70-ac08-412e-b654-e786d4320d2d","Type":"ContainerDied","Data":"429091932a65f6647ccfade0edbd90b651c47d4c481024f7b6437a7598e75110"} Jul 12 08:28:20.044320 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:20.043979 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2zrwd" event={"ID":"a2d0ba70-ac08-412e-b654-e786d4320d2d","Type":"ContainerDied","Data":"b9214f7515db65d224e57240cba175dd3f4691185b0b4e59519c91febc7a12af"} Jul 12 08:28:20.044320 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:20.044004 2566 scope.go:117] "RemoveContainer" containerID="429091932a65f6647ccfade0edbd90b651c47d4c481024f7b6437a7598e75110" Jul 12 08:28:20.052807 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:20.052792 2566 scope.go:117] "RemoveContainer" containerID="429091932a65f6647ccfade0edbd90b651c47d4c481024f7b6437a7598e75110" Jul 12 08:28:20.053056 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:28:20.053040 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"429091932a65f6647ccfade0edbd90b651c47d4c481024f7b6437a7598e75110\": container with ID starting with 429091932a65f6647ccfade0edbd90b651c47d4c481024f7b6437a7598e75110 not found: ID does not exist" containerID="429091932a65f6647ccfade0edbd90b651c47d4c481024f7b6437a7598e75110" Jul 12 08:28:20.053110 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:20.053067 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"429091932a65f6647ccfade0edbd90b651c47d4c481024f7b6437a7598e75110"} err="failed to get container status \"429091932a65f6647ccfade0edbd90b651c47d4c481024f7b6437a7598e75110\": rpc error: code = NotFound desc = could not find container \"429091932a65f6647ccfade0edbd90b651c47d4c481024f7b6437a7598e75110\": container with ID starting with 429091932a65f6647ccfade0edbd90b651c47d4c481024f7b6437a7598e75110 not found: ID does not exist" Jul 12 08:28:20.063830 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:20.063809 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2zrwd"] Jul 12 08:28:20.067594 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:20.067575 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2zrwd"] Jul 12 08:28:21.025962 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:21.025930 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2d0ba70-ac08-412e-b654-e786d4320d2d" path="/var/lib/kubelet/pods/a2d0ba70-ac08-412e-b654-e786d4320d2d/volumes" Jul 12 08:28:26.056539 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:26.056493 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-4nvqv" Jul 12 08:28:26.056985 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:26.056585 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-4nvqv" Jul 12 08:28:26.079213 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:26.079190 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-4nvqv" Jul 12 08:28:26.100340 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:26.100314 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-4nvqv" Jul 12 08:28:38.035494 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:38.035454 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:28:38.650825 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:38.650788 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:28:39.894678 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:39.894579 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:28:41.913942 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:41.913909 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-t49ds"] Jul 12 08:28:41.914313 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:41.914216 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2d0ba70-ac08-412e-b654-e786d4320d2d" containerName="registry-server" Jul 12 08:28:41.914313 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:41.914227 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2d0ba70-ac08-412e-b654-e786d4320d2d" containerName="registry-server" Jul 12 08:28:41.914313 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:41.914283 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2d0ba70-ac08-412e-b654-e786d4320d2d" containerName="registry-server" Jul 12 08:28:41.917073 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:41.917056 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-t49ds" Jul 12 08:28:41.919355 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:41.919339 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-kcv5m\"" Jul 12 08:28:41.930694 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:41.930672 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-t49ds"] Jul 12 08:28:41.971533 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:41.971506 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsk7p\" (UniqueName: \"kubernetes.io/projected/1394821b-2a26-4cf2-9f53-33432f999f4c-kube-api-access-rsk7p\") pod \"authorino-operator-75cc8d5cb7-t49ds\" (UID: \"1394821b-2a26-4cf2-9f53-33432f999f4c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-t49ds" Jul 12 08:28:42.072329 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:42.072283 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsk7p\" (UniqueName: \"kubernetes.io/projected/1394821b-2a26-4cf2-9f53-33432f999f4c-kube-api-access-rsk7p\") pod \"authorino-operator-75cc8d5cb7-t49ds\" (UID: \"1394821b-2a26-4cf2-9f53-33432f999f4c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-t49ds" Jul 12 08:28:42.085751 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:42.085730 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsk7p\" (UniqueName: \"kubernetes.io/projected/1394821b-2a26-4cf2-9f53-33432f999f4c-kube-api-access-rsk7p\") pod \"authorino-operator-75cc8d5cb7-t49ds\" (UID: \"1394821b-2a26-4cf2-9f53-33432f999f4c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-t49ds" Jul 12 08:28:42.227161 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:42.227088 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-t49ds" Jul 12 08:28:42.358297 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:42.358277 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-t49ds"] Jul 12 08:28:42.360713 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:28:42.360680 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1394821b_2a26_4cf2_9f53_33432f999f4c.slice/crio-f33257f334208b1155c7781c04cb32dd6cdbdce8f60a9e884bd6fbece3beb935 WatchSource:0}: Error finding container f33257f334208b1155c7781c04cb32dd6cdbdce8f60a9e884bd6fbece3beb935: Status 404 returned error can't find the container with id f33257f334208b1155c7781c04cb32dd6cdbdce8f60a9e884bd6fbece3beb935 Jul 12 08:28:42.652762 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:42.652710 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:28:42.778150 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:42.778120 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:28:43.144270 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:43.144234 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-t49ds" event={"ID":"1394821b-2a26-4cf2-9f53-33432f999f4c","Type":"ContainerStarted","Data":"f33257f334208b1155c7781c04cb32dd6cdbdce8f60a9e884bd6fbece3beb935"} Jul 12 08:28:45.161880 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:45.161840 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-t49ds" event={"ID":"1394821b-2a26-4cf2-9f53-33432f999f4c","Type":"ContainerStarted","Data":"3a784c64408415a1b8c7af159ee11315d1e8867937e582f97dbc5ad016f658e1"} Jul 12 08:28:45.162249 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:45.161958 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-t49ds" Jul 12 08:28:45.191815 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:45.191769 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-t49ds" podStartSLOduration=2.0401694949999998 podStartE2EDuration="4.191754356s" totalImagesPullingTime="2.151584861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:28:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:28:45.18959866 +0000 UTC m=+448.850394342" watchObservedRunningTime="2026-07-12 08:28:45.191754356 +0000 UTC m=+448.852550015" Jul 12 08:28:46.267687 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:46.267647 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:28:47.631680 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:47.631641 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-4k8lq"] Jul 12 08:28:47.634953 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:47.634935 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4k8lq" Jul 12 08:28:47.639477 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:47.639454 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 12 08:28:47.639584 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:47.639504 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-vq2lt\"" Jul 12 08:28:47.646325 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:47.646283 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-4k8lq"] Jul 12 08:28:47.717965 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:47.717931 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dcpp\" (UniqueName: \"kubernetes.io/projected/c6d21d48-2a6e-4ecf-bcf4-3fc979d11e2a-kube-api-access-2dcpp\") pod \"dns-operator-controller-manager-864dd78c4c-4k8lq\" (UID: \"c6d21d48-2a6e-4ecf-bcf4-3fc979d11e2a\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4k8lq" Jul 12 08:28:47.819324 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:47.819258 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dcpp\" (UniqueName: \"kubernetes.io/projected/c6d21d48-2a6e-4ecf-bcf4-3fc979d11e2a-kube-api-access-2dcpp\") pod \"dns-operator-controller-manager-864dd78c4c-4k8lq\" (UID: \"c6d21d48-2a6e-4ecf-bcf4-3fc979d11e2a\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4k8lq" Jul 12 08:28:47.830593 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:47.830562 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dcpp\" (UniqueName: \"kubernetes.io/projected/c6d21d48-2a6e-4ecf-bcf4-3fc979d11e2a-kube-api-access-2dcpp\") pod \"dns-operator-controller-manager-864dd78c4c-4k8lq\" (UID: \"c6d21d48-2a6e-4ecf-bcf4-3fc979d11e2a\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4k8lq" Jul 12 08:28:47.944223 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:47.944128 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4k8lq" Jul 12 08:28:48.097950 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:48.097927 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-4k8lq"] Jul 12 08:28:48.100593 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:28:48.100538 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6d21d48_2a6e_4ecf_bcf4_3fc979d11e2a.slice/crio-a72e4661fa7d613b611b55229126a40c0b64e4681a668137eff60353994825eb WatchSource:0}: Error finding container a72e4661fa7d613b611b55229126a40c0b64e4681a668137eff60353994825eb: Status 404 returned error can't find the container with id a72e4661fa7d613b611b55229126a40c0b64e4681a668137eff60353994825eb Jul 12 08:28:48.175597 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:48.175533 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4k8lq" event={"ID":"c6d21d48-2a6e-4ecf-bcf4-3fc979d11e2a","Type":"ContainerStarted","Data":"a72e4661fa7d613b611b55229126a40c0b64e4681a668137eff60353994825eb"} Jul 12 08:28:48.266820 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:48.266784 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:28:48.544410 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:48.544306 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:28:50.884708 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:50.884670 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:28:51.191394 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:51.191300 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4k8lq" event={"ID":"c6d21d48-2a6e-4ecf-bcf4-3fc979d11e2a","Type":"ContainerStarted","Data":"58d1899dffb9dbc75c4b2a83cc81f7b4e74d002727ddd4dd79f8372434e9406b"} Jul 12 08:28:51.191562 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:51.191480 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4k8lq" Jul 12 08:28:51.210254 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:51.210202 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4k8lq" podStartSLOduration=2.130026849 podStartE2EDuration="4.210186134s" totalImagesPullingTime="2.080159285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:28:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:28:51.209807362 +0000 UTC m=+454.870603027" watchObservedRunningTime="2026-07-12 08:28:51.210186134 +0000 UTC m=+454.870981795" Jul 12 08:28:52.050752 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:52.050717 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:28:52.122177 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:52.122144 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:28:52.506386 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:52.506350 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:28:52.608527 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:52.608486 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:28:54.319603 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:54.319567 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:28:54.415474 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:54.415444 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:28:56.168467 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:56.168440 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-t49ds" Jul 12 08:28:56.894658 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:56.894625 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:28:56.989703 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:28:56.989667 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:02.198070 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:02.198037 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4k8lq" Jul 12 08:29:04.586678 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:04.586647 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dhjh7"] Jul 12 08:29:04.592836 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:04.592818 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dhjh7" Jul 12 08:29:04.595607 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:04.595586 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-72fmd\"" Jul 12 08:29:04.605727 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:04.605695 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dhjh7"] Jul 12 08:29:04.697237 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:04.697201 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dhjh7"] Jul 12 08:29:04.697454 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:29:04.697435 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-djcrk], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dhjh7" podUID="f536ff43-5cc6-4fc2-8c63-f3f460420c08" Jul 12 08:29:04.752265 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:04.752225 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:04.761797 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:04.761769 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f536ff43-5cc6-4fc2-8c63-f3f460420c08-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-dhjh7\" (UID: \"f536ff43-5cc6-4fc2-8c63-f3f460420c08\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dhjh7" Jul 12 08:29:04.761917 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:04.761802 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djcrk\" (UniqueName: \"kubernetes.io/projected/f536ff43-5cc6-4fc2-8c63-f3f460420c08-kube-api-access-djcrk\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-dhjh7\" (UID: \"f536ff43-5cc6-4fc2-8c63-f3f460420c08\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dhjh7" Jul 12 08:29:04.853805 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:04.853721 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:04.862819 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:04.862796 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f536ff43-5cc6-4fc2-8c63-f3f460420c08-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-dhjh7\" (UID: \"f536ff43-5cc6-4fc2-8c63-f3f460420c08\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dhjh7" Jul 12 08:29:04.862891 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:04.862827 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djcrk\" (UniqueName: \"kubernetes.io/projected/f536ff43-5cc6-4fc2-8c63-f3f460420c08-kube-api-access-djcrk\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-dhjh7\" (UID: \"f536ff43-5cc6-4fc2-8c63-f3f460420c08\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dhjh7" Jul 12 08:29:04.863155 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:04.863138 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f536ff43-5cc6-4fc2-8c63-f3f460420c08-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-dhjh7\" (UID: \"f536ff43-5cc6-4fc2-8c63-f3f460420c08\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dhjh7" Jul 12 08:29:04.871895 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:04.871873 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djcrk\" (UniqueName: \"kubernetes.io/projected/f536ff43-5cc6-4fc2-8c63-f3f460420c08-kube-api-access-djcrk\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-dhjh7\" (UID: \"f536ff43-5cc6-4fc2-8c63-f3f460420c08\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dhjh7" Jul 12 08:29:05.195765 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:05.195677 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dhjh7"] Jul 12 08:29:05.210564 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:05.210523 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dhjh7"] Jul 12 08:29:05.254468 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:05.254437 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dhjh7" Jul 12 08:29:05.256974 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:29:05.256945 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-dhjh7\" is forbidden: User \"system:node:ip-10-0-136-138.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-138.ec2.internal' and this object" podUID="f536ff43-5cc6-4fc2-8c63-f3f460420c08" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dhjh7" Jul 12 08:29:05.262047 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:05.262026 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dhjh7" Jul 12 08:29:05.265452 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:29:05.265427 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-dhjh7\" is forbidden: User \"system:node:ip-10-0-136-138.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-138.ec2.internal' and this object" podUID="f536ff43-5cc6-4fc2-8c63-f3f460420c08" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dhjh7" Jul 12 08:29:05.270750 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:05.270726 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bk5gw"] Jul 12 08:29:05.273864 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:05.273849 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bk5gw" Jul 12 08:29:05.276364 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:29:05.276337 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-dhjh7\" is forbidden: User \"system:node:ip-10-0-136-138.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-138.ec2.internal' and this object" podUID="f536ff43-5cc6-4fc2-8c63-f3f460420c08" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dhjh7" Jul 12 08:29:05.292394 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:05.292372 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bk5gw"] Jul 12 08:29:05.366134 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:05.366104 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f536ff43-5cc6-4fc2-8c63-f3f460420c08-extensions-socket-volume\") pod \"f536ff43-5cc6-4fc2-8c63-f3f460420c08\" (UID: \"f536ff43-5cc6-4fc2-8c63-f3f460420c08\") " Jul 12 08:29:05.366301 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:05.366195 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djcrk\" (UniqueName: \"kubernetes.io/projected/f536ff43-5cc6-4fc2-8c63-f3f460420c08-kube-api-access-djcrk\") pod \"f536ff43-5cc6-4fc2-8c63-f3f460420c08\" (UID: \"f536ff43-5cc6-4fc2-8c63-f3f460420c08\") " Jul 12 08:29:05.366419 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:05.366389 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f536ff43-5cc6-4fc2-8c63-f3f460420c08-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f536ff43-5cc6-4fc2-8c63-f3f460420c08" (UID: "f536ff43-5cc6-4fc2-8c63-f3f460420c08"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:29:05.368224 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:05.368204 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f536ff43-5cc6-4fc2-8c63-f3f460420c08-kube-api-access-djcrk" (OuterVolumeSpecName: "kube-api-access-djcrk") pod "f536ff43-5cc6-4fc2-8c63-f3f460420c08" (UID: "f536ff43-5cc6-4fc2-8c63-f3f460420c08"). InnerVolumeSpecName "kube-api-access-djcrk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:29:05.467162 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:05.467089 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/76bdfade-53b6-4218-94db-79cf6a0d5855-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bk5gw\" (UID: \"76bdfade-53b6-4218-94db-79cf6a0d5855\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bk5gw" Jul 12 08:29:05.467162 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:05.467120 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gvdw\" (UniqueName: \"kubernetes.io/projected/76bdfade-53b6-4218-94db-79cf6a0d5855-kube-api-access-6gvdw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bk5gw\" (UID: \"76bdfade-53b6-4218-94db-79cf6a0d5855\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bk5gw" Jul 12 08:29:05.467311 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:05.467193 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djcrk\" (UniqueName: \"kubernetes.io/projected/f536ff43-5cc6-4fc2-8c63-f3f460420c08-kube-api-access-djcrk\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:29:05.467311 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:05.467203 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f536ff43-5cc6-4fc2-8c63-f3f460420c08-extensions-socket-volume\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:29:05.568437 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:05.568372 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/76bdfade-53b6-4218-94db-79cf6a0d5855-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bk5gw\" (UID: \"76bdfade-53b6-4218-94db-79cf6a0d5855\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bk5gw" Jul 12 08:29:05.568437 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:05.568446 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gvdw\" (UniqueName: \"kubernetes.io/projected/76bdfade-53b6-4218-94db-79cf6a0d5855-kube-api-access-6gvdw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bk5gw\" (UID: \"76bdfade-53b6-4218-94db-79cf6a0d5855\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bk5gw" Jul 12 08:29:05.568854 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:05.568832 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/76bdfade-53b6-4218-94db-79cf6a0d5855-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bk5gw\" (UID: \"76bdfade-53b6-4218-94db-79cf6a0d5855\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bk5gw" Jul 12 08:29:05.588602 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:05.588581 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gvdw\" (UniqueName: \"kubernetes.io/projected/76bdfade-53b6-4218-94db-79cf6a0d5855-kube-api-access-6gvdw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bk5gw\" (UID: \"76bdfade-53b6-4218-94db-79cf6a0d5855\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bk5gw" Jul 12 08:29:05.883407 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:05.883372 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bk5gw" Jul 12 08:29:06.014633 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:06.014611 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bk5gw"] Jul 12 08:29:06.016858 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:29:06.016828 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76bdfade_53b6_4218_94db_79cf6a0d5855.slice/crio-07a3488106b11ab39829184663c19a231db0172582d4d41fcc8c79980818bd9d WatchSource:0}: Error finding container 07a3488106b11ab39829184663c19a231db0172582d4d41fcc8c79980818bd9d: Status 404 returned error can't find the container with id 07a3488106b11ab39829184663c19a231db0172582d4d41fcc8c79980818bd9d Jul 12 08:29:06.260618 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:06.260588 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dhjh7" Jul 12 08:29:06.260618 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:06.260606 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bk5gw" event={"ID":"76bdfade-53b6-4218-94db-79cf6a0d5855","Type":"ContainerStarted","Data":"07a3488106b11ab39829184663c19a231db0172582d4d41fcc8c79980818bd9d"} Jul 12 08:29:06.263050 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:29:06.263024 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-dhjh7\" is forbidden: User \"system:node:ip-10-0-136-138.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-138.ec2.internal' and this object" podUID="f536ff43-5cc6-4fc2-8c63-f3f460420c08" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dhjh7" Jul 12 08:29:06.268684 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:29:06.268660 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-dhjh7\" is forbidden: User \"system:node:ip-10-0-136-138.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-138.ec2.internal' and this object" podUID="f536ff43-5cc6-4fc2-8c63-f3f460420c08" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dhjh7" Jul 12 08:29:07.024110 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:29:07.024067 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-dhjh7\" is forbidden: User \"system:node:ip-10-0-136-138.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-138.ec2.internal' and this object" podUID="f536ff43-5cc6-4fc2-8c63-f3f460420c08" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dhjh7" Jul 12 08:29:07.026766 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:07.026738 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f536ff43-5cc6-4fc2-8c63-f3f460420c08" path="/var/lib/kubelet/pods/f536ff43-5cc6-4fc2-8c63-f3f460420c08/volumes" Jul 12 08:29:09.469935 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:09.469883 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:09.575730 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:09.575691 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:10.284212 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:10.284183 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bk5gw" event={"ID":"76bdfade-53b6-4218-94db-79cf6a0d5855","Type":"ContainerStarted","Data":"b742303cb56ed4e58a9a6caa1f816482c89731c46ddaf794e3998ff8005bfd79"} Jul 12 08:29:10.284362 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:10.284318 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bk5gw" Jul 12 08:29:10.304210 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:10.304167 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bk5gw" podStartSLOduration=1.123111822 podStartE2EDuration="5.304153474s" totalImagesPullingTime="4.181041652s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:29:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:29:10.302947884 +0000 UTC m=+473.963743545" watchObservedRunningTime="2026-07-12 08:29:10.304153474 +0000 UTC m=+473.964949136" Jul 12 08:29:12.574890 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:12.574858 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:12.674834 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:12.674808 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:13.263719 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:13.263674 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:13.320870 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:13.320836 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:13.459502 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:13.459462 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:13.520250 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:13.520180 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:16.016821 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:16.016777 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:16.116897 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:16.116869 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:17.485040 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:17.485008 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bk5gw"] Jul 12 08:29:17.485484 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:17.485233 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bk5gw" podUID="76bdfade-53b6-4218-94db-79cf6a0d5855" containerName="manager" containerID="cri-o://b742303cb56ed4e58a9a6caa1f816482c89731c46ddaf794e3998ff8005bfd79" gracePeriod=10 Jul 12 08:29:17.486859 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:17.486835 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bk5gw" Jul 12 08:29:17.741217 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:17.741157 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bk5gw" Jul 12 08:29:17.761447 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:17.761417 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/76bdfade-53b6-4218-94db-79cf6a0d5855-extensions-socket-volume\") pod \"76bdfade-53b6-4218-94db-79cf6a0d5855\" (UID: \"76bdfade-53b6-4218-94db-79cf6a0d5855\") " Jul 12 08:29:17.761634 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:17.761542 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6gvdw\" (UniqueName: \"kubernetes.io/projected/76bdfade-53b6-4218-94db-79cf6a0d5855-kube-api-access-6gvdw\") pod \"76bdfade-53b6-4218-94db-79cf6a0d5855\" (UID: \"76bdfade-53b6-4218-94db-79cf6a0d5855\") " Jul 12 08:29:17.761844 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:17.761817 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76bdfade-53b6-4218-94db-79cf6a0d5855-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "76bdfade-53b6-4218-94db-79cf6a0d5855" (UID: "76bdfade-53b6-4218-94db-79cf6a0d5855"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:29:17.763706 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:17.763684 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76bdfade-53b6-4218-94db-79cf6a0d5855-kube-api-access-6gvdw" (OuterVolumeSpecName: "kube-api-access-6gvdw") pod "76bdfade-53b6-4218-94db-79cf6a0d5855" (UID: "76bdfade-53b6-4218-94db-79cf6a0d5855"). InnerVolumeSpecName "kube-api-access-6gvdw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:29:17.862674 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:17.862641 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/76bdfade-53b6-4218-94db-79cf6a0d5855-extensions-socket-volume\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:29:17.862674 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:17.862668 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6gvdw\" (UniqueName: \"kubernetes.io/projected/76bdfade-53b6-4218-94db-79cf6a0d5855-kube-api-access-6gvdw\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:29:17.883926 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:17.883888 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:17.988169 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:17.988122 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:18.321579 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:18.321528 2566 generic.go:356] "Generic (PLEG): container finished" podID="76bdfade-53b6-4218-94db-79cf6a0d5855" containerID="b742303cb56ed4e58a9a6caa1f816482c89731c46ddaf794e3998ff8005bfd79" exitCode=0 Jul 12 08:29:18.321738 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:18.321591 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bk5gw" Jul 12 08:29:18.321738 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:18.321616 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bk5gw" event={"ID":"76bdfade-53b6-4218-94db-79cf6a0d5855","Type":"ContainerDied","Data":"b742303cb56ed4e58a9a6caa1f816482c89731c46ddaf794e3998ff8005bfd79"} Jul 12 08:29:18.321738 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:18.321670 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bk5gw" event={"ID":"76bdfade-53b6-4218-94db-79cf6a0d5855","Type":"ContainerDied","Data":"07a3488106b11ab39829184663c19a231db0172582d4d41fcc8c79980818bd9d"} Jul 12 08:29:18.321738 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:18.321693 2566 scope.go:117] "RemoveContainer" containerID="b742303cb56ed4e58a9a6caa1f816482c89731c46ddaf794e3998ff8005bfd79" Jul 12 08:29:18.330843 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:18.330828 2566 scope.go:117] "RemoveContainer" containerID="b742303cb56ed4e58a9a6caa1f816482c89731c46ddaf794e3998ff8005bfd79" Jul 12 08:29:18.331075 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:29:18.331058 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b742303cb56ed4e58a9a6caa1f816482c89731c46ddaf794e3998ff8005bfd79\": container with ID starting with b742303cb56ed4e58a9a6caa1f816482c89731c46ddaf794e3998ff8005bfd79 not found: ID does not exist" containerID="b742303cb56ed4e58a9a6caa1f816482c89731c46ddaf794e3998ff8005bfd79" Jul 12 08:29:18.331145 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:18.331082 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b742303cb56ed4e58a9a6caa1f816482c89731c46ddaf794e3998ff8005bfd79"} err="failed to get container status \"b742303cb56ed4e58a9a6caa1f816482c89731c46ddaf794e3998ff8005bfd79\": rpc error: code = NotFound desc = could not find container \"b742303cb56ed4e58a9a6caa1f816482c89731c46ddaf794e3998ff8005bfd79\": container with ID starting with b742303cb56ed4e58a9a6caa1f816482c89731c46ddaf794e3998ff8005bfd79 not found: ID does not exist" Jul 12 08:29:18.344136 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:18.344105 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bk5gw"] Jul 12 08:29:18.349878 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:18.349859 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bk5gw"] Jul 12 08:29:19.025888 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:19.025858 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76bdfade-53b6-4218-94db-79cf6a0d5855" path="/var/lib/kubelet/pods/76bdfade-53b6-4218-94db-79cf6a0d5855/volumes" Jul 12 08:29:22.517319 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:22.517284 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:22.616911 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:22.616874 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:27.264784 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:27.264753 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:27.371685 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:27.371634 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:28.633800 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:28.633766 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:28.735331 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:28.735298 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:28.835981 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:28.835952 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:28.901654 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:28.901573 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:31.286893 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:31.286861 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:31.388470 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:31.388435 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:31.967292 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:31.967245 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:32.043636 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:32.043610 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:34.187494 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:34.187449 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:34.578810 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:34.578780 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:35.475249 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:35.475212 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:35.576070 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:35.576040 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:44.868531 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:44.868498 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:44.927061 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:44.927028 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:45.419396 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:45.419363 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:45.419842 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:45.419822 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jkb4f"] Jul 12 08:29:45.420140 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:45.420129 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76bdfade-53b6-4218-94db-79cf6a0d5855" containerName="manager" Jul 12 08:29:45.420177 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:45.420141 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="76bdfade-53b6-4218-94db-79cf6a0d5855" containerName="manager" Jul 12 08:29:45.420209 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:45.420201 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="76bdfade-53b6-4218-94db-79cf6a0d5855" containerName="manager" Jul 12 08:29:45.423808 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:45.423792 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-jkb4f" Jul 12 08:29:45.426643 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:45.426622 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-mtxpk\"" Jul 12 08:29:45.426754 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:45.426650 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 12 08:29:45.432612 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:45.432593 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jkb4f"] Jul 12 08:29:45.473438 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:45.473413 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/607b5c57-6ece-452a-a0e2-a9c969a4cbb1-config-file\") pod \"limitador-limitador-64c94fd64f-jkb4f\" (UID: \"607b5c57-6ece-452a-a0e2-a9c969a4cbb1\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jkb4f" Jul 12 08:29:45.473566 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:45.473477 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkjx7\" (UniqueName: \"kubernetes.io/projected/607b5c57-6ece-452a-a0e2-a9c969a4cbb1-kube-api-access-pkjx7\") pod \"limitador-limitador-64c94fd64f-jkb4f\" (UID: \"607b5c57-6ece-452a-a0e2-a9c969a4cbb1\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jkb4f" Jul 12 08:29:45.519158 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:45.519124 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jkb4f"] Jul 12 08:29:45.529043 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:45.529012 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:45.574300 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:45.574272 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkjx7\" (UniqueName: \"kubernetes.io/projected/607b5c57-6ece-452a-a0e2-a9c969a4cbb1-kube-api-access-pkjx7\") pod \"limitador-limitador-64c94fd64f-jkb4f\" (UID: \"607b5c57-6ece-452a-a0e2-a9c969a4cbb1\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jkb4f" Jul 12 08:29:45.574426 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:45.574322 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/607b5c57-6ece-452a-a0e2-a9c969a4cbb1-config-file\") pod \"limitador-limitador-64c94fd64f-jkb4f\" (UID: \"607b5c57-6ece-452a-a0e2-a9c969a4cbb1\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jkb4f" Jul 12 08:29:45.574986 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:45.574969 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/607b5c57-6ece-452a-a0e2-a9c969a4cbb1-config-file\") pod \"limitador-limitador-64c94fd64f-jkb4f\" (UID: \"607b5c57-6ece-452a-a0e2-a9c969a4cbb1\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jkb4f" Jul 12 08:29:45.585019 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:45.584998 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkjx7\" (UniqueName: \"kubernetes.io/projected/607b5c57-6ece-452a-a0e2-a9c969a4cbb1-kube-api-access-pkjx7\") pod \"limitador-limitador-64c94fd64f-jkb4f\" (UID: \"607b5c57-6ece-452a-a0e2-a9c969a4cbb1\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jkb4f" Jul 12 08:29:45.601161 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:45.601133 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:45.733556 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:45.733527 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-jkb4f" Jul 12 08:29:45.864067 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:45.863955 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jkb4f"] Jul 12 08:29:45.866216 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:29:45.866191 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod607b5c57_6ece_452a_a0e2_a9c969a4cbb1.slice/crio-b03470c97f819609a06814f3d0a4ac5b06f0269f64094294cb6577a46daf0f89 WatchSource:0}: Error finding container b03470c97f819609a06814f3d0a4ac5b06f0269f64094294cb6577a46daf0f89: Status 404 returned error can't find the container with id b03470c97f819609a06814f3d0a4ac5b06f0269f64094294cb6577a46daf0f89 Jul 12 08:29:46.291372 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:46.291337 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:46.359649 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:46.359618 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:46.432200 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:46.432172 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-f7x2b"] Jul 12 08:29:46.435306 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:46.435291 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-f7x2b" Jul 12 08:29:46.437744 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:46.437728 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-xgvs5\"" Jul 12 08:29:46.440372 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:46.440352 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-f7x2b"] Jul 12 08:29:46.446675 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:46.446643 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-jkb4f" event={"ID":"607b5c57-6ece-452a-a0e2-a9c969a4cbb1","Type":"ContainerStarted","Data":"b03470c97f819609a06814f3d0a4ac5b06f0269f64094294cb6577a46daf0f89"} Jul 12 08:29:46.482655 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:46.482622 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4f4h\" (UniqueName: \"kubernetes.io/projected/838c5b08-d880-446b-885c-26af4262eb9c-kube-api-access-v4f4h\") pod \"authorino-75d556bd77-f7x2b\" (UID: \"838c5b08-d880-446b-885c-26af4262eb9c\") " pod="kuadrant-system/authorino-75d556bd77-f7x2b" Jul 12 08:29:46.583362 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:46.583277 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4f4h\" (UniqueName: \"kubernetes.io/projected/838c5b08-d880-446b-885c-26af4262eb9c-kube-api-access-v4f4h\") pod \"authorino-75d556bd77-f7x2b\" (UID: \"838c5b08-d880-446b-885c-26af4262eb9c\") " pod="kuadrant-system/authorino-75d556bd77-f7x2b" Jul 12 08:29:46.593019 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:46.592951 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4f4h\" (UniqueName: \"kubernetes.io/projected/838c5b08-d880-446b-885c-26af4262eb9c-kube-api-access-v4f4h\") pod \"authorino-75d556bd77-f7x2b\" (UID: \"838c5b08-d880-446b-885c-26af4262eb9c\") " pod="kuadrant-system/authorino-75d556bd77-f7x2b" Jul 12 08:29:46.745624 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:46.745570 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-f7x2b" Jul 12 08:29:46.934026 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:46.933998 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-f7x2b"] Jul 12 08:29:46.936506 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:29:46.936474 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod838c5b08_d880_446b_885c_26af4262eb9c.slice/crio-c9e3e4f951f87fdd1a84e0435ae94691e41bd5b25bc15163fe6b190675719746 WatchSource:0}: Error finding container c9e3e4f951f87fdd1a84e0435ae94691e41bd5b25bc15163fe6b190675719746: Status 404 returned error can't find the container with id c9e3e4f951f87fdd1a84e0435ae94691e41bd5b25bc15163fe6b190675719746 Jul 12 08:29:47.453424 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:47.453390 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-f7x2b" event={"ID":"838c5b08-d880-446b-885c-26af4262eb9c","Type":"ContainerStarted","Data":"c9e3e4f951f87fdd1a84e0435ae94691e41bd5b25bc15163fe6b190675719746"} Jul 12 08:29:48.867564 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:48.867402 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:48.967726 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:48.967673 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:29:49.468656 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:49.468623 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-jkb4f" event={"ID":"607b5c57-6ece-452a-a0e2-a9c969a4cbb1","Type":"ContainerStarted","Data":"95c2b06d378e8846fbccffad95f7745a8c55f68ea3f5e30fb59dd86304fbb13e"} Jul 12 08:29:49.468830 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:49.468778 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-jkb4f" Jul 12 08:29:49.486621 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:49.486580 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-jkb4f" podStartSLOduration=1.754520987 podStartE2EDuration="4.486563819s" totalImagesPullingTime="2.732042832s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:29:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:29:49.485948434 +0000 UTC m=+513.146744116" watchObservedRunningTime="2026-07-12 08:29:49.486563819 +0000 UTC m=+513.147359478" Jul 12 08:29:50.474840 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:50.474814 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-f7x2b" event={"ID":"838c5b08-d880-446b-885c-26af4262eb9c","Type":"ContainerStarted","Data":"a33c2346f8245b590314ad011af4341a88314f3b91b6e842d3ee1d0d7dada3f7"} Jul 12 08:29:50.494010 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:29:50.493777 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-f7x2b" podStartSLOduration=1.021606971 podStartE2EDuration="4.493759063s" totalImagesPullingTime="3.472152092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:29:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:29:50.491272073 +0000 UTC m=+514.152067752" watchObservedRunningTime="2026-07-12 08:29:50.493759063 +0000 UTC m=+514.154554724" Jul 12 08:30:00.476373 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:00.476345 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-jkb4f" Jul 12 08:30:00.723603 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:00.723568 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jkb4f"] Jul 12 08:30:00.723771 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:00.723750 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-jkb4f" podUID="607b5c57-6ece-452a-a0e2-a9c969a4cbb1" containerName="limitador" containerID="cri-o://95c2b06d378e8846fbccffad95f7745a8c55f68ea3f5e30fb59dd86304fbb13e" gracePeriod=30 Jul 12 08:30:00.874724 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:00.874682 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:30:01.274635 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:01.274612 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-jkb4f" Jul 12 08:30:01.394888 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:01.394804 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pkjx7\" (UniqueName: \"kubernetes.io/projected/607b5c57-6ece-452a-a0e2-a9c969a4cbb1-kube-api-access-pkjx7\") pod \"607b5c57-6ece-452a-a0e2-a9c969a4cbb1\" (UID: \"607b5c57-6ece-452a-a0e2-a9c969a4cbb1\") " Jul 12 08:30:01.394888 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:01.394862 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/607b5c57-6ece-452a-a0e2-a9c969a4cbb1-config-file\") pod \"607b5c57-6ece-452a-a0e2-a9c969a4cbb1\" (UID: \"607b5c57-6ece-452a-a0e2-a9c969a4cbb1\") " Jul 12 08:30:01.395233 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:01.395204 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/607b5c57-6ece-452a-a0e2-a9c969a4cbb1-config-file" (OuterVolumeSpecName: "config-file") pod "607b5c57-6ece-452a-a0e2-a9c969a4cbb1" (UID: "607b5c57-6ece-452a-a0e2-a9c969a4cbb1"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 08:30:01.396926 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:01.396901 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/607b5c57-6ece-452a-a0e2-a9c969a4cbb1-kube-api-access-pkjx7" (OuterVolumeSpecName: "kube-api-access-pkjx7") pod "607b5c57-6ece-452a-a0e2-a9c969a4cbb1" (UID: "607b5c57-6ece-452a-a0e2-a9c969a4cbb1"). InnerVolumeSpecName "kube-api-access-pkjx7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:30:01.496202 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:01.496171 2566 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/607b5c57-6ece-452a-a0e2-a9c969a4cbb1-config-file\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:30:01.496202 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:01.496197 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkjx7\" (UniqueName: \"kubernetes.io/projected/607b5c57-6ece-452a-a0e2-a9c969a4cbb1-kube-api-access-pkjx7\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:30:01.530053 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:01.530025 2566 generic.go:356] "Generic (PLEG): container finished" podID="607b5c57-6ece-452a-a0e2-a9c969a4cbb1" containerID="95c2b06d378e8846fbccffad95f7745a8c55f68ea3f5e30fb59dd86304fbb13e" exitCode=0 Jul 12 08:30:01.530171 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:01.530068 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-jkb4f" Jul 12 08:30:01.530171 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:01.530091 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-jkb4f" event={"ID":"607b5c57-6ece-452a-a0e2-a9c969a4cbb1","Type":"ContainerDied","Data":"95c2b06d378e8846fbccffad95f7745a8c55f68ea3f5e30fb59dd86304fbb13e"} Jul 12 08:30:01.530171 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:01.530130 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-jkb4f" event={"ID":"607b5c57-6ece-452a-a0e2-a9c969a4cbb1","Type":"ContainerDied","Data":"b03470c97f819609a06814f3d0a4ac5b06f0269f64094294cb6577a46daf0f89"} Jul 12 08:30:01.530171 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:01.530150 2566 scope.go:117] "RemoveContainer" containerID="95c2b06d378e8846fbccffad95f7745a8c55f68ea3f5e30fb59dd86304fbb13e" Jul 12 08:30:01.538435 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:01.538417 2566 scope.go:117] "RemoveContainer" containerID="95c2b06d378e8846fbccffad95f7745a8c55f68ea3f5e30fb59dd86304fbb13e" Jul 12 08:30:01.538716 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:30:01.538698 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95c2b06d378e8846fbccffad95f7745a8c55f68ea3f5e30fb59dd86304fbb13e\": container with ID starting with 95c2b06d378e8846fbccffad95f7745a8c55f68ea3f5e30fb59dd86304fbb13e not found: ID does not exist" containerID="95c2b06d378e8846fbccffad95f7745a8c55f68ea3f5e30fb59dd86304fbb13e" Jul 12 08:30:01.538769 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:01.538725 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95c2b06d378e8846fbccffad95f7745a8c55f68ea3f5e30fb59dd86304fbb13e"} err="failed to get container status \"95c2b06d378e8846fbccffad95f7745a8c55f68ea3f5e30fb59dd86304fbb13e\": rpc error: code = NotFound desc = could not find container \"95c2b06d378e8846fbccffad95f7745a8c55f68ea3f5e30fb59dd86304fbb13e\": container with ID starting with 95c2b06d378e8846fbccffad95f7745a8c55f68ea3f5e30fb59dd86304fbb13e not found: ID does not exist" Jul 12 08:30:01.549014 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:01.548991 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jkb4f"] Jul 12 08:30:01.552404 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:01.552384 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jkb4f"] Jul 12 08:30:03.026136 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:03.026096 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="607b5c57-6ece-452a-a0e2-a9c969a4cbb1" path="/var/lib/kubelet/pods/607b5c57-6ece-452a-a0e2-a9c969a4cbb1/volumes" Jul 12 08:30:03.043354 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:03.043314 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:30:03.142133 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:03.142098 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:30:03.469909 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:03.469828 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-jj9j9"] Jul 12 08:30:03.470222 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:03.470206 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="607b5c57-6ece-452a-a0e2-a9c969a4cbb1" containerName="limitador" Jul 12 08:30:03.470304 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:03.470226 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="607b5c57-6ece-452a-a0e2-a9c969a4cbb1" containerName="limitador" Jul 12 08:30:03.470355 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:03.470327 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="607b5c57-6ece-452a-a0e2-a9c969a4cbb1" containerName="limitador" Jul 12 08:30:03.474719 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:03.474698 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jj9j9" Jul 12 08:30:03.477103 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:03.477078 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 12 08:30:03.477103 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:03.477102 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 12 08:30:03.477784 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:03.477763 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 12 08:30:03.477859 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:03.477793 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-5vlq7\"" Jul 12 08:30:03.481170 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:03.481148 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-jj9j9"] Jul 12 08:30:03.515687 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:03.515657 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:30:03.613992 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:03.613969 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fk8f5\" (UniqueName: \"kubernetes.io/projected/08ad23d7-9f77-42b2-8bc1-9289f3aaa6e3-kube-api-access-fk8f5\") pod \"postgres-69fb4594cf-jj9j9\" (UID: \"08ad23d7-9f77-42b2-8bc1-9289f3aaa6e3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jj9j9" Jul 12 08:30:03.614102 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:03.614013 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/08ad23d7-9f77-42b2-8bc1-9289f3aaa6e3-data\") pod \"postgres-69fb4594cf-jj9j9\" (UID: \"08ad23d7-9f77-42b2-8bc1-9289f3aaa6e3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jj9j9" Jul 12 08:30:03.714938 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:03.714912 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fk8f5\" (UniqueName: \"kubernetes.io/projected/08ad23d7-9f77-42b2-8bc1-9289f3aaa6e3-kube-api-access-fk8f5\") pod \"postgres-69fb4594cf-jj9j9\" (UID: \"08ad23d7-9f77-42b2-8bc1-9289f3aaa6e3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jj9j9" Jul 12 08:30:03.715058 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:03.714952 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/08ad23d7-9f77-42b2-8bc1-9289f3aaa6e3-data\") pod \"postgres-69fb4594cf-jj9j9\" (UID: \"08ad23d7-9f77-42b2-8bc1-9289f3aaa6e3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jj9j9" Jul 12 08:30:03.715244 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:03.715229 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/08ad23d7-9f77-42b2-8bc1-9289f3aaa6e3-data\") pod \"postgres-69fb4594cf-jj9j9\" (UID: \"08ad23d7-9f77-42b2-8bc1-9289f3aaa6e3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jj9j9" Jul 12 08:30:03.722682 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:03.722624 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fk8f5\" (UniqueName: \"kubernetes.io/projected/08ad23d7-9f77-42b2-8bc1-9289f3aaa6e3-kube-api-access-fk8f5\") pod \"postgres-69fb4594cf-jj9j9\" (UID: \"08ad23d7-9f77-42b2-8bc1-9289f3aaa6e3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jj9j9" Jul 12 08:30:03.786651 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:03.786624 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jj9j9" Jul 12 08:30:03.949988 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:03.949954 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-jj9j9"] Jul 12 08:30:03.952832 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:30:03.952809 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08ad23d7_9f77_42b2_8bc1_9289f3aaa6e3.slice/crio-ac4191eb615edf50ea6dd220d9b7ac8ebac647fce1d2e630b7f054f718054d9c WatchSource:0}: Error finding container ac4191eb615edf50ea6dd220d9b7ac8ebac647fce1d2e630b7f054f718054d9c: Status 404 returned error can't find the container with id ac4191eb615edf50ea6dd220d9b7ac8ebac647fce1d2e630b7f054f718054d9c Jul 12 08:30:04.544129 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:04.544094 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jj9j9" event={"ID":"08ad23d7-9f77-42b2-8bc1-9289f3aaa6e3","Type":"ContainerStarted","Data":"ac4191eb615edf50ea6dd220d9b7ac8ebac647fce1d2e630b7f054f718054d9c"} Jul 12 08:30:10.572569 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:10.572517 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jj9j9" event={"ID":"08ad23d7-9f77-42b2-8bc1-9289f3aaa6e3","Type":"ContainerStarted","Data":"d5f7c6553342a307b320c71c5bdc20e1cbde23bfa2cf1c4d00bab780505b6d66"} Jul 12 08:30:10.572922 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:10.572600 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jj9j9" Jul 12 08:30:10.587996 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:10.587949 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jj9j9" podStartSLOduration=1.847818646 podStartE2EDuration="7.587935608s" totalImagesPullingTime="5.740116962s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:30:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:30:10.587541154 +0000 UTC m=+534.248336808" watchObservedRunningTime="2026-07-12 08:30:10.587935608 +0000 UTC m=+534.248731269" Jul 12 08:30:16.606008 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:16.605980 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jj9j9" Jul 12 08:30:16.924310 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:16.924208 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:30:17.250731 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:17.250697 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:30:17.375280 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:17.375246 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-864cf568c6-2tts9"] Jul 12 08:30:17.381131 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:17.381113 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-864cf568c6-2tts9" Jul 12 08:30:17.383296 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:17.383275 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 12 08:30:17.389090 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:17.389066 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-864cf568c6-2tts9"] Jul 12 08:30:17.428652 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:17.428614 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4cfc9b31-09d1-4965-93e9-8aeef9587da0-tls-cert\") pod \"authorino-864cf568c6-2tts9\" (UID: \"4cfc9b31-09d1-4965-93e9-8aeef9587da0\") " pod="kuadrant-system/authorino-864cf568c6-2tts9" Jul 12 08:30:17.428792 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:17.428700 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9q25\" (UniqueName: \"kubernetes.io/projected/4cfc9b31-09d1-4965-93e9-8aeef9587da0-kube-api-access-w9q25\") pod \"authorino-864cf568c6-2tts9\" (UID: \"4cfc9b31-09d1-4965-93e9-8aeef9587da0\") " pod="kuadrant-system/authorino-864cf568c6-2tts9" Jul 12 08:30:17.530076 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:17.530000 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4cfc9b31-09d1-4965-93e9-8aeef9587da0-tls-cert\") pod \"authorino-864cf568c6-2tts9\" (UID: \"4cfc9b31-09d1-4965-93e9-8aeef9587da0\") " pod="kuadrant-system/authorino-864cf568c6-2tts9" Jul 12 08:30:17.530214 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:17.530082 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9q25\" (UniqueName: \"kubernetes.io/projected/4cfc9b31-09d1-4965-93e9-8aeef9587da0-kube-api-access-w9q25\") pod \"authorino-864cf568c6-2tts9\" (UID: \"4cfc9b31-09d1-4965-93e9-8aeef9587da0\") " pod="kuadrant-system/authorino-864cf568c6-2tts9" Jul 12 08:30:17.532305 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:17.532274 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4cfc9b31-09d1-4965-93e9-8aeef9587da0-tls-cert\") pod \"authorino-864cf568c6-2tts9\" (UID: \"4cfc9b31-09d1-4965-93e9-8aeef9587da0\") " pod="kuadrant-system/authorino-864cf568c6-2tts9" Jul 12 08:30:17.538606 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:17.538579 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9q25\" (UniqueName: \"kubernetes.io/projected/4cfc9b31-09d1-4965-93e9-8aeef9587da0-kube-api-access-w9q25\") pod \"authorino-864cf568c6-2tts9\" (UID: \"4cfc9b31-09d1-4965-93e9-8aeef9587da0\") " pod="kuadrant-system/authorino-864cf568c6-2tts9" Jul 12 08:30:17.689771 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:17.689736 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-864cf568c6-2tts9" Jul 12 08:30:17.816063 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:17.816039 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-864cf568c6-2tts9"] Jul 12 08:30:17.817926 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:30:17.817899 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4cfc9b31_09d1_4965_93e9_8aeef9587da0.slice/crio-beb398cd6ff00b0f37f859a94d4f0bfd5b1ba0e63504855dee84428cd4d5cd83 WatchSource:0}: Error finding container beb398cd6ff00b0f37f859a94d4f0bfd5b1ba0e63504855dee84428cd4d5cd83: Status 404 returned error can't find the container with id beb398cd6ff00b0f37f859a94d4f0bfd5b1ba0e63504855dee84428cd4d5cd83 Jul 12 08:30:18.609468 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:18.609430 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-864cf568c6-2tts9" event={"ID":"4cfc9b31-09d1-4965-93e9-8aeef9587da0","Type":"ContainerStarted","Data":"2e8225c6870d882f2ff59b0d8b42a53dde6ab1dbbcc53fa2b22071139d6841de"} Jul 12 08:30:18.609644 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:18.609472 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-864cf568c6-2tts9" event={"ID":"4cfc9b31-09d1-4965-93e9-8aeef9587da0","Type":"ContainerStarted","Data":"beb398cd6ff00b0f37f859a94d4f0bfd5b1ba0e63504855dee84428cd4d5cd83"} Jul 12 08:30:18.625501 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:18.625453 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-864cf568c6-2tts9" podStartSLOduration=1.2954671819999999 podStartE2EDuration="1.625439833s" totalImagesPullingTime="329.972651ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:30:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:30:18.6248126 +0000 UTC m=+542.285608266" watchObservedRunningTime="2026-07-12 08:30:18.625439833 +0000 UTC m=+542.286235493" Jul 12 08:30:18.652852 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:18.652821 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-f7x2b"] Jul 12 08:30:18.653051 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:18.653025 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-f7x2b" podUID="838c5b08-d880-446b-885c-26af4262eb9c" containerName="authorino" containerID="cri-o://a33c2346f8245b590314ad011af4341a88314f3b91b6e842d3ee1d0d7dada3f7" gracePeriod=30 Jul 12 08:30:18.893411 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:18.893390 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-f7x2b" Jul 12 08:30:19.040721 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:19.040695 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4f4h\" (UniqueName: \"kubernetes.io/projected/838c5b08-d880-446b-885c-26af4262eb9c-kube-api-access-v4f4h\") pod \"838c5b08-d880-446b-885c-26af4262eb9c\" (UID: \"838c5b08-d880-446b-885c-26af4262eb9c\") " Jul 12 08:30:19.042721 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:19.042693 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/838c5b08-d880-446b-885c-26af4262eb9c-kube-api-access-v4f4h" (OuterVolumeSpecName: "kube-api-access-v4f4h") pod "838c5b08-d880-446b-885c-26af4262eb9c" (UID: "838c5b08-d880-446b-885c-26af4262eb9c"). InnerVolumeSpecName "kube-api-access-v4f4h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:30:19.141983 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:19.141906 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4f4h\" (UniqueName: \"kubernetes.io/projected/838c5b08-d880-446b-885c-26af4262eb9c-kube-api-access-v4f4h\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:30:19.615093 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:19.615062 2566 generic.go:356] "Generic (PLEG): container finished" podID="838c5b08-d880-446b-885c-26af4262eb9c" containerID="a33c2346f8245b590314ad011af4341a88314f3b91b6e842d3ee1d0d7dada3f7" exitCode=0 Jul 12 08:30:19.615253 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:19.615106 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-f7x2b" Jul 12 08:30:19.615253 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:19.615152 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-f7x2b" event={"ID":"838c5b08-d880-446b-885c-26af4262eb9c","Type":"ContainerDied","Data":"a33c2346f8245b590314ad011af4341a88314f3b91b6e842d3ee1d0d7dada3f7"} Jul 12 08:30:19.615253 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:19.615195 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-f7x2b" event={"ID":"838c5b08-d880-446b-885c-26af4262eb9c","Type":"ContainerDied","Data":"c9e3e4f951f87fdd1a84e0435ae94691e41bd5b25bc15163fe6b190675719746"} Jul 12 08:30:19.615253 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:19.615231 2566 scope.go:117] "RemoveContainer" containerID="a33c2346f8245b590314ad011af4341a88314f3b91b6e842d3ee1d0d7dada3f7" Jul 12 08:30:19.623870 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:19.623855 2566 scope.go:117] "RemoveContainer" containerID="a33c2346f8245b590314ad011af4341a88314f3b91b6e842d3ee1d0d7dada3f7" Jul 12 08:30:19.624108 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:30:19.624091 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a33c2346f8245b590314ad011af4341a88314f3b91b6e842d3ee1d0d7dada3f7\": container with ID starting with a33c2346f8245b590314ad011af4341a88314f3b91b6e842d3ee1d0d7dada3f7 not found: ID does not exist" containerID="a33c2346f8245b590314ad011af4341a88314f3b91b6e842d3ee1d0d7dada3f7" Jul 12 08:30:19.624158 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:19.624113 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a33c2346f8245b590314ad011af4341a88314f3b91b6e842d3ee1d0d7dada3f7"} err="failed to get container status \"a33c2346f8245b590314ad011af4341a88314f3b91b6e842d3ee1d0d7dada3f7\": rpc error: code = NotFound desc = could not find container \"a33c2346f8245b590314ad011af4341a88314f3b91b6e842d3ee1d0d7dada3f7\": container with ID starting with a33c2346f8245b590314ad011af4341a88314f3b91b6e842d3ee1d0d7dada3f7 not found: ID does not exist" Jul 12 08:30:19.637798 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:19.637770 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-f7x2b"] Jul 12 08:30:19.640858 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:19.640840 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-f7x2b"] Jul 12 08:30:21.025745 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:21.025711 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="838c5b08-d880-446b-885c-26af4262eb9c" path="/var/lib/kubelet/pods/838c5b08-d880-446b-885c-26af4262eb9c/volumes" Jul 12 08:30:21.590916 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:21.590878 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:30:21.692237 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:21.692210 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:30:22.964536 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:22.964491 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:30:22.972830 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:22.972809 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5d9566f4d5-7vp8h"] Jul 12 08:30:22.973173 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:22.973160 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="838c5b08-d880-446b-885c-26af4262eb9c" containerName="authorino" Jul 12 08:30:22.973210 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:22.973175 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="838c5b08-d880-446b-885c-26af4262eb9c" containerName="authorino" Jul 12 08:30:22.973255 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:22.973245 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="838c5b08-d880-446b-885c-26af4262eb9c" containerName="authorino" Jul 12 08:30:23.053475 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:23.053444 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5d9566f4d5-7vp8h"] Jul 12 08:30:23.053657 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:23.053579 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d9566f4d5-7vp8h" Jul 12 08:30:23.056148 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:23.056128 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 12 08:30:23.056325 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:23.056305 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 12 08:30:23.056424 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:23.056401 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-hq6z8\"" Jul 12 08:30:23.061043 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:23.060824 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:30:23.067380 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:23.067347 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsdcg\" (UniqueName: \"kubernetes.io/projected/d269c326-cc57-4972-a78d-5057f95ada05-kube-api-access-qsdcg\") pod \"maas-controller-5d9566f4d5-7vp8h\" (UID: \"d269c326-cc57-4972-a78d-5057f95ada05\") " pod="opendatahub/maas-controller-5d9566f4d5-7vp8h" Jul 12 08:30:23.067480 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:23.067430 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d269c326-cc57-4972-a78d-5057f95ada05-webhook-cert\") pod \"maas-controller-5d9566f4d5-7vp8h\" (UID: \"d269c326-cc57-4972-a78d-5057f95ada05\") " pod="opendatahub/maas-controller-5d9566f4d5-7vp8h" Jul 12 08:30:23.134886 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:23.134859 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:30:23.138689 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:23.138669 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-69948768-686nc"] Jul 12 08:30:23.157639 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:23.157619 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-69948768-686nc"] Jul 12 08:30:23.157741 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:23.157711 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69948768-686nc" Jul 12 08:30:23.168203 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:23.168183 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d269c326-cc57-4972-a78d-5057f95ada05-webhook-cert\") pod \"maas-controller-5d9566f4d5-7vp8h\" (UID: \"d269c326-cc57-4972-a78d-5057f95ada05\") " pod="opendatahub/maas-controller-5d9566f4d5-7vp8h" Jul 12 08:30:23.168322 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:23.168226 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsdcg\" (UniqueName: \"kubernetes.io/projected/d269c326-cc57-4972-a78d-5057f95ada05-kube-api-access-qsdcg\") pod \"maas-controller-5d9566f4d5-7vp8h\" (UID: \"d269c326-cc57-4972-a78d-5057f95ada05\") " pod="opendatahub/maas-controller-5d9566f4d5-7vp8h" Jul 12 08:30:23.168322 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:23.168261 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z585\" (UniqueName: \"kubernetes.io/projected/e7a1cfe8-651a-413b-8b8b-5517fa99dacb-kube-api-access-6z585\") pod \"maas-controller-69948768-686nc\" (UID: \"e7a1cfe8-651a-413b-8b8b-5517fa99dacb\") " pod="opendatahub/maas-controller-69948768-686nc" Jul 12 08:30:23.168322 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:23.168294 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7a1cfe8-651a-413b-8b8b-5517fa99dacb-webhook-cert\") pod \"maas-controller-69948768-686nc\" (UID: \"e7a1cfe8-651a-413b-8b8b-5517fa99dacb\") " pod="opendatahub/maas-controller-69948768-686nc" Jul 12 08:30:23.168322 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:30:23.168305 2566 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 12 08:30:23.168521 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:30:23.168372 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d269c326-cc57-4972-a78d-5057f95ada05-webhook-cert podName:d269c326-cc57-4972-a78d-5057f95ada05 nodeName:}" failed. No retries permitted until 2026-07-12 08:30:23.66835581 +0000 UTC m=+547.329151462 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/d269c326-cc57-4972-a78d-5057f95ada05-webhook-cert") pod "maas-controller-5d9566f4d5-7vp8h" (UID: "d269c326-cc57-4972-a78d-5057f95ada05") : secret "maas-controller-webhook-cert" not found Jul 12 08:30:23.178634 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:23.178608 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsdcg\" (UniqueName: \"kubernetes.io/projected/d269c326-cc57-4972-a78d-5057f95ada05-kube-api-access-qsdcg\") pod \"maas-controller-5d9566f4d5-7vp8h\" (UID: \"d269c326-cc57-4972-a78d-5057f95ada05\") " pod="opendatahub/maas-controller-5d9566f4d5-7vp8h" Jul 12 08:30:23.269460 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:23.269427 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z585\" (UniqueName: \"kubernetes.io/projected/e7a1cfe8-651a-413b-8b8b-5517fa99dacb-kube-api-access-6z585\") pod \"maas-controller-69948768-686nc\" (UID: \"e7a1cfe8-651a-413b-8b8b-5517fa99dacb\") " pod="opendatahub/maas-controller-69948768-686nc" Jul 12 08:30:23.269615 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:23.269479 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7a1cfe8-651a-413b-8b8b-5517fa99dacb-webhook-cert\") pod \"maas-controller-69948768-686nc\" (UID: \"e7a1cfe8-651a-413b-8b8b-5517fa99dacb\") " pod="opendatahub/maas-controller-69948768-686nc" Jul 12 08:30:23.269667 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:30:23.269638 2566 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 12 08:30:23.269746 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:30:23.269734 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7a1cfe8-651a-413b-8b8b-5517fa99dacb-webhook-cert podName:e7a1cfe8-651a-413b-8b8b-5517fa99dacb nodeName:}" failed. No retries permitted until 2026-07-12 08:30:23.769713743 +0000 UTC m=+547.430509382 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/e7a1cfe8-651a-413b-8b8b-5517fa99dacb-webhook-cert") pod "maas-controller-69948768-686nc" (UID: "e7a1cfe8-651a-413b-8b8b-5517fa99dacb") : secret "maas-controller-webhook-cert" not found Jul 12 08:30:23.287655 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:23.287631 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z585\" (UniqueName: \"kubernetes.io/projected/e7a1cfe8-651a-413b-8b8b-5517fa99dacb-kube-api-access-6z585\") pod \"maas-controller-69948768-686nc\" (UID: \"e7a1cfe8-651a-413b-8b8b-5517fa99dacb\") " pod="opendatahub/maas-controller-69948768-686nc" Jul 12 08:30:23.422632 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:23.422597 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:30:23.671795 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:23.671722 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d269c326-cc57-4972-a78d-5057f95ada05-webhook-cert\") pod \"maas-controller-5d9566f4d5-7vp8h\" (UID: \"d269c326-cc57-4972-a78d-5057f95ada05\") " pod="opendatahub/maas-controller-5d9566f4d5-7vp8h" Jul 12 08:30:23.673957 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:23.673927 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d269c326-cc57-4972-a78d-5057f95ada05-webhook-cert\") pod \"maas-controller-5d9566f4d5-7vp8h\" (UID: \"d269c326-cc57-4972-a78d-5057f95ada05\") " pod="opendatahub/maas-controller-5d9566f4d5-7vp8h" Jul 12 08:30:23.772503 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:23.772473 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7a1cfe8-651a-413b-8b8b-5517fa99dacb-webhook-cert\") pod \"maas-controller-69948768-686nc\" (UID: \"e7a1cfe8-651a-413b-8b8b-5517fa99dacb\") " pod="opendatahub/maas-controller-69948768-686nc" Jul 12 08:30:23.774671 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:23.774651 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7a1cfe8-651a-413b-8b8b-5517fa99dacb-webhook-cert\") pod \"maas-controller-69948768-686nc\" (UID: \"e7a1cfe8-651a-413b-8b8b-5517fa99dacb\") " pod="opendatahub/maas-controller-69948768-686nc" Jul 12 08:30:23.964966 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:23.964896 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d9566f4d5-7vp8h" Jul 12 08:30:24.067521 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:24.067484 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69948768-686nc" Jul 12 08:30:24.225195 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:24.225137 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-69948768-686nc"] Jul 12 08:30:24.227636 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:30:24.227605 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7a1cfe8_651a_413b_8b8b_5517fa99dacb.slice/crio-3a928249655617a258968a7d9763a72173d3afcac47b06f5f4dfdf3fe1af0736 WatchSource:0}: Error finding container 3a928249655617a258968a7d9763a72173d3afcac47b06f5f4dfdf3fe1af0736: Status 404 returned error can't find the container with id 3a928249655617a258968a7d9763a72173d3afcac47b06f5f4dfdf3fe1af0736 Jul 12 08:30:24.403929 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:24.403907 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5d9566f4d5-7vp8h"] Jul 12 08:30:24.405650 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:30:24.405621 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd269c326_cc57_4972_a78d_5057f95ada05.slice/crio-939aeead17f5d8b3e72bfed755f0898e23747e9dc562cce88aa4d7a8b92bb321 WatchSource:0}: Error finding container 939aeead17f5d8b3e72bfed755f0898e23747e9dc562cce88aa4d7a8b92bb321: Status 404 returned error can't find the container with id 939aeead17f5d8b3e72bfed755f0898e23747e9dc562cce88aa4d7a8b92bb321 Jul 12 08:30:24.639179 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:24.639147 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d9566f4d5-7vp8h" event={"ID":"d269c326-cc57-4972-a78d-5057f95ada05","Type":"ContainerStarted","Data":"939aeead17f5d8b3e72bfed755f0898e23747e9dc562cce88aa4d7a8b92bb321"} Jul 12 08:30:24.640203 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:24.640182 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69948768-686nc" event={"ID":"e7a1cfe8-651a-413b-8b8b-5517fa99dacb","Type":"ContainerStarted","Data":"3a928249655617a258968a7d9763a72173d3afcac47b06f5f4dfdf3fe1af0736"} Jul 12 08:30:24.973263 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:24.973118 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:30:25.076219 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:25.076165 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:30:27.498311 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:27.498275 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:30:27.618222 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:27.618134 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:30:27.656951 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:27.656924 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d9566f4d5-7vp8h" event={"ID":"d269c326-cc57-4972-a78d-5057f95ada05","Type":"ContainerStarted","Data":"0c8d827832022590ebc2f86157e8c06540e070e396ca57c181da1605b8271b06"} Jul 12 08:30:27.657084 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:27.656996 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5d9566f4d5-7vp8h" Jul 12 08:30:27.658451 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:27.658428 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69948768-686nc" event={"ID":"e7a1cfe8-651a-413b-8b8b-5517fa99dacb","Type":"ContainerStarted","Data":"73d0b3bfe45935001c0ec475e929d540efd6a9fcbeb2f93164e74bb63092f7a6"} Jul 12 08:30:27.658570 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:27.658492 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-69948768-686nc" Jul 12 08:30:27.688053 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:27.687993 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5d9566f4d5-7vp8h" podStartSLOduration=2.789156127 podStartE2EDuration="5.687982234s" totalImagesPullingTime="2.898826107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:30:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:30:27.687330168 +0000 UTC m=+551.348125847" watchObservedRunningTime="2026-07-12 08:30:27.687982234 +0000 UTC m=+551.348777894" Jul 12 08:30:27.730163 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:27.730105 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-69948768-686nc" podStartSLOduration=1.641667306 podStartE2EDuration="4.730085738s" totalImagesPullingTime="3.088418432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:30:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:30:27.728439758 +0000 UTC m=+551.389235419" watchObservedRunningTime="2026-07-12 08:30:27.730085738 +0000 UTC m=+551.390881403" Jul 12 08:30:38.668609 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:38.668577 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-69948768-686nc" Jul 12 08:30:38.669014 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:38.668646 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5d9566f4d5-7vp8h" Jul 12 08:30:38.727197 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:38.727169 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5d9566f4d5-7vp8h"] Jul 12 08:30:38.727440 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:38.727413 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5d9566f4d5-7vp8h" podUID="d269c326-cc57-4972-a78d-5057f95ada05" containerName="manager" containerID="cri-o://0c8d827832022590ebc2f86157e8c06540e070e396ca57c181da1605b8271b06" gracePeriod=30 Jul 12 08:30:38.969521 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:38.969498 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d9566f4d5-7vp8h" Jul 12 08:30:39.090966 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:39.090941 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qsdcg\" (UniqueName: \"kubernetes.io/projected/d269c326-cc57-4972-a78d-5057f95ada05-kube-api-access-qsdcg\") pod \"d269c326-cc57-4972-a78d-5057f95ada05\" (UID: \"d269c326-cc57-4972-a78d-5057f95ada05\") " Jul 12 08:30:39.091128 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:39.091030 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d269c326-cc57-4972-a78d-5057f95ada05-webhook-cert\") pod \"d269c326-cc57-4972-a78d-5057f95ada05\" (UID: \"d269c326-cc57-4972-a78d-5057f95ada05\") " Jul 12 08:30:39.093094 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:39.093063 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d269c326-cc57-4972-a78d-5057f95ada05-kube-api-access-qsdcg" (OuterVolumeSpecName: "kube-api-access-qsdcg") pod "d269c326-cc57-4972-a78d-5057f95ada05" (UID: "d269c326-cc57-4972-a78d-5057f95ada05"). InnerVolumeSpecName "kube-api-access-qsdcg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:30:39.093094 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:39.093069 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d269c326-cc57-4972-a78d-5057f95ada05-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d269c326-cc57-4972-a78d-5057f95ada05" (UID: "d269c326-cc57-4972-a78d-5057f95ada05"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 08:30:39.191783 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:39.191755 2566 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d269c326-cc57-4972-a78d-5057f95ada05-webhook-cert\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:30:39.191783 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:39.191778 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qsdcg\" (UniqueName: \"kubernetes.io/projected/d269c326-cc57-4972-a78d-5057f95ada05-kube-api-access-qsdcg\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:30:39.713334 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:39.713297 2566 generic.go:356] "Generic (PLEG): container finished" podID="d269c326-cc57-4972-a78d-5057f95ada05" containerID="0c8d827832022590ebc2f86157e8c06540e070e396ca57c181da1605b8271b06" exitCode=0 Jul 12 08:30:39.713334 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:39.713338 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d9566f4d5-7vp8h" Jul 12 08:30:39.713818 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:39.713347 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d9566f4d5-7vp8h" event={"ID":"d269c326-cc57-4972-a78d-5057f95ada05","Type":"ContainerDied","Data":"0c8d827832022590ebc2f86157e8c06540e070e396ca57c181da1605b8271b06"} Jul 12 08:30:39.713818 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:39.713399 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d9566f4d5-7vp8h" event={"ID":"d269c326-cc57-4972-a78d-5057f95ada05","Type":"ContainerDied","Data":"939aeead17f5d8b3e72bfed755f0898e23747e9dc562cce88aa4d7a8b92bb321"} Jul 12 08:30:39.713818 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:39.713427 2566 scope.go:117] "RemoveContainer" containerID="0c8d827832022590ebc2f86157e8c06540e070e396ca57c181da1605b8271b06" Jul 12 08:30:39.722810 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:39.722793 2566 scope.go:117] "RemoveContainer" containerID="0c8d827832022590ebc2f86157e8c06540e070e396ca57c181da1605b8271b06" Jul 12 08:30:39.723033 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:30:39.723018 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c8d827832022590ebc2f86157e8c06540e070e396ca57c181da1605b8271b06\": container with ID starting with 0c8d827832022590ebc2f86157e8c06540e070e396ca57c181da1605b8271b06 not found: ID does not exist" containerID="0c8d827832022590ebc2f86157e8c06540e070e396ca57c181da1605b8271b06" Jul 12 08:30:39.723100 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:39.723039 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c8d827832022590ebc2f86157e8c06540e070e396ca57c181da1605b8271b06"} err="failed to get container status \"0c8d827832022590ebc2f86157e8c06540e070e396ca57c181da1605b8271b06\": rpc error: code = NotFound desc = could not find container \"0c8d827832022590ebc2f86157e8c06540e070e396ca57c181da1605b8271b06\": container with ID starting with 0c8d827832022590ebc2f86157e8c06540e070e396ca57c181da1605b8271b06 not found: ID does not exist" Jul 12 08:30:39.734669 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:39.734649 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5d9566f4d5-7vp8h"] Jul 12 08:30:39.738221 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:39.738197 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5d9566f4d5-7vp8h"] Jul 12 08:30:41.026138 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:41.026105 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d269c326-cc57-4972-a78d-5057f95ada05" path="/var/lib/kubelet/pods/d269c326-cc57-4972-a78d-5057f95ada05/volumes" Jul 12 08:30:59.149101 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:59.149061 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:30:59.625380 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:59.625345 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:30:59.665679 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:59.665652 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-f79c5dcb7-w8c8s"] Jul 12 08:30:59.666008 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:59.665996 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d269c326-cc57-4972-a78d-5057f95ada05" containerName="manager" Jul 12 08:30:59.666051 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:59.666009 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d269c326-cc57-4972-a78d-5057f95ada05" containerName="manager" Jul 12 08:30:59.666088 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:59.666072 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d269c326-cc57-4972-a78d-5057f95ada05" containerName="manager" Jul 12 08:30:59.670645 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:59.670630 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-f79c5dcb7-w8c8s" Jul 12 08:30:59.673451 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:59.673429 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 12 08:30:59.673586 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:59.673429 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8ggw6\"" Jul 12 08:30:59.681222 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:59.681201 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-f79c5dcb7-w8c8s"] Jul 12 08:30:59.740695 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:59.740671 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3af92f3b-e3cc-4207-a49c-1090c5cee314-maas-api-tls\") pod \"maas-api-f79c5dcb7-w8c8s\" (UID: \"3af92f3b-e3cc-4207-a49c-1090c5cee314\") " pod="odh-ai-gateway-infra/maas-api-f79c5dcb7-w8c8s" Jul 12 08:30:59.740821 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:59.740704 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdthm\" (UniqueName: \"kubernetes.io/projected/3af92f3b-e3cc-4207-a49c-1090c5cee314-kube-api-access-pdthm\") pod \"maas-api-f79c5dcb7-w8c8s\" (UID: \"3af92f3b-e3cc-4207-a49c-1090c5cee314\") " pod="odh-ai-gateway-infra/maas-api-f79c5dcb7-w8c8s" Jul 12 08:30:59.740821 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:59.740736 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3af92f3b-e3cc-4207-a49c-1090c5cee314-openshift-service-ca\") pod \"maas-api-f79c5dcb7-w8c8s\" (UID: \"3af92f3b-e3cc-4207-a49c-1090c5cee314\") " pod="odh-ai-gateway-infra/maas-api-f79c5dcb7-w8c8s" Jul 12 08:30:59.818570 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:59.818533 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:30:59.841644 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:59.841621 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3af92f3b-e3cc-4207-a49c-1090c5cee314-openshift-service-ca\") pod \"maas-api-f79c5dcb7-w8c8s\" (UID: \"3af92f3b-e3cc-4207-a49c-1090c5cee314\") " pod="odh-ai-gateway-infra/maas-api-f79c5dcb7-w8c8s" Jul 12 08:30:59.841760 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:59.841687 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3af92f3b-e3cc-4207-a49c-1090c5cee314-maas-api-tls\") pod \"maas-api-f79c5dcb7-w8c8s\" (UID: \"3af92f3b-e3cc-4207-a49c-1090c5cee314\") " pod="odh-ai-gateway-infra/maas-api-f79c5dcb7-w8c8s" Jul 12 08:30:59.841760 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:59.841708 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdthm\" (UniqueName: \"kubernetes.io/projected/3af92f3b-e3cc-4207-a49c-1090c5cee314-kube-api-access-pdthm\") pod \"maas-api-f79c5dcb7-w8c8s\" (UID: \"3af92f3b-e3cc-4207-a49c-1090c5cee314\") " pod="odh-ai-gateway-infra/maas-api-f79c5dcb7-w8c8s" Jul 12 08:30:59.842305 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:59.842287 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3af92f3b-e3cc-4207-a49c-1090c5cee314-openshift-service-ca\") pod \"maas-api-f79c5dcb7-w8c8s\" (UID: \"3af92f3b-e3cc-4207-a49c-1090c5cee314\") " pod="odh-ai-gateway-infra/maas-api-f79c5dcb7-w8c8s" Jul 12 08:30:59.844060 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:59.844040 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3af92f3b-e3cc-4207-a49c-1090c5cee314-maas-api-tls\") pod \"maas-api-f79c5dcb7-w8c8s\" (UID: \"3af92f3b-e3cc-4207-a49c-1090c5cee314\") " pod="odh-ai-gateway-infra/maas-api-f79c5dcb7-w8c8s" Jul 12 08:30:59.850566 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:59.850527 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdthm\" (UniqueName: \"kubernetes.io/projected/3af92f3b-e3cc-4207-a49c-1090c5cee314-kube-api-access-pdthm\") pod \"maas-api-f79c5dcb7-w8c8s\" (UID: \"3af92f3b-e3cc-4207-a49c-1090c5cee314\") " pod="odh-ai-gateway-infra/maas-api-f79c5dcb7-w8c8s" Jul 12 08:30:59.982366 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:30:59.982311 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-f79c5dcb7-w8c8s" Jul 12 08:31:00.122276 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:00.122249 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-f79c5dcb7-w8c8s"] Jul 12 08:31:00.124484 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:31:00.124454 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3af92f3b_e3cc_4207_a49c_1090c5cee314.slice/crio-dff1d2f6ad889dd0a7229dd62fd9d8817f6b89af72b0832953c52d2d444c4176 WatchSource:0}: Error finding container dff1d2f6ad889dd0a7229dd62fd9d8817f6b89af72b0832953c52d2d444c4176: Status 404 returned error can't find the container with id dff1d2f6ad889dd0a7229dd62fd9d8817f6b89af72b0832953c52d2d444c4176 Jul 12 08:31:00.176150 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:00.176118 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:31:00.266761 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:00.266733 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:31:00.267130 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:00.267109 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-r8g9d"] Jul 12 08:31:00.270978 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:00.270962 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-r8g9d" Jul 12 08:31:00.273297 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:00.273278 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 12 08:31:00.280234 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:00.280213 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-r8g9d"] Jul 12 08:31:00.320854 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:00.320829 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-f6sp4"] Jul 12 08:31:00.323596 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:00.323578 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-f6sp4" Jul 12 08:31:00.326459 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:00.326439 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-jlcvs\"" Jul 12 08:31:00.337788 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:00.337763 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-f6sp4"] Jul 12 08:31:00.345257 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:00.345237 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnkxx\" (UniqueName: \"kubernetes.io/projected/f325a226-e2c4-44e0-ab4c-81b481a5427f-kube-api-access-fnkxx\") pod \"payload-pre-processing-7964457696-r8g9d\" (UID: \"f325a226-e2c4-44e0-ab4c-81b481a5427f\") " pod="openshift-ingress/payload-pre-processing-7964457696-r8g9d" Jul 12 08:31:00.345348 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:00.345319 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f325a226-e2c4-44e0-ab4c-81b481a5427f-plugins-config-volume\") pod \"payload-pre-processing-7964457696-r8g9d\" (UID: \"f325a226-e2c4-44e0-ab4c-81b481a5427f\") " pod="openshift-ingress/payload-pre-processing-7964457696-r8g9d" Jul 12 08:31:00.446542 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:00.446506 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f325a226-e2c4-44e0-ab4c-81b481a5427f-plugins-config-volume\") pod \"payload-pre-processing-7964457696-r8g9d\" (UID: \"f325a226-e2c4-44e0-ab4c-81b481a5427f\") " pod="openshift-ingress/payload-pre-processing-7964457696-r8g9d" Jul 12 08:31:00.446762 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:00.446604 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnkxx\" (UniqueName: \"kubernetes.io/projected/f325a226-e2c4-44e0-ab4c-81b481a5427f-kube-api-access-fnkxx\") pod \"payload-pre-processing-7964457696-r8g9d\" (UID: \"f325a226-e2c4-44e0-ab4c-81b481a5427f\") " pod="openshift-ingress/payload-pre-processing-7964457696-r8g9d" Jul 12 08:31:00.446762 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:00.446644 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76d96201-18a0-4e3d-bfa8-3b34e6f52117-plugins-config-volume\") pod \"payload-processing-58f86c7798-f6sp4\" (UID: \"76d96201-18a0-4e3d-bfa8-3b34e6f52117\") " pod="openshift-ingress/payload-processing-58f86c7798-f6sp4" Jul 12 08:31:00.446762 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:00.446668 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqhrf\" (UniqueName: \"kubernetes.io/projected/76d96201-18a0-4e3d-bfa8-3b34e6f52117-kube-api-access-tqhrf\") pod \"payload-processing-58f86c7798-f6sp4\" (UID: \"76d96201-18a0-4e3d-bfa8-3b34e6f52117\") " pod="openshift-ingress/payload-processing-58f86c7798-f6sp4" Jul 12 08:31:00.447498 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:00.447476 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f325a226-e2c4-44e0-ab4c-81b481a5427f-plugins-config-volume\") pod \"payload-pre-processing-7964457696-r8g9d\" (UID: \"f325a226-e2c4-44e0-ab4c-81b481a5427f\") " pod="openshift-ingress/payload-pre-processing-7964457696-r8g9d" Jul 12 08:31:00.456189 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:00.456166 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnkxx\" (UniqueName: \"kubernetes.io/projected/f325a226-e2c4-44e0-ab4c-81b481a5427f-kube-api-access-fnkxx\") pod \"payload-pre-processing-7964457696-r8g9d\" (UID: \"f325a226-e2c4-44e0-ab4c-81b481a5427f\") " pod="openshift-ingress/payload-pre-processing-7964457696-r8g9d" Jul 12 08:31:00.548007 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:00.547941 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76d96201-18a0-4e3d-bfa8-3b34e6f52117-plugins-config-volume\") pod \"payload-processing-58f86c7798-f6sp4\" (UID: \"76d96201-18a0-4e3d-bfa8-3b34e6f52117\") " pod="openshift-ingress/payload-processing-58f86c7798-f6sp4" Jul 12 08:31:00.548007 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:00.547975 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqhrf\" (UniqueName: \"kubernetes.io/projected/76d96201-18a0-4e3d-bfa8-3b34e6f52117-kube-api-access-tqhrf\") pod \"payload-processing-58f86c7798-f6sp4\" (UID: \"76d96201-18a0-4e3d-bfa8-3b34e6f52117\") " pod="openshift-ingress/payload-processing-58f86c7798-f6sp4" Jul 12 08:31:00.548607 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:00.548587 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76d96201-18a0-4e3d-bfa8-3b34e6f52117-plugins-config-volume\") pod \"payload-processing-58f86c7798-f6sp4\" (UID: \"76d96201-18a0-4e3d-bfa8-3b34e6f52117\") " pod="openshift-ingress/payload-processing-58f86c7798-f6sp4" Jul 12 08:31:00.555742 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:00.555724 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqhrf\" (UniqueName: \"kubernetes.io/projected/76d96201-18a0-4e3d-bfa8-3b34e6f52117-kube-api-access-tqhrf\") pod \"payload-processing-58f86c7798-f6sp4\" (UID: \"76d96201-18a0-4e3d-bfa8-3b34e6f52117\") " pod="openshift-ingress/payload-processing-58f86c7798-f6sp4" Jul 12 08:31:00.582581 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:00.582538 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-r8g9d" Jul 12 08:31:00.633152 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:00.633111 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-f6sp4" Jul 12 08:31:00.736293 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:00.736223 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-r8g9d"] Jul 12 08:31:00.742449 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:31:00.742419 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf325a226_e2c4_44e0_ab4c_81b481a5427f.slice/crio-0ff9efc2d1529496023f192b9703eb787c7a1ab53ee8e956a2b5d0113f60ad39 WatchSource:0}: Error finding container 0ff9efc2d1529496023f192b9703eb787c7a1ab53ee8e956a2b5d0113f60ad39: Status 404 returned error can't find the container with id 0ff9efc2d1529496023f192b9703eb787c7a1ab53ee8e956a2b5d0113f60ad39 Jul 12 08:31:00.794065 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:00.794037 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-f6sp4"] Jul 12 08:31:00.795884 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:31:00.795844 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76d96201_18a0_4e3d_bfa8_3b34e6f52117.slice/crio-745399d814d9e06480a6ea197fec6a0074cbac155067e6d5d24e37baa027c8fe WatchSource:0}: Error finding container 745399d814d9e06480a6ea197fec6a0074cbac155067e6d5d24e37baa027c8fe: Status 404 returned error can't find the container with id 745399d814d9e06480a6ea197fec6a0074cbac155067e6d5d24e37baa027c8fe Jul 12 08:31:00.816732 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:00.816687 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-f6sp4" event={"ID":"76d96201-18a0-4e3d-bfa8-3b34e6f52117","Type":"ContainerStarted","Data":"745399d814d9e06480a6ea197fec6a0074cbac155067e6d5d24e37baa027c8fe"} Jul 12 08:31:00.818204 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:00.818175 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-r8g9d" event={"ID":"f325a226-e2c4-44e0-ab4c-81b481a5427f","Type":"ContainerStarted","Data":"0ff9efc2d1529496023f192b9703eb787c7a1ab53ee8e956a2b5d0113f60ad39"} Jul 12 08:31:00.819481 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:00.819456 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-f79c5dcb7-w8c8s" event={"ID":"3af92f3b-e3cc-4207-a49c-1090c5cee314","Type":"ContainerStarted","Data":"dff1d2f6ad889dd0a7229dd62fd9d8817f6b89af72b0832953c52d2d444c4176"} Jul 12 08:31:01.829585 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:01.829440 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-f79c5dcb7-w8c8s" event={"ID":"3af92f3b-e3cc-4207-a49c-1090c5cee314","Type":"ContainerStarted","Data":"dd4f00b193820be38a7284ab1f88a8ab0e179dd1e6aac10f5dfbbe824188f8e0"} Jul 12 08:31:01.830045 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:01.829674 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-f79c5dcb7-w8c8s" Jul 12 08:31:01.849447 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:01.849391 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-f79c5dcb7-w8c8s" podStartSLOduration=1.458242858 podStartE2EDuration="2.849371867s" totalImagesPullingTime="1.391129009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:30:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:31:01.845864954 +0000 UTC m=+585.506660630" watchObservedRunningTime="2026-07-12 08:31:01.849371867 +0000 UTC m=+585.510167530" Jul 12 08:31:02.837158 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:02.837122 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-r8g9d" event={"ID":"f325a226-e2c4-44e0-ab4c-81b481a5427f","Type":"ContainerStarted","Data":"5588a4a65170c6717d65b4cc366eb031f12ea7db305ac5ba5130cd7dfe5ede63"} Jul 12 08:31:02.837626 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:02.837302 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-r8g9d" Jul 12 08:31:02.838617 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:02.838593 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-f6sp4" event={"ID":"76d96201-18a0-4e3d-bfa8-3b34e6f52117","Type":"ContainerStarted","Data":"e9775bd77041a403f91c42975b4f276a8e6547323b429a70e9d77786e4e0dee1"} Jul 12 08:31:02.838760 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:02.838744 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-f6sp4" Jul 12 08:31:02.857818 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:02.857778 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-r8g9d" podStartSLOduration=1.255654758 podStartE2EDuration="2.857765477s" totalImagesPullingTime="1.602110719s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:31:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:31:02.856389907 +0000 UTC m=+586.517185571" watchObservedRunningTime="2026-07-12 08:31:02.857765477 +0000 UTC m=+586.518561137" Jul 12 08:31:02.873714 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:02.873673 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-f6sp4" podStartSLOduration=1.326399189 podStartE2EDuration="2.873662898s" totalImagesPullingTime="1.547263709s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:31:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:31:02.872404828 +0000 UTC m=+586.533200503" watchObservedRunningTime="2026-07-12 08:31:02.873662898 +0000 UTC m=+586.534458558" Jul 12 08:31:07.844927 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:07.844895 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-f79c5dcb7-w8c8s" Jul 12 08:31:10.292159 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:10.292114 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:31:10.387493 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:10.387464 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:31:13.849132 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:13.849103 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-r8g9d" Jul 12 08:31:13.849529 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:13.849173 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-f6sp4" Jul 12 08:31:16.760721 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:16.760691 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-k2ltl_bc3f8e97-c9ab-4370-b437-8a47514c869c/console-operator/1.log" Jul 12 08:31:16.761450 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:16.761430 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-k2ltl_bc3f8e97-c9ab-4370-b437-8a47514c869c/console-operator/1.log" Jul 12 08:31:16.770119 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:16.770094 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lfbsq_abe468eb-f6d3-4af4-bf5e-a3c029a404ab/ovn-acl-logging/0.log" Jul 12 08:31:16.770119 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:16.770117 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lfbsq_abe468eb-f6d3-4af4-bf5e-a3c029a404ab/ovn-acl-logging/0.log" Jul 12 08:31:33.328383 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:33.328343 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:31:34.618262 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:34.618225 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:31:49.628017 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:49.627982 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:31:56.201577 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:56.201531 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5mzk6"] Jul 12 08:31:56.204991 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:56.204974 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5mzk6" Jul 12 08:31:56.207304 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:56.207280 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 12 08:31:56.208067 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:56.208046 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 12 08:31:56.208172 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:56.208154 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-bf7sq\"" Jul 12 08:31:56.208294 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:56.208280 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 12 08:31:56.214083 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:56.214058 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5mzk6"] Jul 12 08:31:56.234876 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:56.234849 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:31:56.276139 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:56.276107 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1965a474-d47b-4dcb-af7d-f6e332f4c45a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5mzk6\" (UID: \"1965a474-d47b-4dcb-af7d-f6e332f4c45a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5mzk6" Jul 12 08:31:56.276227 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:56.276180 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1965a474-d47b-4dcb-af7d-f6e332f4c45a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5mzk6\" (UID: \"1965a474-d47b-4dcb-af7d-f6e332f4c45a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5mzk6" Jul 12 08:31:56.276275 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:56.276223 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1965a474-d47b-4dcb-af7d-f6e332f4c45a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5mzk6\" (UID: \"1965a474-d47b-4dcb-af7d-f6e332f4c45a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5mzk6" Jul 12 08:31:56.276275 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:56.276247 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27qjg\" (UniqueName: \"kubernetes.io/projected/1965a474-d47b-4dcb-af7d-f6e332f4c45a-kube-api-access-27qjg\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5mzk6\" (UID: \"1965a474-d47b-4dcb-af7d-f6e332f4c45a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5mzk6" Jul 12 08:31:56.276353 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:56.276293 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1965a474-d47b-4dcb-af7d-f6e332f4c45a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5mzk6\" (UID: \"1965a474-d47b-4dcb-af7d-f6e332f4c45a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5mzk6" Jul 12 08:31:56.276353 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:56.276334 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1965a474-d47b-4dcb-af7d-f6e332f4c45a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5mzk6\" (UID: \"1965a474-d47b-4dcb-af7d-f6e332f4c45a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5mzk6" Jul 12 08:31:56.377206 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:56.377183 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1965a474-d47b-4dcb-af7d-f6e332f4c45a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5mzk6\" (UID: \"1965a474-d47b-4dcb-af7d-f6e332f4c45a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5mzk6" Jul 12 08:31:56.377315 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:56.377214 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27qjg\" (UniqueName: \"kubernetes.io/projected/1965a474-d47b-4dcb-af7d-f6e332f4c45a-kube-api-access-27qjg\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5mzk6\" (UID: \"1965a474-d47b-4dcb-af7d-f6e332f4c45a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5mzk6" Jul 12 08:31:56.377315 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:56.377243 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1965a474-d47b-4dcb-af7d-f6e332f4c45a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5mzk6\" (UID: \"1965a474-d47b-4dcb-af7d-f6e332f4c45a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5mzk6" Jul 12 08:31:56.377315 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:56.377268 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1965a474-d47b-4dcb-af7d-f6e332f4c45a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5mzk6\" (UID: \"1965a474-d47b-4dcb-af7d-f6e332f4c45a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5mzk6" Jul 12 08:31:56.377441 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:56.377331 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1965a474-d47b-4dcb-af7d-f6e332f4c45a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5mzk6\" (UID: \"1965a474-d47b-4dcb-af7d-f6e332f4c45a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5mzk6" Jul 12 08:31:56.377441 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:56.377373 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1965a474-d47b-4dcb-af7d-f6e332f4c45a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5mzk6\" (UID: \"1965a474-d47b-4dcb-af7d-f6e332f4c45a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5mzk6" Jul 12 08:31:56.377664 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:56.377643 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1965a474-d47b-4dcb-af7d-f6e332f4c45a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5mzk6\" (UID: \"1965a474-d47b-4dcb-af7d-f6e332f4c45a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5mzk6" Jul 12 08:31:56.377705 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:56.377678 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1965a474-d47b-4dcb-af7d-f6e332f4c45a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5mzk6\" (UID: \"1965a474-d47b-4dcb-af7d-f6e332f4c45a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5mzk6" Jul 12 08:31:56.377750 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:56.377705 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1965a474-d47b-4dcb-af7d-f6e332f4c45a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5mzk6\" (UID: \"1965a474-d47b-4dcb-af7d-f6e332f4c45a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5mzk6" Jul 12 08:31:56.379595 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:56.379565 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1965a474-d47b-4dcb-af7d-f6e332f4c45a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5mzk6\" (UID: \"1965a474-d47b-4dcb-af7d-f6e332f4c45a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5mzk6" Jul 12 08:31:56.379904 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:56.379883 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1965a474-d47b-4dcb-af7d-f6e332f4c45a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5mzk6\" (UID: \"1965a474-d47b-4dcb-af7d-f6e332f4c45a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5mzk6" Jul 12 08:31:56.384140 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:56.384117 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27qjg\" (UniqueName: \"kubernetes.io/projected/1965a474-d47b-4dcb-af7d-f6e332f4c45a-kube-api-access-27qjg\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5mzk6\" (UID: \"1965a474-d47b-4dcb-af7d-f6e332f4c45a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5mzk6" Jul 12 08:31:56.517290 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:56.517258 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5mzk6" Jul 12 08:31:56.648153 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:56.648130 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5mzk6"] Jul 12 08:31:56.650143 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:31:56.650119 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1965a474_d47b_4dcb_af7d_f6e332f4c45a.slice/crio-95a72c0a7396009078acad9d74226f670ebde2dfe8ff66b61adf868b61fdcb88 WatchSource:0}: Error finding container 95a72c0a7396009078acad9d74226f670ebde2dfe8ff66b61adf868b61fdcb88: Status 404 returned error can't find the container with id 95a72c0a7396009078acad9d74226f670ebde2dfe8ff66b61adf868b61fdcb88 Jul 12 08:31:56.651471 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:56.651449 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 08:31:57.089114 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:57.089076 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5mzk6" event={"ID":"1965a474-d47b-4dcb-af7d-f6e332f4c45a","Type":"ContainerStarted","Data":"95a72c0a7396009078acad9d74226f670ebde2dfe8ff66b61adf868b61fdcb88"} Jul 12 08:31:58.096406 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:58.096369 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5mzk6" event={"ID":"1965a474-d47b-4dcb-af7d-f6e332f4c45a","Type":"ContainerStarted","Data":"0ad469573fd291083d16d570d7a60623ddbd133b28a049e697b55e72d06ac70e"} Jul 12 08:31:58.114065 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:58.114009 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5mzk6" podStartSLOduration=1.044365139 podStartE2EDuration="2.113994013s" totalImagesPullingTime="1.069628874s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:31:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:31:58.112761695 +0000 UTC m=+641.773557358" watchObservedRunningTime="2026-07-12 08:31:58.113994013 +0000 UTC m=+641.774789676" Jul 12 08:31:58.601767 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:58.601732 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:31:58.653031 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:31:58.652982 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:32:05.498728 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:32:05.498693 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:32:06.517876 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:32:06.517844 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5mzk6" Jul 12 08:32:06.518328 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:32:06.517907 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5mzk6" Jul 12 08:32:06.530382 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:32:06.530360 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5mzk6" Jul 12 08:32:07.149680 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:32:07.149653 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5mzk6" Jul 12 08:32:11.641196 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:32:11.641163 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:32:21.823763 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:32:21.823714 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:04.469672 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:04.469635 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:04.521235 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:04.521205 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:32.672716 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:32.672636 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-d67dd4f5c-748fl"] Jul 12 08:33:32.676060 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:32.676037 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d67dd4f5c-748fl" Jul 12 08:33:32.683497 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:32.683469 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d67dd4f5c-748fl"] Jul 12 08:33:32.783489 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:32.783462 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4692l\" (UniqueName: \"kubernetes.io/projected/61815d5f-206f-43d9-abde-cee35b61f923-kube-api-access-4692l\") pod \"authorino-d67dd4f5c-748fl\" (UID: \"61815d5f-206f-43d9-abde-cee35b61f923\") " pod="kuadrant-system/authorino-d67dd4f5c-748fl" Jul 12 08:33:32.783632 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:32.783508 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/61815d5f-206f-43d9-abde-cee35b61f923-tls-cert\") pod \"authorino-d67dd4f5c-748fl\" (UID: \"61815d5f-206f-43d9-abde-cee35b61f923\") " pod="kuadrant-system/authorino-d67dd4f5c-748fl" Jul 12 08:33:32.884539 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:32.884508 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/61815d5f-206f-43d9-abde-cee35b61f923-tls-cert\") pod \"authorino-d67dd4f5c-748fl\" (UID: \"61815d5f-206f-43d9-abde-cee35b61f923\") " pod="kuadrant-system/authorino-d67dd4f5c-748fl" Jul 12 08:33:32.884670 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:32.884633 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4692l\" (UniqueName: \"kubernetes.io/projected/61815d5f-206f-43d9-abde-cee35b61f923-kube-api-access-4692l\") pod \"authorino-d67dd4f5c-748fl\" (UID: \"61815d5f-206f-43d9-abde-cee35b61f923\") " pod="kuadrant-system/authorino-d67dd4f5c-748fl" Jul 12 08:33:32.887009 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:32.886990 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/61815d5f-206f-43d9-abde-cee35b61f923-tls-cert\") pod \"authorino-d67dd4f5c-748fl\" (UID: \"61815d5f-206f-43d9-abde-cee35b61f923\") " pod="kuadrant-system/authorino-d67dd4f5c-748fl" Jul 12 08:33:32.895144 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:32.895125 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4692l\" (UniqueName: \"kubernetes.io/projected/61815d5f-206f-43d9-abde-cee35b61f923-kube-api-access-4692l\") pod \"authorino-d67dd4f5c-748fl\" (UID: \"61815d5f-206f-43d9-abde-cee35b61f923\") " pod="kuadrant-system/authorino-d67dd4f5c-748fl" Jul 12 08:33:32.986170 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:32.986147 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d67dd4f5c-748fl" Jul 12 08:33:32.999113 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:32.999078 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:33.130302 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:33.130281 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d67dd4f5c-748fl"] Jul 12 08:33:33.132516 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:33:33.132485 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod61815d5f_206f_43d9_abde_cee35b61f923.slice/crio-486760d9b288addd8e5923ac77995f4ebd0aff20d7de3642070974db62b2db3a WatchSource:0}: Error finding container 486760d9b288addd8e5923ac77995f4ebd0aff20d7de3642070974db62b2db3a: Status 404 returned error can't find the container with id 486760d9b288addd8e5923ac77995f4ebd0aff20d7de3642070974db62b2db3a Jul 12 08:33:33.133067 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:33.133042 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:33.545667 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:33.545641 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d67dd4f5c-748fl" event={"ID":"61815d5f-206f-43d9-abde-cee35b61f923","Type":"ContainerStarted","Data":"486760d9b288addd8e5923ac77995f4ebd0aff20d7de3642070974db62b2db3a"} Jul 12 08:33:34.162361 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:34.162324 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:34.259943 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:34.259892 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:33:34.553428 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:34.553391 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d67dd4f5c-748fl" event={"ID":"61815d5f-206f-43d9-abde-cee35b61f923","Type":"ContainerStarted","Data":"2ab71028f6cc8ab541b39ea676e912ca6b5e37094d5c66be0458045abe874464"} Jul 12 08:33:34.572581 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:34.572512 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-d67dd4f5c-748fl" podStartSLOduration=2.206465588 podStartE2EDuration="2.572493407s" totalImagesPullingTime="366.027819ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:33:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:33:34.57140412 +0000 UTC m=+738.232199823" watchObservedRunningTime="2026-07-12 08:33:34.572493407 +0000 UTC m=+738.233289068" Jul 12 08:33:34.597478 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:34.597442 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-864cf568c6-2tts9"] Jul 12 08:33:34.597681 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:34.597657 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-864cf568c6-2tts9" podUID="4cfc9b31-09d1-4965-93e9-8aeef9587da0" containerName="authorino" containerID="cri-o://2e8225c6870d882f2ff59b0d8b42a53dde6ab1dbbcc53fa2b22071139d6841de" gracePeriod=30 Jul 12 08:33:34.850315 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:34.850285 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-864cf568c6-2tts9" Jul 12 08:33:35.006592 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:35.006531 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4cfc9b31-09d1-4965-93e9-8aeef9587da0-tls-cert\") pod \"4cfc9b31-09d1-4965-93e9-8aeef9587da0\" (UID: \"4cfc9b31-09d1-4965-93e9-8aeef9587da0\") " Jul 12 08:33:35.006592 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:35.006589 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w9q25\" (UniqueName: \"kubernetes.io/projected/4cfc9b31-09d1-4965-93e9-8aeef9587da0-kube-api-access-w9q25\") pod \"4cfc9b31-09d1-4965-93e9-8aeef9587da0\" (UID: \"4cfc9b31-09d1-4965-93e9-8aeef9587da0\") " Jul 12 08:33:35.009091 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:35.009055 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cfc9b31-09d1-4965-93e9-8aeef9587da0-kube-api-access-w9q25" (OuterVolumeSpecName: "kube-api-access-w9q25") pod "4cfc9b31-09d1-4965-93e9-8aeef9587da0" (UID: "4cfc9b31-09d1-4965-93e9-8aeef9587da0"). InnerVolumeSpecName "kube-api-access-w9q25". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:33:35.019568 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:35.019514 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4cfc9b31-09d1-4965-93e9-8aeef9587da0-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "4cfc9b31-09d1-4965-93e9-8aeef9587da0" (UID: "4cfc9b31-09d1-4965-93e9-8aeef9587da0"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 08:33:35.107841 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:35.107757 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9q25\" (UniqueName: \"kubernetes.io/projected/4cfc9b31-09d1-4965-93e9-8aeef9587da0-kube-api-access-w9q25\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:33:35.107841 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:35.107794 2566 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4cfc9b31-09d1-4965-93e9-8aeef9587da0-tls-cert\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:33:35.559109 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:35.559079 2566 generic.go:356] "Generic (PLEG): container finished" podID="4cfc9b31-09d1-4965-93e9-8aeef9587da0" containerID="2e8225c6870d882f2ff59b0d8b42a53dde6ab1dbbcc53fa2b22071139d6841de" exitCode=0 Jul 12 08:33:35.559496 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:35.559122 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-864cf568c6-2tts9" Jul 12 08:33:35.559496 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:35.559170 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-864cf568c6-2tts9" event={"ID":"4cfc9b31-09d1-4965-93e9-8aeef9587da0","Type":"ContainerDied","Data":"2e8225c6870d882f2ff59b0d8b42a53dde6ab1dbbcc53fa2b22071139d6841de"} Jul 12 08:33:35.559496 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:35.559205 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-864cf568c6-2tts9" event={"ID":"4cfc9b31-09d1-4965-93e9-8aeef9587da0","Type":"ContainerDied","Data":"beb398cd6ff00b0f37f859a94d4f0bfd5b1ba0e63504855dee84428cd4d5cd83"} Jul 12 08:33:35.559496 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:35.559222 2566 scope.go:117] "RemoveContainer" containerID="2e8225c6870d882f2ff59b0d8b42a53dde6ab1dbbcc53fa2b22071139d6841de" Jul 12 08:33:35.567901 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:35.567882 2566 scope.go:117] "RemoveContainer" containerID="2e8225c6870d882f2ff59b0d8b42a53dde6ab1dbbcc53fa2b22071139d6841de" Jul 12 08:33:35.568183 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:33:35.568161 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e8225c6870d882f2ff59b0d8b42a53dde6ab1dbbcc53fa2b22071139d6841de\": container with ID starting with 2e8225c6870d882f2ff59b0d8b42a53dde6ab1dbbcc53fa2b22071139d6841de not found: ID does not exist" containerID="2e8225c6870d882f2ff59b0d8b42a53dde6ab1dbbcc53fa2b22071139d6841de" Jul 12 08:33:35.568273 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:35.568194 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e8225c6870d882f2ff59b0d8b42a53dde6ab1dbbcc53fa2b22071139d6841de"} err="failed to get container status \"2e8225c6870d882f2ff59b0d8b42a53dde6ab1dbbcc53fa2b22071139d6841de\": rpc error: code = NotFound desc = could not find container \"2e8225c6870d882f2ff59b0d8b42a53dde6ab1dbbcc53fa2b22071139d6841de\": container with ID starting with 2e8225c6870d882f2ff59b0d8b42a53dde6ab1dbbcc53fa2b22071139d6841de not found: ID does not exist" Jul 12 08:33:35.576597 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:35.576573 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-864cf568c6-2tts9"] Jul 12 08:33:35.585579 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:35.581813 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-864cf568c6-2tts9"] Jul 12 08:33:37.026387 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:33:37.026353 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4cfc9b31-09d1-4965-93e9-8aeef9587da0" path="/var/lib/kubelet/pods/4cfc9b31-09d1-4965-93e9-8aeef9587da0/volumes" Jul 12 08:34:50.671006 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:34:50.670972 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:34:50.718434 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:34:50.718398 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:05.309953 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:05.309907 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:05.416883 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:05.416850 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-69948768-686nc"] Jul 12 08:35:05.417071 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:05.417050 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-69948768-686nc" podUID="e7a1cfe8-651a-413b-8b8b-5517fa99dacb" containerName="manager" containerID="cri-o://73d0b3bfe45935001c0ec475e929d540efd6a9fcbeb2f93164e74bb63092f7a6" gracePeriod=30 Jul 12 08:35:05.673032 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:05.673002 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69948768-686nc" Jul 12 08:35:05.788242 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:05.788208 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6z585\" (UniqueName: \"kubernetes.io/projected/e7a1cfe8-651a-413b-8b8b-5517fa99dacb-kube-api-access-6z585\") pod \"e7a1cfe8-651a-413b-8b8b-5517fa99dacb\" (UID: \"e7a1cfe8-651a-413b-8b8b-5517fa99dacb\") " Jul 12 08:35:05.788445 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:05.788289 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7a1cfe8-651a-413b-8b8b-5517fa99dacb-webhook-cert\") pod \"e7a1cfe8-651a-413b-8b8b-5517fa99dacb\" (UID: \"e7a1cfe8-651a-413b-8b8b-5517fa99dacb\") " Jul 12 08:35:05.790402 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:05.790373 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7a1cfe8-651a-413b-8b8b-5517fa99dacb-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "e7a1cfe8-651a-413b-8b8b-5517fa99dacb" (UID: "e7a1cfe8-651a-413b-8b8b-5517fa99dacb"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 08:35:05.790749 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:05.790524 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7a1cfe8-651a-413b-8b8b-5517fa99dacb-kube-api-access-6z585" (OuterVolumeSpecName: "kube-api-access-6z585") pod "e7a1cfe8-651a-413b-8b8b-5517fa99dacb" (UID: "e7a1cfe8-651a-413b-8b8b-5517fa99dacb"). InnerVolumeSpecName "kube-api-access-6z585". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:35:05.889912 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:05.889829 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6z585\" (UniqueName: \"kubernetes.io/projected/e7a1cfe8-651a-413b-8b8b-5517fa99dacb-kube-api-access-6z585\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:35:05.889912 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:05.889856 2566 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7a1cfe8-651a-413b-8b8b-5517fa99dacb-webhook-cert\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:35:06.002962 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:06.002925 2566 generic.go:356] "Generic (PLEG): container finished" podID="e7a1cfe8-651a-413b-8b8b-5517fa99dacb" containerID="73d0b3bfe45935001c0ec475e929d540efd6a9fcbeb2f93164e74bb63092f7a6" exitCode=0 Jul 12 08:35:06.003119 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:06.002972 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69948768-686nc" Jul 12 08:35:06.003119 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:06.003011 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69948768-686nc" event={"ID":"e7a1cfe8-651a-413b-8b8b-5517fa99dacb","Type":"ContainerDied","Data":"73d0b3bfe45935001c0ec475e929d540efd6a9fcbeb2f93164e74bb63092f7a6"} Jul 12 08:35:06.003119 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:06.003063 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69948768-686nc" event={"ID":"e7a1cfe8-651a-413b-8b8b-5517fa99dacb","Type":"ContainerDied","Data":"3a928249655617a258968a7d9763a72173d3afcac47b06f5f4dfdf3fe1af0736"} Jul 12 08:35:06.003119 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:06.003085 2566 scope.go:117] "RemoveContainer" containerID="73d0b3bfe45935001c0ec475e929d540efd6a9fcbeb2f93164e74bb63092f7a6" Jul 12 08:35:06.012375 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:06.012358 2566 scope.go:117] "RemoveContainer" containerID="73d0b3bfe45935001c0ec475e929d540efd6a9fcbeb2f93164e74bb63092f7a6" Jul 12 08:35:06.012650 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:35:06.012630 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73d0b3bfe45935001c0ec475e929d540efd6a9fcbeb2f93164e74bb63092f7a6\": container with ID starting with 73d0b3bfe45935001c0ec475e929d540efd6a9fcbeb2f93164e74bb63092f7a6 not found: ID does not exist" containerID="73d0b3bfe45935001c0ec475e929d540efd6a9fcbeb2f93164e74bb63092f7a6" Jul 12 08:35:06.012707 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:06.012659 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73d0b3bfe45935001c0ec475e929d540efd6a9fcbeb2f93164e74bb63092f7a6"} err="failed to get container status \"73d0b3bfe45935001c0ec475e929d540efd6a9fcbeb2f93164e74bb63092f7a6\": rpc error: code = NotFound desc = could not find container \"73d0b3bfe45935001c0ec475e929d540efd6a9fcbeb2f93164e74bb63092f7a6\": container with ID starting with 73d0b3bfe45935001c0ec475e929d540efd6a9fcbeb2f93164e74bb63092f7a6 not found: ID does not exist" Jul 12 08:35:06.023166 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:06.023136 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-69948768-686nc"] Jul 12 08:35:06.024845 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:06.024824 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-69948768-686nc"] Jul 12 08:35:06.652371 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:06.652335 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:06.771595 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:06.771536 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-69948768-xzlj6"] Jul 12 08:35:06.772035 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:06.772018 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cfc9b31-09d1-4965-93e9-8aeef9587da0" containerName="authorino" Jul 12 08:35:06.772126 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:06.772038 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cfc9b31-09d1-4965-93e9-8aeef9587da0" containerName="authorino" Jul 12 08:35:06.772126 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:06.772067 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7a1cfe8-651a-413b-8b8b-5517fa99dacb" containerName="manager" Jul 12 08:35:06.772126 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:06.772076 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7a1cfe8-651a-413b-8b8b-5517fa99dacb" containerName="manager" Jul 12 08:35:06.772281 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:06.772181 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7a1cfe8-651a-413b-8b8b-5517fa99dacb" containerName="manager" Jul 12 08:35:06.772281 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:06.772193 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cfc9b31-09d1-4965-93e9-8aeef9587da0" containerName="authorino" Jul 12 08:35:06.776528 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:06.776506 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69948768-xzlj6" Jul 12 08:35:06.778852 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:06.778834 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 12 08:35:06.779725 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:06.779707 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 12 08:35:06.779799 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:06.779729 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-hq6z8\"" Jul 12 08:35:06.783002 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:06.782976 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-69948768-xzlj6"] Jul 12 08:35:06.898055 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:06.898021 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/745fa8c6-2e93-43bf-b960-97c09b42ced2-webhook-cert\") pod \"maas-controller-69948768-xzlj6\" (UID: \"745fa8c6-2e93-43bf-b960-97c09b42ced2\") " pod="opendatahub/maas-controller-69948768-xzlj6" Jul 12 08:35:06.898234 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:06.898098 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv4d2\" (UniqueName: \"kubernetes.io/projected/745fa8c6-2e93-43bf-b960-97c09b42ced2-kube-api-access-tv4d2\") pod \"maas-controller-69948768-xzlj6\" (UID: \"745fa8c6-2e93-43bf-b960-97c09b42ced2\") " pod="opendatahub/maas-controller-69948768-xzlj6" Jul 12 08:35:06.998720 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:06.998686 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/745fa8c6-2e93-43bf-b960-97c09b42ced2-webhook-cert\") pod \"maas-controller-69948768-xzlj6\" (UID: \"745fa8c6-2e93-43bf-b960-97c09b42ced2\") " pod="opendatahub/maas-controller-69948768-xzlj6" Jul 12 08:35:06.998905 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:06.998839 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv4d2\" (UniqueName: \"kubernetes.io/projected/745fa8c6-2e93-43bf-b960-97c09b42ced2-kube-api-access-tv4d2\") pod \"maas-controller-69948768-xzlj6\" (UID: \"745fa8c6-2e93-43bf-b960-97c09b42ced2\") " pod="opendatahub/maas-controller-69948768-xzlj6" Jul 12 08:35:07.001217 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:07.001189 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/745fa8c6-2e93-43bf-b960-97c09b42ced2-webhook-cert\") pod \"maas-controller-69948768-xzlj6\" (UID: \"745fa8c6-2e93-43bf-b960-97c09b42ced2\") " pod="opendatahub/maas-controller-69948768-xzlj6" Jul 12 08:35:07.006036 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:07.006013 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv4d2\" (UniqueName: \"kubernetes.io/projected/745fa8c6-2e93-43bf-b960-97c09b42ced2-kube-api-access-tv4d2\") pod \"maas-controller-69948768-xzlj6\" (UID: \"745fa8c6-2e93-43bf-b960-97c09b42ced2\") " pod="opendatahub/maas-controller-69948768-xzlj6" Jul 12 08:35:07.034833 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:07.034799 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7a1cfe8-651a-413b-8b8b-5517fa99dacb" path="/var/lib/kubelet/pods/e7a1cfe8-651a-413b-8b8b-5517fa99dacb/volumes" Jul 12 08:35:07.087984 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:07.087945 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69948768-xzlj6" Jul 12 08:35:07.221088 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:07.221062 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-69948768-xzlj6"] Jul 12 08:35:07.223731 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:35:07.223702 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod745fa8c6_2e93_43bf_b960_97c09b42ced2.slice/crio-ea5e29acf65be6e08ed0ef689ffc2cb5e0aea1ce6ab61522344d022c1fbf0338 WatchSource:0}: Error finding container ea5e29acf65be6e08ed0ef689ffc2cb5e0aea1ce6ab61522344d022c1fbf0338: Status 404 returned error can't find the container with id ea5e29acf65be6e08ed0ef689ffc2cb5e0aea1ce6ab61522344d022c1fbf0338 Jul 12 08:35:08.015827 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:08.015795 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69948768-xzlj6" event={"ID":"745fa8c6-2e93-43bf-b960-97c09b42ced2","Type":"ContainerStarted","Data":"34225550441b3066e5a0c9127444c4fb19bc995fd67617669baa89728ff1360b"} Jul 12 08:35:08.016187 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:08.015846 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69948768-xzlj6" event={"ID":"745fa8c6-2e93-43bf-b960-97c09b42ced2","Type":"ContainerStarted","Data":"ea5e29acf65be6e08ed0ef689ffc2cb5e0aea1ce6ab61522344d022c1fbf0338"} Jul 12 08:35:08.016187 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:08.015937 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-69948768-xzlj6" Jul 12 08:35:08.030518 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:08.030474 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-69948768-xzlj6" podStartSLOduration=1.58212891 podStartE2EDuration="2.030459723s" totalImagesPullingTime="448.330813ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:35:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:35:08.029292425 +0000 UTC m=+831.690088106" watchObservedRunningTime="2026-07-12 08:35:08.030459723 +0000 UTC m=+831.691255419" Jul 12 08:35:19.027482 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:19.027454 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-69948768-xzlj6" Jul 12 08:35:28.955006 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:28.954963 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:40.021898 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:40.021860 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:40.122982 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:40.122949 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:40.233017 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:40.232985 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:45.945846 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:45.945812 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:46.054313 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:46.054266 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:46.146222 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:46.146183 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:46.289989 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:46.289955 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:47.273319 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:47.273283 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:51.815933 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:51.815898 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:35:51.875209 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:35:51.875174 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:36:16.792636 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:36:16.792605 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-k2ltl_bc3f8e97-c9ab-4370-b437-8a47514c869c/console-operator/1.log" Jul 12 08:36:16.794239 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:36:16.794218 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-k2ltl_bc3f8e97-c9ab-4370-b437-8a47514c869c/console-operator/1.log" Jul 12 08:36:16.798388 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:36:16.798373 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lfbsq_abe468eb-f6d3-4af4-bf5e-a3c029a404ab/ovn-acl-logging/0.log" Jul 12 08:36:16.799711 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:36:16.799682 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lfbsq_abe468eb-f6d3-4af4-bf5e-a3c029a404ab/ovn-acl-logging/0.log" Jul 12 08:37:48.971960 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:37:48.971929 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:38:39.659234 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:38:39.659198 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:38:39.760027 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:38:39.759995 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:41:16.823413 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:41:16.823381 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-k2ltl_bc3f8e97-c9ab-4370-b437-8a47514c869c/console-operator/1.log" Jul 12 08:41:16.827198 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:41:16.827174 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-k2ltl_bc3f8e97-c9ab-4370-b437-8a47514c869c/console-operator/1.log" Jul 12 08:41:16.829673 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:41:16.829648 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lfbsq_abe468eb-f6d3-4af4-bf5e-a3c029a404ab/ovn-acl-logging/0.log" Jul 12 08:41:16.832584 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:41:16.832567 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lfbsq_abe468eb-f6d3-4af4-bf5e-a3c029a404ab/ovn-acl-logging/0.log" Jul 12 08:44:16.864523 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:44:16.864493 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:45:00.140797 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:45:00.140761 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29730765-8sfrc"] Jul 12 08:45:00.144060 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:45:00.144042 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730765-8sfrc" Jul 12 08:45:00.158919 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:45:00.158894 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29730765-8sfrc"] Jul 12 08:45:00.224159 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:45:00.224131 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hwmd\" (UniqueName: \"kubernetes.io/projected/a5773191-0b52-4787-a154-10637350936b-kube-api-access-6hwmd\") pod \"maas-api-key-cleanup-29730765-8sfrc\" (UID: \"a5773191-0b52-4787-a154-10637350936b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730765-8sfrc" Jul 12 08:45:00.325389 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:45:00.325352 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hwmd\" (UniqueName: \"kubernetes.io/projected/a5773191-0b52-4787-a154-10637350936b-kube-api-access-6hwmd\") pod \"maas-api-key-cleanup-29730765-8sfrc\" (UID: \"a5773191-0b52-4787-a154-10637350936b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730765-8sfrc" Jul 12 08:45:00.332892 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:45:00.332863 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hwmd\" (UniqueName: \"kubernetes.io/projected/a5773191-0b52-4787-a154-10637350936b-kube-api-access-6hwmd\") pod \"maas-api-key-cleanup-29730765-8sfrc\" (UID: \"a5773191-0b52-4787-a154-10637350936b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730765-8sfrc" Jul 12 08:45:00.454162 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:45:00.454083 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730765-8sfrc" Jul 12 08:45:00.581719 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:45:00.581693 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29730765-8sfrc"] Jul 12 08:45:00.583542 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:45:00.583517 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5773191_0b52_4787_a154_10637350936b.slice/crio-facc807f698c70e6e6ab5a50e2838d0626ccf94d07374cf48a9af4aa0f1f6f3d WatchSource:0}: Error finding container facc807f698c70e6e6ab5a50e2838d0626ccf94d07374cf48a9af4aa0f1f6f3d: Status 404 returned error can't find the container with id facc807f698c70e6e6ab5a50e2838d0626ccf94d07374cf48a9af4aa0f1f6f3d Jul 12 08:45:00.585344 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:45:00.585330 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 08:45:00.829341 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:45:00.829306 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730765-8sfrc" event={"ID":"a5773191-0b52-4787-a154-10637350936b","Type":"ContainerStarted","Data":"facc807f698c70e6e6ab5a50e2838d0626ccf94d07374cf48a9af4aa0f1f6f3d"} Jul 12 08:45:06.862680 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:45:06.862649 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730765-8sfrc" event={"ID":"a5773191-0b52-4787-a154-10637350936b","Type":"ContainerStarted","Data":"f32d9de97e6c6e0783d8c50219edfd436cf96d1c86a46739aa050802dec7aa2b"} Jul 12 08:45:06.878751 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:45:06.878701 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730765-8sfrc" podStartSLOduration=1.773065036 podStartE2EDuration="6.87868598s" totalImagesPullingTime="5.105620944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:45:06.877316981 +0000 UTC m=+1430.538112677" watchObservedRunningTime="2026-07-12 08:45:06.87868598 +0000 UTC m=+1430.539481656" Jul 12 08:46:00.009829 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:00.009789 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29730765-8sfrc"] Jul 12 08:46:00.010339 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:00.010013 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730765-8sfrc" podUID="a5773191-0b52-4787-a154-10637350936b" containerName="cleanup" containerID="cri-o://f32d9de97e6c6e0783d8c50219edfd436cf96d1c86a46739aa050802dec7aa2b" gracePeriod=30 Jul 12 08:46:16.855294 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:16.855261 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-k2ltl_bc3f8e97-c9ab-4370-b437-8a47514c869c/console-operator/1.log" Jul 12 08:46:16.859534 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:16.859512 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-k2ltl_bc3f8e97-c9ab-4370-b437-8a47514c869c/console-operator/1.log" Jul 12 08:46:16.861588 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:16.861568 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lfbsq_abe468eb-f6d3-4af4-bf5e-a3c029a404ab/ovn-acl-logging/0.log" Jul 12 08:46:16.865449 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:16.865429 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lfbsq_abe468eb-f6d3-4af4-bf5e-a3c029a404ab/ovn-acl-logging/0.log" Jul 12 08:46:30.203094 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:30.203068 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730765-8sfrc" Jul 12 08:46:30.260023 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:30.259986 2566 generic.go:356] "Generic (PLEG): container finished" podID="a5773191-0b52-4787-a154-10637350936b" containerID="f32d9de97e6c6e0783d8c50219edfd436cf96d1c86a46739aa050802dec7aa2b" exitCode=137 Jul 12 08:46:30.260184 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:30.260028 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730765-8sfrc" Jul 12 08:46:30.260184 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:30.260030 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730765-8sfrc" event={"ID":"a5773191-0b52-4787-a154-10637350936b","Type":"ContainerDied","Data":"f32d9de97e6c6e0783d8c50219edfd436cf96d1c86a46739aa050802dec7aa2b"} Jul 12 08:46:30.260184 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:30.260138 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730765-8sfrc" event={"ID":"a5773191-0b52-4787-a154-10637350936b","Type":"ContainerDied","Data":"facc807f698c70e6e6ab5a50e2838d0626ccf94d07374cf48a9af4aa0f1f6f3d"} Jul 12 08:46:30.260184 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:30.260165 2566 scope.go:117] "RemoveContainer" containerID="f32d9de97e6c6e0783d8c50219edfd436cf96d1c86a46739aa050802dec7aa2b" Jul 12 08:46:30.268368 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:30.268349 2566 scope.go:117] "RemoveContainer" containerID="f32d9de97e6c6e0783d8c50219edfd436cf96d1c86a46739aa050802dec7aa2b" Jul 12 08:46:30.268672 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:46:30.268654 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f32d9de97e6c6e0783d8c50219edfd436cf96d1c86a46739aa050802dec7aa2b\": container with ID starting with f32d9de97e6c6e0783d8c50219edfd436cf96d1c86a46739aa050802dec7aa2b not found: ID does not exist" containerID="f32d9de97e6c6e0783d8c50219edfd436cf96d1c86a46739aa050802dec7aa2b" Jul 12 08:46:30.268726 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:30.268683 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f32d9de97e6c6e0783d8c50219edfd436cf96d1c86a46739aa050802dec7aa2b"} err="failed to get container status \"f32d9de97e6c6e0783d8c50219edfd436cf96d1c86a46739aa050802dec7aa2b\": rpc error: code = NotFound desc = could not find container \"f32d9de97e6c6e0783d8c50219edfd436cf96d1c86a46739aa050802dec7aa2b\": container with ID starting with f32d9de97e6c6e0783d8c50219edfd436cf96d1c86a46739aa050802dec7aa2b not found: ID does not exist" Jul 12 08:46:30.299980 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:30.299952 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6hwmd\" (UniqueName: \"kubernetes.io/projected/a5773191-0b52-4787-a154-10637350936b-kube-api-access-6hwmd\") pod \"a5773191-0b52-4787-a154-10637350936b\" (UID: \"a5773191-0b52-4787-a154-10637350936b\") " Jul 12 08:46:30.302033 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:30.302010 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5773191-0b52-4787-a154-10637350936b-kube-api-access-6hwmd" (OuterVolumeSpecName: "kube-api-access-6hwmd") pod "a5773191-0b52-4787-a154-10637350936b" (UID: "a5773191-0b52-4787-a154-10637350936b"). InnerVolumeSpecName "kube-api-access-6hwmd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:46:30.400745 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:30.400714 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hwmd\" (UniqueName: \"kubernetes.io/projected/a5773191-0b52-4787-a154-10637350936b-kube-api-access-6hwmd\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:46:30.579281 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:30.579222 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29730765-8sfrc"] Jul 12 08:46:30.582578 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:30.582532 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29730765-8sfrc"] Jul 12 08:46:31.026638 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:31.026607 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5773191-0b52-4787-a154-10637350936b" path="/var/lib/kubelet/pods/a5773191-0b52-4787-a154-10637350936b/volumes" Jul 12 08:46:41.909939 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:41.909893 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:46:41.997758 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:41.997725 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:46:47.436202 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:47.436163 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:46:47.494720 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:47.494684 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:46:47.667928 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:47.667896 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:46:47.766660 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:47.766625 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:46:48.835710 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:48.835672 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:46:48.939690 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:48.939656 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:46:49.061448 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:49.061418 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ea05cb34-6954bb5995-dbs8n"] Jul 12 08:46:49.061810 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:49.061796 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5773191-0b52-4787-a154-10637350936b" containerName="cleanup" Jul 12 08:46:49.061876 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:49.061812 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5773191-0b52-4787-a154-10637350936b" containerName="cleanup" Jul 12 08:46:49.061931 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:49.061888 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5773191-0b52-4787-a154-10637350936b" containerName="cleanup" Jul 12 08:46:49.065037 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:49.065020 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea05cb34-6954bb5995-dbs8n" Jul 12 08:46:49.067452 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:49.067413 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-ea05cb34-serving-cert\"" Jul 12 08:46:49.073762 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:49.073737 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ea05cb34-6954bb5995-dbs8n"] Jul 12 08:46:49.092306 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:49.092221 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:46:49.156943 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:49.156903 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae122f57-8282-44fb-aa5a-ded471e84513-maas-api-tls\") pod \"maas-api-e2e-ait-ea05cb34-6954bb5995-dbs8n\" (UID: \"ae122f57-8282-44fb-aa5a-ded471e84513\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea05cb34-6954bb5995-dbs8n" Jul 12 08:46:49.157133 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:49.156986 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjd2g\" (UniqueName: \"kubernetes.io/projected/ae122f57-8282-44fb-aa5a-ded471e84513-kube-api-access-pjd2g\") pod \"maas-api-e2e-ait-ea05cb34-6954bb5995-dbs8n\" (UID: \"ae122f57-8282-44fb-aa5a-ded471e84513\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea05cb34-6954bb5995-dbs8n" Jul 12 08:46:49.157133 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:49.157033 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae122f57-8282-44fb-aa5a-ded471e84513-openshift-service-ca\") pod \"maas-api-e2e-ait-ea05cb34-6954bb5995-dbs8n\" (UID: \"ae122f57-8282-44fb-aa5a-ded471e84513\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea05cb34-6954bb5995-dbs8n" Jul 12 08:46:49.257937 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:49.257900 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae122f57-8282-44fb-aa5a-ded471e84513-maas-api-tls\") pod \"maas-api-e2e-ait-ea05cb34-6954bb5995-dbs8n\" (UID: \"ae122f57-8282-44fb-aa5a-ded471e84513\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea05cb34-6954bb5995-dbs8n" Jul 12 08:46:49.258120 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:49.257962 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjd2g\" (UniqueName: \"kubernetes.io/projected/ae122f57-8282-44fb-aa5a-ded471e84513-kube-api-access-pjd2g\") pod \"maas-api-e2e-ait-ea05cb34-6954bb5995-dbs8n\" (UID: \"ae122f57-8282-44fb-aa5a-ded471e84513\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea05cb34-6954bb5995-dbs8n" Jul 12 08:46:49.258120 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:49.257991 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae122f57-8282-44fb-aa5a-ded471e84513-openshift-service-ca\") pod \"maas-api-e2e-ait-ea05cb34-6954bb5995-dbs8n\" (UID: \"ae122f57-8282-44fb-aa5a-ded471e84513\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea05cb34-6954bb5995-dbs8n" Jul 12 08:46:49.258627 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:49.258607 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae122f57-8282-44fb-aa5a-ded471e84513-openshift-service-ca\") pod \"maas-api-e2e-ait-ea05cb34-6954bb5995-dbs8n\" (UID: \"ae122f57-8282-44fb-aa5a-ded471e84513\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea05cb34-6954bb5995-dbs8n" Jul 12 08:46:49.260246 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:49.260211 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:46:49.260632 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:49.260614 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae122f57-8282-44fb-aa5a-ded471e84513-maas-api-tls\") pod \"maas-api-e2e-ait-ea05cb34-6954bb5995-dbs8n\" (UID: \"ae122f57-8282-44fb-aa5a-ded471e84513\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea05cb34-6954bb5995-dbs8n" Jul 12 08:46:49.271675 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:49.271651 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjd2g\" (UniqueName: \"kubernetes.io/projected/ae122f57-8282-44fb-aa5a-ded471e84513-kube-api-access-pjd2g\") pod \"maas-api-e2e-ait-ea05cb34-6954bb5995-dbs8n\" (UID: \"ae122f57-8282-44fb-aa5a-ded471e84513\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea05cb34-6954bb5995-dbs8n" Jul 12 08:46:49.377748 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:49.377659 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea05cb34-6954bb5995-dbs8n" Jul 12 08:46:49.514895 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:49.514865 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ea05cb34-6954bb5995-dbs8n"] Jul 12 08:46:49.517098 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:46:49.517072 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae122f57_8282_44fb_aa5a_ded471e84513.slice/crio-d832af7f6a576812067cab2f1dd93080b3331ab527620eef67a387f5f9ffbd31 WatchSource:0}: Error finding container d832af7f6a576812067cab2f1dd93080b3331ab527620eef67a387f5f9ffbd31: Status 404 returned error can't find the container with id d832af7f6a576812067cab2f1dd93080b3331ab527620eef67a387f5f9ffbd31 Jul 12 08:46:49.696343 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:49.696240 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:46:49.797290 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:49.797257 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:46:50.355758 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:50.355724 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea05cb34-6954bb5995-dbs8n" event={"ID":"ae122f57-8282-44fb-aa5a-ded471e84513","Type":"ContainerStarted","Data":"994f8aa251e7d7c65a894f419596f58a2b82d3c3bfd4486aa555d38dcdf4ea62"} Jul 12 08:46:50.356145 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:50.355775 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea05cb34-6954bb5995-dbs8n" event={"ID":"ae122f57-8282-44fb-aa5a-ded471e84513","Type":"ContainerStarted","Data":"d832af7f6a576812067cab2f1dd93080b3331ab527620eef67a387f5f9ffbd31"} Jul 12 08:46:50.356145 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:50.355803 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea05cb34-6954bb5995-dbs8n" Jul 12 08:46:50.371514 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:50.371465 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea05cb34-6954bb5995-dbs8n" podStartSLOduration=0.940993346 podStartE2EDuration="1.371446731s" totalImagesPullingTime="430.453385ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:46:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:46:50.369903441 +0000 UTC m=+1534.030699128" watchObservedRunningTime="2026-07-12 08:46:50.371446731 +0000 UTC m=+1534.032242393" Jul 12 08:46:51.408842 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:51.408804 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:46:53.850977 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:53.850944 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:46:54.046256 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:54.046222 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:46:54.148505 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:54.148422 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:46:54.154978 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:54.154949 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ea05cb34-6954bb5995-dbs8n"] Jul 12 08:46:54.155163 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:54.155137 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea05cb34-6954bb5995-dbs8n" podUID="ae122f57-8282-44fb-aa5a-ded471e84513" containerName="maas-api" containerID="cri-o://994f8aa251e7d7c65a894f419596f58a2b82d3c3bfd4486aa555d38dcdf4ea62" gracePeriod=30 Jul 12 08:46:54.248856 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:54.248826 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:46:54.382804 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:54.382776 2566 generic.go:356] "Generic (PLEG): container finished" podID="ae122f57-8282-44fb-aa5a-ded471e84513" containerID="994f8aa251e7d7c65a894f419596f58a2b82d3c3bfd4486aa555d38dcdf4ea62" exitCode=0 Jul 12 08:46:54.382950 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:54.382844 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea05cb34-6954bb5995-dbs8n" event={"ID":"ae122f57-8282-44fb-aa5a-ded471e84513","Type":"ContainerDied","Data":"994f8aa251e7d7c65a894f419596f58a2b82d3c3bfd4486aa555d38dcdf4ea62"} Jul 12 08:46:54.407619 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:54.407570 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea05cb34-6954bb5995-dbs8n" Jul 12 08:46:54.504715 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:54.504683 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjd2g\" (UniqueName: \"kubernetes.io/projected/ae122f57-8282-44fb-aa5a-ded471e84513-kube-api-access-pjd2g\") pod \"ae122f57-8282-44fb-aa5a-ded471e84513\" (UID: \"ae122f57-8282-44fb-aa5a-ded471e84513\") " Jul 12 08:46:54.504890 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:54.504728 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae122f57-8282-44fb-aa5a-ded471e84513-openshift-service-ca\") pod \"ae122f57-8282-44fb-aa5a-ded471e84513\" (UID: \"ae122f57-8282-44fb-aa5a-ded471e84513\") " Jul 12 08:46:54.504890 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:54.504757 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae122f57-8282-44fb-aa5a-ded471e84513-maas-api-tls\") pod \"ae122f57-8282-44fb-aa5a-ded471e84513\" (UID: \"ae122f57-8282-44fb-aa5a-ded471e84513\") " Jul 12 08:46:54.505104 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:54.505080 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae122f57-8282-44fb-aa5a-ded471e84513-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ae122f57-8282-44fb-aa5a-ded471e84513" (UID: "ae122f57-8282-44fb-aa5a-ded471e84513"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 08:46:54.506789 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:54.506761 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae122f57-8282-44fb-aa5a-ded471e84513-kube-api-access-pjd2g" (OuterVolumeSpecName: "kube-api-access-pjd2g") pod "ae122f57-8282-44fb-aa5a-ded471e84513" (UID: "ae122f57-8282-44fb-aa5a-ded471e84513"). InnerVolumeSpecName "kube-api-access-pjd2g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:46:54.506853 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:54.506786 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae122f57-8282-44fb-aa5a-ded471e84513-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ae122f57-8282-44fb-aa5a-ded471e84513" (UID: "ae122f57-8282-44fb-aa5a-ded471e84513"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 08:46:54.605726 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:54.605699 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjd2g\" (UniqueName: \"kubernetes.io/projected/ae122f57-8282-44fb-aa5a-ded471e84513-kube-api-access-pjd2g\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:46:54.605726 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:54.605720 2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae122f57-8282-44fb-aa5a-ded471e84513-openshift-service-ca\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:46:54.605726 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:54.605730 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae122f57-8282-44fb-aa5a-ded471e84513-maas-api-tls\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:46:55.388348 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:55.388256 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea05cb34-6954bb5995-dbs8n" Jul 12 08:46:55.388348 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:55.388287 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea05cb34-6954bb5995-dbs8n" event={"ID":"ae122f57-8282-44fb-aa5a-ded471e84513","Type":"ContainerDied","Data":"d832af7f6a576812067cab2f1dd93080b3331ab527620eef67a387f5f9ffbd31"} Jul 12 08:46:55.388801 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:55.388360 2566 scope.go:117] "RemoveContainer" containerID="994f8aa251e7d7c65a894f419596f58a2b82d3c3bfd4486aa555d38dcdf4ea62" Jul 12 08:46:55.402971 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:55.402945 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ea05cb34-6954bb5995-dbs8n"] Jul 12 08:46:55.404720 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:55.404700 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ea05cb34-6954bb5995-dbs8n"] Jul 12 08:46:55.504144 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:55.504110 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:46:57.032576 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:57.032528 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae122f57-8282-44fb-aa5a-ded471e84513" path="/var/lib/kubelet/pods/ae122f57-8282-44fb-aa5a-ded471e84513/volumes" Jul 12 08:46:59.872507 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:59.872472 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:46:59.923934 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:46:59.923902 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:00.085296 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:00.085264 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:00.185786 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:00.185705 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:01.169783 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:01.169752 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:01.270166 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:01.270130 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:01.371744 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:01.371698 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z"] Jul 12 08:47:01.372342 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:01.372320 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae122f57-8282-44fb-aa5a-ded471e84513" containerName="maas-api" Jul 12 08:47:01.372404 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:01.372345 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae122f57-8282-44fb-aa5a-ded471e84513" containerName="maas-api" Jul 12 08:47:01.372471 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:01.372458 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae122f57-8282-44fb-aa5a-ded471e84513" containerName="maas-api" Jul 12 08:47:01.377164 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:01.377141 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z" Jul 12 08:47:01.379329 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:01.379297 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-7d19ceaa-serving-cert\"" Jul 12 08:47:01.385083 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:01.384866 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z"] Jul 12 08:47:01.411988 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:01.411943 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:01.468206 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:01.468103 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfq7v\" (UniqueName: \"kubernetes.io/projected/251afbe1-02c9-440a-aa40-8a780471efb6-kube-api-access-gfq7v\") pod \"maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z\" (UID: \"251afbe1-02c9-440a-aa40-8a780471efb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z" Jul 12 08:47:01.468394 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:01.468216 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/251afbe1-02c9-440a-aa40-8a780471efb6-openshift-service-ca\") pod \"maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z\" (UID: \"251afbe1-02c9-440a-aa40-8a780471efb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z" Jul 12 08:47:01.468394 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:01.468241 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/251afbe1-02c9-440a-aa40-8a780471efb6-maas-api-tls\") pod \"maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z\" (UID: \"251afbe1-02c9-440a-aa40-8a780471efb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z" Jul 12 08:47:01.569012 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:01.568971 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/251afbe1-02c9-440a-aa40-8a780471efb6-openshift-service-ca\") pod \"maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z\" (UID: \"251afbe1-02c9-440a-aa40-8a780471efb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z" Jul 12 08:47:01.569012 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:01.569019 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/251afbe1-02c9-440a-aa40-8a780471efb6-maas-api-tls\") pod \"maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z\" (UID: \"251afbe1-02c9-440a-aa40-8a780471efb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z" Jul 12 08:47:01.569272 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:01.569052 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfq7v\" (UniqueName: \"kubernetes.io/projected/251afbe1-02c9-440a-aa40-8a780471efb6-kube-api-access-gfq7v\") pod \"maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z\" (UID: \"251afbe1-02c9-440a-aa40-8a780471efb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z" Jul 12 08:47:01.569272 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:47:01.569192 2566 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-7d19ceaa-serving-cert: secret "maas-api-e2e-ait-7d19ceaa-serving-cert" not found Jul 12 08:47:01.569272 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:47:01.569259 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/251afbe1-02c9-440a-aa40-8a780471efb6-maas-api-tls podName:251afbe1-02c9-440a-aa40-8a780471efb6 nodeName:}" failed. No retries permitted until 2026-07-12 08:47:02.069242678 +0000 UTC m=+1545.730038332 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/251afbe1-02c9-440a-aa40-8a780471efb6-maas-api-tls") pod "maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z" (UID: "251afbe1-02c9-440a-aa40-8a780471efb6") : secret "maas-api-e2e-ait-7d19ceaa-serving-cert" not found Jul 12 08:47:01.569789 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:01.569759 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/251afbe1-02c9-440a-aa40-8a780471efb6-openshift-service-ca\") pod \"maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z\" (UID: \"251afbe1-02c9-440a-aa40-8a780471efb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z" Jul 12 08:47:01.577534 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:01.577504 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfq7v\" (UniqueName: \"kubernetes.io/projected/251afbe1-02c9-440a-aa40-8a780471efb6-kube-api-access-gfq7v\") pod \"maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z\" (UID: \"251afbe1-02c9-440a-aa40-8a780471efb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z" Jul 12 08:47:01.820167 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:01.820135 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:01.919234 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:01.919188 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:02.018673 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:02.018631 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:02.073114 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:02.073023 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/251afbe1-02c9-440a-aa40-8a780471efb6-maas-api-tls\") pod \"maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z\" (UID: \"251afbe1-02c9-440a-aa40-8a780471efb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z" Jul 12 08:47:02.075602 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:02.075574 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/251afbe1-02c9-440a-aa40-8a780471efb6-maas-api-tls\") pod \"maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z\" (UID: \"251afbe1-02c9-440a-aa40-8a780471efb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z" Jul 12 08:47:02.289709 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:02.289670 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z" Jul 12 08:47:02.431949 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:02.431922 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z"] Jul 12 08:47:02.433671 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:47:02.433648 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod251afbe1_02c9_440a_aa40_8a780471efb6.slice/crio-67c1a58ce55b7df4dff23790d34fab63f0bb6990855916ec7c2c5f65103e512f WatchSource:0}: Error finding container 67c1a58ce55b7df4dff23790d34fab63f0bb6990855916ec7c2c5f65103e512f: Status 404 returned error can't find the container with id 67c1a58ce55b7df4dff23790d34fab63f0bb6990855916ec7c2c5f65103e512f Jul 12 08:47:03.390664 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:03.390629 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:03.429746 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:03.429711 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z" event={"ID":"251afbe1-02c9-440a-aa40-8a780471efb6","Type":"ContainerStarted","Data":"b4648352dae28e564f2628469b1e5a5d7de16c65da508667e7f0ffa6a4d58587"} Jul 12 08:47:03.429945 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:03.429926 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z" Jul 12 08:47:03.429991 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:03.429969 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z" event={"ID":"251afbe1-02c9-440a-aa40-8a780471efb6","Type":"ContainerStarted","Data":"67c1a58ce55b7df4dff23790d34fab63f0bb6990855916ec7c2c5f65103e512f"} Jul 12 08:47:03.446036 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:03.445986 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z" podStartSLOduration=2.084439426 podStartE2EDuration="2.445972493s" totalImagesPullingTime="361.533067ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:47:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:47:03.4436941 +0000 UTC m=+1547.104489761" watchObservedRunningTime="2026-07-12 08:47:03.445972493 +0000 UTC m=+1547.106768154" Jul 12 08:47:06.307618 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:06.307588 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:06.596768 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:06.596682 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z"] Jul 12 08:47:06.596925 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:06.596899 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z" podUID="251afbe1-02c9-440a-aa40-8a780471efb6" containerName="maas-api" containerID="cri-o://b4648352dae28e564f2628469b1e5a5d7de16c65da508667e7f0ffa6a4d58587" gracePeriod=30 Jul 12 08:47:06.621435 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:06.621414 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:06.869068 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:06.869043 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z" Jul 12 08:47:07.017357 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:07.017323 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/251afbe1-02c9-440a-aa40-8a780471efb6-openshift-service-ca\") pod \"251afbe1-02c9-440a-aa40-8a780471efb6\" (UID: \"251afbe1-02c9-440a-aa40-8a780471efb6\") " Jul 12 08:47:07.017575 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:07.017370 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfq7v\" (UniqueName: \"kubernetes.io/projected/251afbe1-02c9-440a-aa40-8a780471efb6-kube-api-access-gfq7v\") pod \"251afbe1-02c9-440a-aa40-8a780471efb6\" (UID: \"251afbe1-02c9-440a-aa40-8a780471efb6\") " Jul 12 08:47:07.017575 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:07.017423 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/251afbe1-02c9-440a-aa40-8a780471efb6-maas-api-tls\") pod \"251afbe1-02c9-440a-aa40-8a780471efb6\" (UID: \"251afbe1-02c9-440a-aa40-8a780471efb6\") " Jul 12 08:47:07.017792 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:07.017763 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/251afbe1-02c9-440a-aa40-8a780471efb6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "251afbe1-02c9-440a-aa40-8a780471efb6" (UID: "251afbe1-02c9-440a-aa40-8a780471efb6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 08:47:07.019512 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:07.019491 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/251afbe1-02c9-440a-aa40-8a780471efb6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "251afbe1-02c9-440a-aa40-8a780471efb6" (UID: "251afbe1-02c9-440a-aa40-8a780471efb6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 08:47:07.019586 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:07.019499 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/251afbe1-02c9-440a-aa40-8a780471efb6-kube-api-access-gfq7v" (OuterVolumeSpecName: "kube-api-access-gfq7v") pod "251afbe1-02c9-440a-aa40-8a780471efb6" (UID: "251afbe1-02c9-440a-aa40-8a780471efb6"). InnerVolumeSpecName "kube-api-access-gfq7v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:47:07.118082 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:07.117999 2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/251afbe1-02c9-440a-aa40-8a780471efb6-openshift-service-ca\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:47:07.118082 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:07.118023 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfq7v\" (UniqueName: \"kubernetes.io/projected/251afbe1-02c9-440a-aa40-8a780471efb6-kube-api-access-gfq7v\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:47:07.118082 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:07.118034 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/251afbe1-02c9-440a-aa40-8a780471efb6-maas-api-tls\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:47:07.450263 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:07.450176 2566 generic.go:356] "Generic (PLEG): container finished" podID="251afbe1-02c9-440a-aa40-8a780471efb6" containerID="b4648352dae28e564f2628469b1e5a5d7de16c65da508667e7f0ffa6a4d58587" exitCode=0 Jul 12 08:47:07.450263 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:07.450222 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z" Jul 12 08:47:07.450745 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:07.450272 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z" event={"ID":"251afbe1-02c9-440a-aa40-8a780471efb6","Type":"ContainerDied","Data":"b4648352dae28e564f2628469b1e5a5d7de16c65da508667e7f0ffa6a4d58587"} Jul 12 08:47:07.450745 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:07.450321 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z" event={"ID":"251afbe1-02c9-440a-aa40-8a780471efb6","Type":"ContainerDied","Data":"67c1a58ce55b7df4dff23790d34fab63f0bb6990855916ec7c2c5f65103e512f"} Jul 12 08:47:07.450745 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:07.450343 2566 scope.go:117] "RemoveContainer" containerID="b4648352dae28e564f2628469b1e5a5d7de16c65da508667e7f0ffa6a4d58587" Jul 12 08:47:07.458743 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:07.458725 2566 scope.go:117] "RemoveContainer" containerID="b4648352dae28e564f2628469b1e5a5d7de16c65da508667e7f0ffa6a4d58587" Jul 12 08:47:07.458993 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:47:07.458974 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4648352dae28e564f2628469b1e5a5d7de16c65da508667e7f0ffa6a4d58587\": container with ID starting with b4648352dae28e564f2628469b1e5a5d7de16c65da508667e7f0ffa6a4d58587 not found: ID does not exist" containerID="b4648352dae28e564f2628469b1e5a5d7de16c65da508667e7f0ffa6a4d58587" Jul 12 08:47:07.459046 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:07.459001 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4648352dae28e564f2628469b1e5a5d7de16c65da508667e7f0ffa6a4d58587"} err="failed to get container status \"b4648352dae28e564f2628469b1e5a5d7de16c65da508667e7f0ffa6a4d58587\": rpc error: code = NotFound desc = could not find container \"b4648352dae28e564f2628469b1e5a5d7de16c65da508667e7f0ffa6a4d58587\": container with ID starting with b4648352dae28e564f2628469b1e5a5d7de16c65da508667e7f0ffa6a4d58587 not found: ID does not exist" Jul 12 08:47:07.465016 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:07.464996 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z"] Jul 12 08:47:07.468844 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:07.468825 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7d19ceaa-56467dfd7-8rb5z"] Jul 12 08:47:09.027623 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:09.027578 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="251afbe1-02c9-440a-aa40-8a780471efb6" path="/var/lib/kubelet/pods/251afbe1-02c9-440a-aa40-8a780471efb6/volumes" Jul 12 08:47:12.377855 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:12.377799 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:12.469471 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:12.469441 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:17.996847 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:17.996810 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:18.099430 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.099399 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:18.172452 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.172419 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc"] Jul 12 08:47:18.172887 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.172872 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="251afbe1-02c9-440a-aa40-8a780471efb6" containerName="maas-api" Jul 12 08:47:18.172887 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.172888 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="251afbe1-02c9-440a-aa40-8a780471efb6" containerName="maas-api" Jul 12 08:47:18.173015 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.172949 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="251afbe1-02c9-440a-aa40-8a780471efb6" containerName="maas-api" Jul 12 08:47:18.175843 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.175823 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" Jul 12 08:47:18.178137 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.178116 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-94e99940-openshift-default-dockercfg-dq8j8\"" Jul 12 08:47:18.178264 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.178176 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 12 08:47:18.187987 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.187961 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc"] Jul 12 08:47:18.204241 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.204214 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:18.316160 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.316054 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwgtz\" (UniqueName: \"kubernetes.io/projected/7cfe5df2-6b7a-4450-ac95-66cb7846f082-kube-api-access-wwgtz\") pod \"e2e-derive-94e99940-openshift-default-7666644d79-dmdcc\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" Jul 12 08:47:18.316343 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.316174 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cfe5df2-6b7a-4450-ac95-66cb7846f082-istio-podinfo\") pod \"e2e-derive-94e99940-openshift-default-7666644d79-dmdcc\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" Jul 12 08:47:18.316343 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.316207 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cfe5df2-6b7a-4450-ac95-66cb7846f082-istio-data\") pod \"e2e-derive-94e99940-openshift-default-7666644d79-dmdcc\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" Jul 12 08:47:18.316343 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.316283 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cfe5df2-6b7a-4450-ac95-66cb7846f082-workload-socket\") pod \"e2e-derive-94e99940-openshift-default-7666644d79-dmdcc\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" Jul 12 08:47:18.316343 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.316323 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cfe5df2-6b7a-4450-ac95-66cb7846f082-workload-certs\") pod \"e2e-derive-94e99940-openshift-default-7666644d79-dmdcc\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" Jul 12 08:47:18.316533 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.316353 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cfe5df2-6b7a-4450-ac95-66cb7846f082-istio-envoy\") pod \"e2e-derive-94e99940-openshift-default-7666644d79-dmdcc\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" Jul 12 08:47:18.316533 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.316390 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cfe5df2-6b7a-4450-ac95-66cb7846f082-istiod-ca-cert\") pod \"e2e-derive-94e99940-openshift-default-7666644d79-dmdcc\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" Jul 12 08:47:18.316533 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.316427 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cfe5df2-6b7a-4450-ac95-66cb7846f082-credential-socket\") pod \"e2e-derive-94e99940-openshift-default-7666644d79-dmdcc\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" Jul 12 08:47:18.316533 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.316452 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cfe5df2-6b7a-4450-ac95-66cb7846f082-istio-token\") pod \"e2e-derive-94e99940-openshift-default-7666644d79-dmdcc\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" Jul 12 08:47:18.417736 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.417688 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cfe5df2-6b7a-4450-ac95-66cb7846f082-istio-podinfo\") pod \"e2e-derive-94e99940-openshift-default-7666644d79-dmdcc\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" Jul 12 08:47:18.417736 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.417737 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cfe5df2-6b7a-4450-ac95-66cb7846f082-istio-data\") pod \"e2e-derive-94e99940-openshift-default-7666644d79-dmdcc\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" Jul 12 08:47:18.417958 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.417862 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cfe5df2-6b7a-4450-ac95-66cb7846f082-workload-socket\") pod \"e2e-derive-94e99940-openshift-default-7666644d79-dmdcc\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" Jul 12 08:47:18.417958 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.417888 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cfe5df2-6b7a-4450-ac95-66cb7846f082-workload-certs\") pod \"e2e-derive-94e99940-openshift-default-7666644d79-dmdcc\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" Jul 12 08:47:18.417958 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.417921 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cfe5df2-6b7a-4450-ac95-66cb7846f082-istio-envoy\") pod \"e2e-derive-94e99940-openshift-default-7666644d79-dmdcc\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" Jul 12 08:47:18.417958 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.417953 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cfe5df2-6b7a-4450-ac95-66cb7846f082-istiod-ca-cert\") pod \"e2e-derive-94e99940-openshift-default-7666644d79-dmdcc\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" Jul 12 08:47:18.418163 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.417987 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cfe5df2-6b7a-4450-ac95-66cb7846f082-credential-socket\") pod \"e2e-derive-94e99940-openshift-default-7666644d79-dmdcc\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" Jul 12 08:47:18.418163 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.418010 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cfe5df2-6b7a-4450-ac95-66cb7846f082-istio-token\") pod \"e2e-derive-94e99940-openshift-default-7666644d79-dmdcc\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" Jul 12 08:47:18.418163 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.418046 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwgtz\" (UniqueName: \"kubernetes.io/projected/7cfe5df2-6b7a-4450-ac95-66cb7846f082-kube-api-access-wwgtz\") pod \"e2e-derive-94e99940-openshift-default-7666644d79-dmdcc\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" Jul 12 08:47:18.418163 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.418157 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cfe5df2-6b7a-4450-ac95-66cb7846f082-istio-data\") pod \"e2e-derive-94e99940-openshift-default-7666644d79-dmdcc\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" Jul 12 08:47:18.418364 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.418271 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cfe5df2-6b7a-4450-ac95-66cb7846f082-workload-certs\") pod \"e2e-derive-94e99940-openshift-default-7666644d79-dmdcc\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" Jul 12 08:47:18.418437 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.418355 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cfe5df2-6b7a-4450-ac95-66cb7846f082-credential-socket\") pod \"e2e-derive-94e99940-openshift-default-7666644d79-dmdcc\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" Jul 12 08:47:18.418437 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.418393 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cfe5df2-6b7a-4450-ac95-66cb7846f082-workload-socket\") pod \"e2e-derive-94e99940-openshift-default-7666644d79-dmdcc\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" Jul 12 08:47:18.418701 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.418674 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cfe5df2-6b7a-4450-ac95-66cb7846f082-istiod-ca-cert\") pod \"e2e-derive-94e99940-openshift-default-7666644d79-dmdcc\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" Jul 12 08:47:18.420196 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.420173 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cfe5df2-6b7a-4450-ac95-66cb7846f082-istio-envoy\") pod \"e2e-derive-94e99940-openshift-default-7666644d79-dmdcc\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" Jul 12 08:47:18.420310 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.420283 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cfe5df2-6b7a-4450-ac95-66cb7846f082-istio-podinfo\") pod \"e2e-derive-94e99940-openshift-default-7666644d79-dmdcc\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" Jul 12 08:47:18.424914 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.424893 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cfe5df2-6b7a-4450-ac95-66cb7846f082-istio-token\") pod \"e2e-derive-94e99940-openshift-default-7666644d79-dmdcc\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" Jul 12 08:47:18.425770 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.425738 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwgtz\" (UniqueName: \"kubernetes.io/projected/7cfe5df2-6b7a-4450-ac95-66cb7846f082-kube-api-access-wwgtz\") pod \"e2e-derive-94e99940-openshift-default-7666644d79-dmdcc\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" Jul 12 08:47:18.488812 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.488778 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" Jul 12 08:47:18.632281 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:18.632254 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc"] Jul 12 08:47:18.634003 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:47:18.633977 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cfe5df2_6b7a_4450_ac95_66cb7846f082.slice/crio-62f95458acfcc8cb679caa8f9537207eb99ef9a111d08c5bb724897657a8b5ac WatchSource:0}: Error finding container 62f95458acfcc8cb679caa8f9537207eb99ef9a111d08c5bb724897657a8b5ac: Status 404 returned error can't find the container with id 62f95458acfcc8cb679caa8f9537207eb99ef9a111d08c5bb724897657a8b5ac Jul 12 08:47:19.384648 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:19.384597 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:19.474244 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:19.474103 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:19.480138 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:19.479825 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z"] Jul 12 08:47:19.486422 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:19.486397 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z" Jul 12 08:47:19.489005 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:19.488980 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-94e99940-serving-cert\"" Jul 12 08:47:19.502716 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:19.501656 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z"] Jul 12 08:47:19.517242 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:19.517179 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" event={"ID":"7cfe5df2-6b7a-4450-ac95-66cb7846f082","Type":"ContainerStarted","Data":"62f95458acfcc8cb679caa8f9537207eb99ef9a111d08c5bb724897657a8b5ac"} Jul 12 08:47:19.533610 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:19.533574 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:19.630977 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:19.629846 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4m7k\" (UniqueName: \"kubernetes.io/projected/8571744e-a31f-417a-9d7f-a72b1e4792c8-kube-api-access-d4m7k\") pod \"maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z\" (UID: \"8571744e-a31f-417a-9d7f-a72b1e4792c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z" Jul 12 08:47:19.630977 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:19.629907 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8571744e-a31f-417a-9d7f-a72b1e4792c8-maas-api-tls\") pod \"maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z\" (UID: \"8571744e-a31f-417a-9d7f-a72b1e4792c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z" Jul 12 08:47:19.630977 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:19.629935 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8571744e-a31f-417a-9d7f-a72b1e4792c8-openshift-service-ca\") pod \"maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z\" (UID: \"8571744e-a31f-417a-9d7f-a72b1e4792c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z" Jul 12 08:47:19.673333 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:19.673296 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:19.730874 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:19.730834 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4m7k\" (UniqueName: \"kubernetes.io/projected/8571744e-a31f-417a-9d7f-a72b1e4792c8-kube-api-access-d4m7k\") pod \"maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z\" (UID: \"8571744e-a31f-417a-9d7f-a72b1e4792c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z" Jul 12 08:47:19.731071 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:19.730893 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8571744e-a31f-417a-9d7f-a72b1e4792c8-maas-api-tls\") pod \"maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z\" (UID: \"8571744e-a31f-417a-9d7f-a72b1e4792c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z" Jul 12 08:47:19.731071 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:19.730918 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8571744e-a31f-417a-9d7f-a72b1e4792c8-openshift-service-ca\") pod \"maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z\" (UID: \"8571744e-a31f-417a-9d7f-a72b1e4792c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z" Jul 12 08:47:19.731272 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:47:19.731249 2566 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-94e99940-serving-cert: secret "maas-api-e2e-derive-94e99940-serving-cert" not found Jul 12 08:47:19.731354 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:47:19.731340 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8571744e-a31f-417a-9d7f-a72b1e4792c8-maas-api-tls podName:8571744e-a31f-417a-9d7f-a72b1e4792c8 nodeName:}" failed. No retries permitted until 2026-07-12 08:47:20.23131554 +0000 UTC m=+1563.892111181 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8571744e-a31f-417a-9d7f-a72b1e4792c8-maas-api-tls") pod "maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z" (UID: "8571744e-a31f-417a-9d7f-a72b1e4792c8") : secret "maas-api-e2e-derive-94e99940-serving-cert" not found Jul 12 08:47:19.731749 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:19.731708 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8571744e-a31f-417a-9d7f-a72b1e4792c8-openshift-service-ca\") pod \"maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z\" (UID: \"8571744e-a31f-417a-9d7f-a72b1e4792c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z" Jul 12 08:47:19.740341 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:19.740260 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4m7k\" (UniqueName: \"kubernetes.io/projected/8571744e-a31f-417a-9d7f-a72b1e4792c8-kube-api-access-d4m7k\") pod \"maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z\" (UID: \"8571744e-a31f-417a-9d7f-a72b1e4792c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z" Jul 12 08:47:19.770012 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:19.769979 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:19.909102 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:19.909010 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:20.037304 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:20.037275 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:20.137838 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:20.137799 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:20.235683 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:20.235652 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8571744e-a31f-417a-9d7f-a72b1e4792c8-maas-api-tls\") pod \"maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z\" (UID: \"8571744e-a31f-417a-9d7f-a72b1e4792c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z" Jul 12 08:47:20.238606 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:20.238579 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8571744e-a31f-417a-9d7f-a72b1e4792c8-maas-api-tls\") pod \"maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z\" (UID: \"8571744e-a31f-417a-9d7f-a72b1e4792c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z" Jul 12 08:47:20.404485 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:20.404448 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z" Jul 12 08:47:21.224620 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:21.224576 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:47:21.224765 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:21.224658 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:47:21.224765 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:21.224687 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:47:21.293196 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:21.293167 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z"] Jul 12 08:47:21.294887 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:47:21.294856 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8571744e_a31f_417a_9d7f_a72b1e4792c8.slice/crio-5570aedeed52c17ffb8ffa6f7090d0148fb8afc6b0b8be5256b7f6965413fd23 WatchSource:0}: Error finding container 5570aedeed52c17ffb8ffa6f7090d0148fb8afc6b0b8be5256b7f6965413fd23: Status 404 returned error can't find the container with id 5570aedeed52c17ffb8ffa6f7090d0148fb8afc6b0b8be5256b7f6965413fd23 Jul 12 08:47:21.532386 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:21.532348 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" event={"ID":"7cfe5df2-6b7a-4450-ac95-66cb7846f082","Type":"ContainerStarted","Data":"d21221e236ae68a4a53ab08b10427a89ecbcd2c0c396fe57d48dedb4ddd20381"} Jul 12 08:47:21.533945 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:21.533916 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z" event={"ID":"8571744e-a31f-417a-9d7f-a72b1e4792c8","Type":"ContainerStarted","Data":"5570aedeed52c17ffb8ffa6f7090d0148fb8afc6b0b8be5256b7f6965413fd23"} Jul 12 08:47:21.555119 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:21.555077 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" podStartSLOduration=0.966974096 podStartE2EDuration="3.555063749s" totalImagesPullingTime="2.588089653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:47:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:47:21.553707222 +0000 UTC m=+1565.214502905" watchObservedRunningTime="2026-07-12 08:47:21.555063749 +0000 UTC m=+1565.215859411" Jul 12 08:47:22.489286 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:22.489251 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" Jul 12 08:47:22.494681 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:22.494656 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" Jul 12 08:47:22.540912 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:22.540876 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z" event={"ID":"8571744e-a31f-417a-9d7f-a72b1e4792c8","Type":"ContainerStarted","Data":"2ce5dd0f8969865b941d1ab73f393804f3640766208bfbf129dbad639e285c8e"} Jul 12 08:47:22.541344 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:22.541009 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z" Jul 12 08:47:22.541344 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:22.541216 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" Jul 12 08:47:22.541875 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:22.541858 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" Jul 12 08:47:22.556032 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:22.555992 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z" podStartSLOduration=3.247807052 podStartE2EDuration="3.555979803s" totalImagesPullingTime="308.172751ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:47:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:47:22.555082609 +0000 UTC m=+1566.215878294" watchObservedRunningTime="2026-07-12 08:47:22.555979803 +0000 UTC m=+1566.216775514" Jul 12 08:47:23.012128 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:23.012078 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:23.732703 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:23.732635 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:23.923916 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:23.923881 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc"] Jul 12 08:47:23.941800 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:23.941768 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:24.050165 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:24.050086 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:24.193645 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:24.193602 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z"] Jul 12 08:47:24.217897 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:24.217797 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:24.550241 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:24.550200 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z" podUID="8571744e-a31f-417a-9d7f-a72b1e4792c8" containerName="maas-api" containerID="cri-o://2ce5dd0f8969865b941d1ab73f393804f3640766208bfbf129dbad639e285c8e" gracePeriod=30 Jul 12 08:47:24.550490 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:24.550465 2566 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" secret="" err="secret \"e2e-derive-94e99940-openshift-default-dockercfg-dq8j8\" not found" Jul 12 08:47:24.843028 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:24.843001 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z" Jul 12 08:47:24.980607 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:24.980567 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8571744e-a31f-417a-9d7f-a72b1e4792c8-maas-api-tls\") pod \"8571744e-a31f-417a-9d7f-a72b1e4792c8\" (UID: \"8571744e-a31f-417a-9d7f-a72b1e4792c8\") " Jul 12 08:47:24.980771 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:24.980657 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8571744e-a31f-417a-9d7f-a72b1e4792c8-openshift-service-ca\") pod \"8571744e-a31f-417a-9d7f-a72b1e4792c8\" (UID: \"8571744e-a31f-417a-9d7f-a72b1e4792c8\") " Jul 12 08:47:24.980771 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:24.980700 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4m7k\" (UniqueName: \"kubernetes.io/projected/8571744e-a31f-417a-9d7f-a72b1e4792c8-kube-api-access-d4m7k\") pod \"8571744e-a31f-417a-9d7f-a72b1e4792c8\" (UID: \"8571744e-a31f-417a-9d7f-a72b1e4792c8\") " Jul 12 08:47:24.981039 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:24.980994 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8571744e-a31f-417a-9d7f-a72b1e4792c8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8571744e-a31f-417a-9d7f-a72b1e4792c8" (UID: "8571744e-a31f-417a-9d7f-a72b1e4792c8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 08:47:24.982737 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:24.982717 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8571744e-a31f-417a-9d7f-a72b1e4792c8-kube-api-access-d4m7k" (OuterVolumeSpecName: "kube-api-access-d4m7k") pod "8571744e-a31f-417a-9d7f-a72b1e4792c8" (UID: "8571744e-a31f-417a-9d7f-a72b1e4792c8"). InnerVolumeSpecName "kube-api-access-d4m7k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:47:24.982797 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:24.982760 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8571744e-a31f-417a-9d7f-a72b1e4792c8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8571744e-a31f-417a-9d7f-a72b1e4792c8" (UID: "8571744e-a31f-417a-9d7f-a72b1e4792c8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 08:47:25.081960 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:25.081880 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4m7k\" (UniqueName: \"kubernetes.io/projected/8571744e-a31f-417a-9d7f-a72b1e4792c8-kube-api-access-d4m7k\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:47:25.081960 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:25.081910 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8571744e-a31f-417a-9d7f-a72b1e4792c8-maas-api-tls\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:47:25.081960 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:25.081921 2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8571744e-a31f-417a-9d7f-a72b1e4792c8-openshift-service-ca\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:47:25.555268 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:25.555236 2566 generic.go:356] "Generic (PLEG): container finished" podID="8571744e-a31f-417a-9d7f-a72b1e4792c8" containerID="2ce5dd0f8969865b941d1ab73f393804f3640766208bfbf129dbad639e285c8e" exitCode=0 Jul 12 08:47:25.555425 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:25.555290 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z" Jul 12 08:47:25.555425 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:25.555330 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z" event={"ID":"8571744e-a31f-417a-9d7f-a72b1e4792c8","Type":"ContainerDied","Data":"2ce5dd0f8969865b941d1ab73f393804f3640766208bfbf129dbad639e285c8e"} Jul 12 08:47:25.555425 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:25.555386 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z" event={"ID":"8571744e-a31f-417a-9d7f-a72b1e4792c8","Type":"ContainerDied","Data":"5570aedeed52c17ffb8ffa6f7090d0148fb8afc6b0b8be5256b7f6965413fd23"} Jul 12 08:47:25.555425 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:25.555415 2566 scope.go:117] "RemoveContainer" containerID="2ce5dd0f8969865b941d1ab73f393804f3640766208bfbf129dbad639e285c8e" Jul 12 08:47:25.555643 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:25.555621 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" podUID="7cfe5df2-6b7a-4450-ac95-66cb7846f082" containerName="istio-proxy" containerID="cri-o://d21221e236ae68a4a53ab08b10427a89ecbcd2c0c396fe57d48dedb4ddd20381" gracePeriod=30 Jul 12 08:47:25.563965 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:25.563947 2566 scope.go:117] "RemoveContainer" containerID="2ce5dd0f8969865b941d1ab73f393804f3640766208bfbf129dbad639e285c8e" Jul 12 08:47:25.564228 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:47:25.564209 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ce5dd0f8969865b941d1ab73f393804f3640766208bfbf129dbad639e285c8e\": container with ID starting with 2ce5dd0f8969865b941d1ab73f393804f3640766208bfbf129dbad639e285c8e not found: ID does not exist" containerID="2ce5dd0f8969865b941d1ab73f393804f3640766208bfbf129dbad639e285c8e" Jul 12 08:47:25.564289 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:25.564241 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ce5dd0f8969865b941d1ab73f393804f3640766208bfbf129dbad639e285c8e"} err="failed to get container status \"2ce5dd0f8969865b941d1ab73f393804f3640766208bfbf129dbad639e285c8e\": rpc error: code = NotFound desc = could not find container \"2ce5dd0f8969865b941d1ab73f393804f3640766208bfbf129dbad639e285c8e\": container with ID starting with 2ce5dd0f8969865b941d1ab73f393804f3640766208bfbf129dbad639e285c8e not found: ID does not exist" Jul 12 08:47:25.585409 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:25.585383 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z"] Jul 12 08:47:25.588853 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:25.588832 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-94e99940-74cb7d5b74-tt49z"] Jul 12 08:47:25.720099 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:25.720006 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:27.034233 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:27.034194 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8571744e-a31f-417a-9d7f-a72b1e4792c8" path="/var/lib/kubelet/pods/8571744e-a31f-417a-9d7f-a72b1e4792c8/volumes" Jul 12 08:47:29.549250 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:29.549205 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:29.609703 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:29.609675 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:29.906123 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:29.906103 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" Jul 12 08:47:30.009697 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.009658 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:30.025407 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.025382 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cfe5df2-6b7a-4450-ac95-66cb7846f082-workload-certs\") pod \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " Jul 12 08:47:30.025561 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.025420 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cfe5df2-6b7a-4450-ac95-66cb7846f082-credential-socket\") pod \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " Jul 12 08:47:30.025561 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.025448 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wwgtz\" (UniqueName: \"kubernetes.io/projected/7cfe5df2-6b7a-4450-ac95-66cb7846f082-kube-api-access-wwgtz\") pod \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " Jul 12 08:47:30.025561 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.025478 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cfe5df2-6b7a-4450-ac95-66cb7846f082-istio-podinfo\") pod \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " Jul 12 08:47:30.025561 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.025506 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cfe5df2-6b7a-4450-ac95-66cb7846f082-istio-data\") pod \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " Jul 12 08:47:30.025561 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.025534 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cfe5df2-6b7a-4450-ac95-66cb7846f082-workload-socket\") pod \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " Jul 12 08:47:30.025763 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.025585 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cfe5df2-6b7a-4450-ac95-66cb7846f082-istio-envoy\") pod \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " Jul 12 08:47:30.025763 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.025615 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cfe5df2-6b7a-4450-ac95-66cb7846f082-istiod-ca-cert\") pod \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " Jul 12 08:47:30.025763 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.025661 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cfe5df2-6b7a-4450-ac95-66cb7846f082-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7cfe5df2-6b7a-4450-ac95-66cb7846f082" (UID: "7cfe5df2-6b7a-4450-ac95-66cb7846f082"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:47:30.025763 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.025676 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cfe5df2-6b7a-4450-ac95-66cb7846f082-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7cfe5df2-6b7a-4450-ac95-66cb7846f082" (UID: "7cfe5df2-6b7a-4450-ac95-66cb7846f082"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:47:30.025763 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.025697 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cfe5df2-6b7a-4450-ac95-66cb7846f082-istio-token\") pod \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\" (UID: \"7cfe5df2-6b7a-4450-ac95-66cb7846f082\") " Jul 12 08:47:30.026040 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.025854 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cfe5df2-6b7a-4450-ac95-66cb7846f082-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7cfe5df2-6b7a-4450-ac95-66cb7846f082" (UID: "7cfe5df2-6b7a-4450-ac95-66cb7846f082"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:47:30.026040 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.026017 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cfe5df2-6b7a-4450-ac95-66cb7846f082-workload-certs\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:47:30.026159 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.026034 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cfe5df2-6b7a-4450-ac95-66cb7846f082-credential-socket\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:47:30.026159 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.026074 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cfe5df2-6b7a-4450-ac95-66cb7846f082-workload-socket\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:47:30.026262 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.026220 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7cfe5df2-6b7a-4450-ac95-66cb7846f082-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7cfe5df2-6b7a-4450-ac95-66cb7846f082" (UID: "7cfe5df2-6b7a-4450-ac95-66cb7846f082"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 08:47:30.026390 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.026368 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cfe5df2-6b7a-4450-ac95-66cb7846f082-istio-data" (OuterVolumeSpecName: "istio-data") pod "7cfe5df2-6b7a-4450-ac95-66cb7846f082" (UID: "7cfe5df2-6b7a-4450-ac95-66cb7846f082"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:47:30.027882 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.027858 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cfe5df2-6b7a-4450-ac95-66cb7846f082-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7cfe5df2-6b7a-4450-ac95-66cb7846f082" (UID: "7cfe5df2-6b7a-4450-ac95-66cb7846f082"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:47:30.027986 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.027950 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cfe5df2-6b7a-4450-ac95-66cb7846f082-kube-api-access-wwgtz" (OuterVolumeSpecName: "kube-api-access-wwgtz") pod "7cfe5df2-6b7a-4450-ac95-66cb7846f082" (UID: "7cfe5df2-6b7a-4450-ac95-66cb7846f082"). InnerVolumeSpecName "kube-api-access-wwgtz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:47:30.028122 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.028101 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cfe5df2-6b7a-4450-ac95-66cb7846f082-istio-token" (OuterVolumeSpecName: "istio-token") pod "7cfe5df2-6b7a-4450-ac95-66cb7846f082" (UID: "7cfe5df2-6b7a-4450-ac95-66cb7846f082"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:47:30.028122 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.028115 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7cfe5df2-6b7a-4450-ac95-66cb7846f082-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7cfe5df2-6b7a-4450-ac95-66cb7846f082" (UID: "7cfe5df2-6b7a-4450-ac95-66cb7846f082"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 08:47:30.111230 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.111167 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:30.127370 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.127346 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cfe5df2-6b7a-4450-ac95-66cb7846f082-istio-token\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:47:30.127370 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.127369 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwgtz\" (UniqueName: \"kubernetes.io/projected/7cfe5df2-6b7a-4450-ac95-66cb7846f082-kube-api-access-wwgtz\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:47:30.127516 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.127379 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cfe5df2-6b7a-4450-ac95-66cb7846f082-istio-podinfo\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:47:30.127516 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.127388 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cfe5df2-6b7a-4450-ac95-66cb7846f082-istio-data\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:47:30.127516 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.127400 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cfe5df2-6b7a-4450-ac95-66cb7846f082-istio-envoy\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:47:30.127516 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.127411 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cfe5df2-6b7a-4450-ac95-66cb7846f082-istiod-ca-cert\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:47:30.209591 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.209513 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:30.587508 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.587467 2566 generic.go:356] "Generic (PLEG): container finished" podID="7cfe5df2-6b7a-4450-ac95-66cb7846f082" containerID="d21221e236ae68a4a53ab08b10427a89ecbcd2c0c396fe57d48dedb4ddd20381" exitCode=0 Jul 12 08:47:30.588032 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.587525 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" Jul 12 08:47:30.588032 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.587570 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" event={"ID":"7cfe5df2-6b7a-4450-ac95-66cb7846f082","Type":"ContainerDied","Data":"d21221e236ae68a4a53ab08b10427a89ecbcd2c0c396fe57d48dedb4ddd20381"} Jul 12 08:47:30.588032 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.587661 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc" event={"ID":"7cfe5df2-6b7a-4450-ac95-66cb7846f082","Type":"ContainerDied","Data":"62f95458acfcc8cb679caa8f9537207eb99ef9a111d08c5bb724897657a8b5ac"} Jul 12 08:47:30.588032 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.587696 2566 scope.go:117] "RemoveContainer" containerID="d21221e236ae68a4a53ab08b10427a89ecbcd2c0c396fe57d48dedb4ddd20381" Jul 12 08:47:30.597181 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.597160 2566 scope.go:117] "RemoveContainer" containerID="d21221e236ae68a4a53ab08b10427a89ecbcd2c0c396fe57d48dedb4ddd20381" Jul 12 08:47:30.597437 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:47:30.597418 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d21221e236ae68a4a53ab08b10427a89ecbcd2c0c396fe57d48dedb4ddd20381\": container with ID starting with d21221e236ae68a4a53ab08b10427a89ecbcd2c0c396fe57d48dedb4ddd20381 not found: ID does not exist" containerID="d21221e236ae68a4a53ab08b10427a89ecbcd2c0c396fe57d48dedb4ddd20381" Jul 12 08:47:30.597494 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.597444 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d21221e236ae68a4a53ab08b10427a89ecbcd2c0c396fe57d48dedb4ddd20381"} err="failed to get container status \"d21221e236ae68a4a53ab08b10427a89ecbcd2c0c396fe57d48dedb4ddd20381\": rpc error: code = NotFound desc = could not find container \"d21221e236ae68a4a53ab08b10427a89ecbcd2c0c396fe57d48dedb4ddd20381\": container with ID starting with d21221e236ae68a4a53ab08b10427a89ecbcd2c0c396fe57d48dedb4ddd20381 not found: ID does not exist" Jul 12 08:47:30.608411 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.608389 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc"] Jul 12 08:47:30.613911 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:30.613888 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-94e99940-openshift-default-7666644d79-dmdcc"] Jul 12 08:47:31.027409 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:31.027374 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cfe5df2-6b7a-4450-ac95-66cb7846f082" path="/var/lib/kubelet/pods/7cfe5df2-6b7a-4450-ac95-66cb7846f082/volumes" Jul 12 08:47:31.586608 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:31.586574 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:37.224685 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:37.224642 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:37.332267 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:37.332234 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:37.427080 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:37.427050 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:37.526520 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:37.526494 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:47:37.907475 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:47:37.907396 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:48:03.853029 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:48:03.852998 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:48:07.121128 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:48:07.121091 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:48:07.172787 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:48:07.172758 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:48:09.510594 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:48:09.510487 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:48:09.568103 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:48:09.568070 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:48:09.645182 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:48:09.645150 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:48:25.559542 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:48:25.559505 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:48:31.210791 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:48:31.210752 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:48:31.315137 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:48:31.315103 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:48:31.411986 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:48:31.411955 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:48:38.197605 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:48:38.197569 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:48:41.906911 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:48:41.906873 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:48:42.709812 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:48:42.709780 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:48:48.339531 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:48:48.339492 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:48:48.394107 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:48:48.394073 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:48:48.470048 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:48:48.470009 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:48:48.568642 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:48:48.568602 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:48:48.677325 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:48:48.677237 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:48:49.397329 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:48:49.397288 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:48:49.493897 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:48:49.493860 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:48:49.590393 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:48:49.590359 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:48:51.166762 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:48:51.166725 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:48:56.791996 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:48:56.791964 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:48:56.892320 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:48:56.892282 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:48:57.459593 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:48:57.459543 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:03.359861 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:03.359824 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:03.460526 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:03.460495 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:03.589202 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:03.589164 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:03.865580 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:03.865536 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz"] Jul 12 08:49:03.866023 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:03.866001 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8571744e-a31f-417a-9d7f-a72b1e4792c8" containerName="maas-api" Jul 12 08:49:03.866023 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:03.866024 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8571744e-a31f-417a-9d7f-a72b1e4792c8" containerName="maas-api" Jul 12 08:49:03.866186 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:03.866059 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cfe5df2-6b7a-4450-ac95-66cb7846f082" containerName="istio-proxy" Jul 12 08:49:03.866186 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:03.866067 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cfe5df2-6b7a-4450-ac95-66cb7846f082" containerName="istio-proxy" Jul 12 08:49:03.866186 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:03.866177 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cfe5df2-6b7a-4450-ac95-66cb7846f082" containerName="istio-proxy" Jul 12 08:49:03.866352 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:03.866193 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8571744e-a31f-417a-9d7f-a72b1e4792c8" containerName="maas-api" Jul 12 08:49:03.869347 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:03.869328 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:03.871542 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:03.871512 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 12 08:49:03.871737 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:03.871722 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-d44bc757-openshift-default-dockercfg-zkf4x\"" Jul 12 08:49:03.888807 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:03.888783 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz"] Jul 12 08:49:03.891633 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:03.891606 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:04.015194 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:04.015162 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da6163ad-8525-45f2-9f09-7e03c7c97971-istio-token\") pod \"e2e-mt-d44bc757-openshift-default-548d778759-f4qkz\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:04.015384 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:04.015204 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da6163ad-8525-45f2-9f09-7e03c7c97971-workload-certs\") pod \"e2e-mt-d44bc757-openshift-default-548d778759-f4qkz\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:04.015384 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:04.015231 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da6163ad-8525-45f2-9f09-7e03c7c97971-workload-socket\") pod \"e2e-mt-d44bc757-openshift-default-548d778759-f4qkz\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:04.015384 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:04.015295 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da6163ad-8525-45f2-9f09-7e03c7c97971-istio-podinfo\") pod \"e2e-mt-d44bc757-openshift-default-548d778759-f4qkz\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:04.015384 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:04.015333 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tk8tr\" (UniqueName: \"kubernetes.io/projected/da6163ad-8525-45f2-9f09-7e03c7c97971-kube-api-access-tk8tr\") pod \"e2e-mt-d44bc757-openshift-default-548d778759-f4qkz\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:04.015384 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:04.015377 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da6163ad-8525-45f2-9f09-7e03c7c97971-istio-envoy\") pod \"e2e-mt-d44bc757-openshift-default-548d778759-f4qkz\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:04.015674 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:04.015409 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da6163ad-8525-45f2-9f09-7e03c7c97971-istio-data\") pod \"e2e-mt-d44bc757-openshift-default-548d778759-f4qkz\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:04.015674 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:04.015435 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da6163ad-8525-45f2-9f09-7e03c7c97971-istiod-ca-cert\") pod \"e2e-mt-d44bc757-openshift-default-548d778759-f4qkz\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:04.015674 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:04.015532 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da6163ad-8525-45f2-9f09-7e03c7c97971-credential-socket\") pod \"e2e-mt-d44bc757-openshift-default-548d778759-f4qkz\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:04.116737 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:04.116652 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da6163ad-8525-45f2-9f09-7e03c7c97971-credential-socket\") pod \"e2e-mt-d44bc757-openshift-default-548d778759-f4qkz\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:04.116737 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:04.116728 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da6163ad-8525-45f2-9f09-7e03c7c97971-istio-token\") pod \"e2e-mt-d44bc757-openshift-default-548d778759-f4qkz\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:04.116938 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:04.116847 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da6163ad-8525-45f2-9f09-7e03c7c97971-workload-certs\") pod \"e2e-mt-d44bc757-openshift-default-548d778759-f4qkz\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:04.116938 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:04.116908 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da6163ad-8525-45f2-9f09-7e03c7c97971-workload-socket\") pod \"e2e-mt-d44bc757-openshift-default-548d778759-f4qkz\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:04.117046 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:04.117013 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da6163ad-8525-45f2-9f09-7e03c7c97971-istio-podinfo\") pod \"e2e-mt-d44bc757-openshift-default-548d778759-f4qkz\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:04.117100 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:04.117052 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tk8tr\" (UniqueName: \"kubernetes.io/projected/da6163ad-8525-45f2-9f09-7e03c7c97971-kube-api-access-tk8tr\") pod \"e2e-mt-d44bc757-openshift-default-548d778759-f4qkz\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:04.117159 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:04.117116 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da6163ad-8525-45f2-9f09-7e03c7c97971-istio-envoy\") pod \"e2e-mt-d44bc757-openshift-default-548d778759-f4qkz\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:04.117159 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:04.117127 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da6163ad-8525-45f2-9f09-7e03c7c97971-credential-socket\") pod \"e2e-mt-d44bc757-openshift-default-548d778759-f4qkz\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:04.117269 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:04.117168 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da6163ad-8525-45f2-9f09-7e03c7c97971-istio-data\") pod \"e2e-mt-d44bc757-openshift-default-548d778759-f4qkz\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:04.117269 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:04.117198 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da6163ad-8525-45f2-9f09-7e03c7c97971-istiod-ca-cert\") pod \"e2e-mt-d44bc757-openshift-default-548d778759-f4qkz\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:04.117269 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:04.117206 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da6163ad-8525-45f2-9f09-7e03c7c97971-workload-socket\") pod \"e2e-mt-d44bc757-openshift-default-548d778759-f4qkz\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:04.117426 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:04.117264 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da6163ad-8525-45f2-9f09-7e03c7c97971-workload-certs\") pod \"e2e-mt-d44bc757-openshift-default-548d778759-f4qkz\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:04.117601 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:04.117575 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da6163ad-8525-45f2-9f09-7e03c7c97971-istio-data\") pod \"e2e-mt-d44bc757-openshift-default-548d778759-f4qkz\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:04.117906 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:04.117889 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da6163ad-8525-45f2-9f09-7e03c7c97971-istiod-ca-cert\") pod \"e2e-mt-d44bc757-openshift-default-548d778759-f4qkz\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:04.119657 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:04.119636 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da6163ad-8525-45f2-9f09-7e03c7c97971-istio-envoy\") pod \"e2e-mt-d44bc757-openshift-default-548d778759-f4qkz\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:04.119746 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:04.119698 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da6163ad-8525-45f2-9f09-7e03c7c97971-istio-podinfo\") pod \"e2e-mt-d44bc757-openshift-default-548d778759-f4qkz\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:04.124220 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:04.124195 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da6163ad-8525-45f2-9f09-7e03c7c97971-istio-token\") pod \"e2e-mt-d44bc757-openshift-default-548d778759-f4qkz\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:04.124453 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:04.124433 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tk8tr\" (UniqueName: \"kubernetes.io/projected/da6163ad-8525-45f2-9f09-7e03c7c97971-kube-api-access-tk8tr\") pod \"e2e-mt-d44bc757-openshift-default-548d778759-f4qkz\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:04.180862 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:04.180832 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:04.323825 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:04.323796 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz"] Jul 12 08:49:04.325701 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:49:04.325670 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda6163ad_8525_45f2_9f09_7e03c7c97971.slice/crio-a5bd1c0c38cade4271988067665689bb7767e040567ec752f0b2d413c28845e0 WatchSource:0}: Error finding container a5bd1c0c38cade4271988067665689bb7767e040567ec752f0b2d413c28845e0: Status 404 returned error can't find the container with id a5bd1c0c38cade4271988067665689bb7767e040567ec752f0b2d413c28845e0 Jul 12 08:49:04.330618 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:04.330581 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:49:04.330720 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:04.330660 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:49:04.330720 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:04.330700 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:49:04.349032 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:04.349008 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:05.054807 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:05.054772 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" event={"ID":"da6163ad-8525-45f2-9f09-7e03c7c97971","Type":"ContainerStarted","Data":"7348c3ed3bf245bd3efeed1f24e96c33a3c870cf6663fcde479d9b4d1749a60f"} Jul 12 08:49:05.055227 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:05.054851 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" event={"ID":"da6163ad-8525-45f2-9f09-7e03c7c97971","Type":"ContainerStarted","Data":"a5bd1c0c38cade4271988067665689bb7767e040567ec752f0b2d413c28845e0"} Jul 12 08:49:05.074185 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:05.074132 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" podStartSLOduration=2.074116739 podStartE2EDuration="2.074116739s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:49:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:49:05.071709543 +0000 UTC m=+1668.732505205" watchObservedRunningTime="2026-07-12 08:49:05.074116739 +0000 UTC m=+1668.734912401" Jul 12 08:49:05.181986 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:05.181955 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:05.187126 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:05.187099 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:06.059025 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:06.058995 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:06.059943 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:06.059923 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:06.973663 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:06.973628 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:07.050042 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:07.050006 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d44bc757-7f6b656979-gd86h"] Jul 12 08:49:07.054200 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:07.054174 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d44bc757-7f6b656979-gd86h" Jul 12 08:49:07.054532 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:07.054505 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:07.058121 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:07.058087 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-d44bc757-serving-cert\"" Jul 12 08:49:07.063434 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:07.063370 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d44bc757-7f6b656979-gd86h"] Jul 12 08:49:07.121425 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:07.121398 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:07.145164 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:07.145110 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dv87v\" (UniqueName: \"kubernetes.io/projected/ded5c016-60ea-423b-8733-5be68505d94b-kube-api-access-dv87v\") pod \"maas-api-e2e-mt-d44bc757-7f6b656979-gd86h\" (UID: \"ded5c016-60ea-423b-8733-5be68505d94b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d44bc757-7f6b656979-gd86h" Jul 12 08:49:07.145341 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:07.145277 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ded5c016-60ea-423b-8733-5be68505d94b-openshift-service-ca\") pod \"maas-api-e2e-mt-d44bc757-7f6b656979-gd86h\" (UID: \"ded5c016-60ea-423b-8733-5be68505d94b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d44bc757-7f6b656979-gd86h" Jul 12 08:49:07.145418 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:07.145373 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ded5c016-60ea-423b-8733-5be68505d94b-maas-api-tls\") pod \"maas-api-e2e-mt-d44bc757-7f6b656979-gd86h\" (UID: \"ded5c016-60ea-423b-8733-5be68505d94b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d44bc757-7f6b656979-gd86h" Jul 12 08:49:07.227135 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:07.227045 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:07.246721 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:07.246688 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ded5c016-60ea-423b-8733-5be68505d94b-openshift-service-ca\") pod \"maas-api-e2e-mt-d44bc757-7f6b656979-gd86h\" (UID: \"ded5c016-60ea-423b-8733-5be68505d94b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d44bc757-7f6b656979-gd86h" Jul 12 08:49:07.246882 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:07.246835 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ded5c016-60ea-423b-8733-5be68505d94b-maas-api-tls\") pod \"maas-api-e2e-mt-d44bc757-7f6b656979-gd86h\" (UID: \"ded5c016-60ea-423b-8733-5be68505d94b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d44bc757-7f6b656979-gd86h" Jul 12 08:49:07.246931 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:07.246905 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dv87v\" (UniqueName: \"kubernetes.io/projected/ded5c016-60ea-423b-8733-5be68505d94b-kube-api-access-dv87v\") pod \"maas-api-e2e-mt-d44bc757-7f6b656979-gd86h\" (UID: \"ded5c016-60ea-423b-8733-5be68505d94b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d44bc757-7f6b656979-gd86h" Jul 12 08:49:07.247370 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:07.247340 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ded5c016-60ea-423b-8733-5be68505d94b-openshift-service-ca\") pod \"maas-api-e2e-mt-d44bc757-7f6b656979-gd86h\" (UID: \"ded5c016-60ea-423b-8733-5be68505d94b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d44bc757-7f6b656979-gd86h" Jul 12 08:49:07.249644 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:07.249619 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ded5c016-60ea-423b-8733-5be68505d94b-maas-api-tls\") pod \"maas-api-e2e-mt-d44bc757-7f6b656979-gd86h\" (UID: \"ded5c016-60ea-423b-8733-5be68505d94b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d44bc757-7f6b656979-gd86h" Jul 12 08:49:07.255078 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:07.255053 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dv87v\" (UniqueName: \"kubernetes.io/projected/ded5c016-60ea-423b-8733-5be68505d94b-kube-api-access-dv87v\") pod \"maas-api-e2e-mt-d44bc757-7f6b656979-gd86h\" (UID: \"ded5c016-60ea-423b-8733-5be68505d94b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d44bc757-7f6b656979-gd86h" Jul 12 08:49:07.367949 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:07.367918 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d44bc757-7f6b656979-gd86h" Jul 12 08:49:07.539638 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:07.539604 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d44bc757-7f6b656979-gd86h"] Jul 12 08:49:07.547900 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:49:07.547866 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podded5c016_60ea_423b_8733_5be68505d94b.slice/crio-0c6d2822e787ea1616c860c546a4ca722274b6d2fffe885b33beeb4b7f907b26 WatchSource:0}: Error finding container 0c6d2822e787ea1616c860c546a4ca722274b6d2fffe885b33beeb4b7f907b26: Status 404 returned error can't find the container with id 0c6d2822e787ea1616c860c546a4ca722274b6d2fffe885b33beeb4b7f907b26 Jul 12 08:49:07.750310 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:07.750269 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:07.850673 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:07.850583 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:08.071522 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:08.071485 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d44bc757-7f6b656979-gd86h" event={"ID":"ded5c016-60ea-423b-8733-5be68505d94b","Type":"ContainerStarted","Data":"015c1de1a9fe5593d4d71bd86c6beacae5098faffcc4efd898efae0501a047c6"} Jul 12 08:49:08.071522 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:08.071556 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d44bc757-7f6b656979-gd86h" event={"ID":"ded5c016-60ea-423b-8733-5be68505d94b","Type":"ContainerStarted","Data":"0c6d2822e787ea1616c860c546a4ca722274b6d2fffe885b33beeb4b7f907b26"} Jul 12 08:49:08.072032 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:08.071788 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d44bc757-7f6b656979-gd86h" Jul 12 08:49:08.087574 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:08.087508 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d44bc757-7f6b656979-gd86h" podStartSLOduration=0.663032174 podStartE2EDuration="1.087494256s" totalImagesPullingTime="424.462082ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:49:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:49:08.087387852 +0000 UTC m=+1671.748183514" watchObservedRunningTime="2026-07-12 08:49:08.087494256 +0000 UTC m=+1671.748289917" Jul 12 08:49:11.466302 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:11.466266 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:12.124467 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:12.124431 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:12.225124 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:12.225087 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:12.542379 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:12.542315 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:14.081947 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:14.081918 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d44bc757-7f6b656979-gd86h" Jul 12 08:49:18.228572 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:18.228513 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:18.946964 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:18.946876 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d44bc757-7f6b656979-gd86h"] Jul 12 08:49:18.947153 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:18.947126 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d44bc757-7f6b656979-gd86h" podUID="ded5c016-60ea-423b-8733-5be68505d94b" containerName="maas-api" containerID="cri-o://015c1de1a9fe5593d4d71bd86c6beacae5098faffcc4efd898efae0501a047c6" gracePeriod=30 Jul 12 08:49:19.077989 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:19.077945 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d44bc757-7f6b656979-gd86h" podUID="ded5c016-60ea-423b-8733-5be68505d94b" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.55:8443/health\": dial tcp 10.132.0.55:8443: connect: connection refused" Jul 12 08:49:19.130815 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:19.130779 2566 generic.go:356] "Generic (PLEG): container finished" podID="ded5c016-60ea-423b-8733-5be68505d94b" containerID="015c1de1a9fe5593d4d71bd86c6beacae5098faffcc4efd898efae0501a047c6" exitCode=0 Jul 12 08:49:19.130979 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:19.130923 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d44bc757-7f6b656979-gd86h" event={"ID":"ded5c016-60ea-423b-8733-5be68505d94b","Type":"ContainerDied","Data":"015c1de1a9fe5593d4d71bd86c6beacae5098faffcc4efd898efae0501a047c6"} Jul 12 08:49:19.207252 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:19.207190 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d44bc757-7f6b656979-gd86h" Jul 12 08:49:19.264691 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:19.264661 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dv87v\" (UniqueName: \"kubernetes.io/projected/ded5c016-60ea-423b-8733-5be68505d94b-kube-api-access-dv87v\") pod \"ded5c016-60ea-423b-8733-5be68505d94b\" (UID: \"ded5c016-60ea-423b-8733-5be68505d94b\") " Jul 12 08:49:19.265160 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:19.264715 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ded5c016-60ea-423b-8733-5be68505d94b-maas-api-tls\") pod \"ded5c016-60ea-423b-8733-5be68505d94b\" (UID: \"ded5c016-60ea-423b-8733-5be68505d94b\") " Jul 12 08:49:19.265160 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:19.264753 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ded5c016-60ea-423b-8733-5be68505d94b-openshift-service-ca\") pod \"ded5c016-60ea-423b-8733-5be68505d94b\" (UID: \"ded5c016-60ea-423b-8733-5be68505d94b\") " Jul 12 08:49:19.265160 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:19.265149 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ded5c016-60ea-423b-8733-5be68505d94b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ded5c016-60ea-423b-8733-5be68505d94b" (UID: "ded5c016-60ea-423b-8733-5be68505d94b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 08:49:19.266891 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:19.266854 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ded5c016-60ea-423b-8733-5be68505d94b-kube-api-access-dv87v" (OuterVolumeSpecName: "kube-api-access-dv87v") pod "ded5c016-60ea-423b-8733-5be68505d94b" (UID: "ded5c016-60ea-423b-8733-5be68505d94b"). InnerVolumeSpecName "kube-api-access-dv87v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:49:19.266891 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:19.266874 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ded5c016-60ea-423b-8733-5be68505d94b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ded5c016-60ea-423b-8733-5be68505d94b" (UID: "ded5c016-60ea-423b-8733-5be68505d94b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 08:49:19.351644 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:19.351604 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:19.365857 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:19.365826 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dv87v\" (UniqueName: \"kubernetes.io/projected/ded5c016-60ea-423b-8733-5be68505d94b-kube-api-access-dv87v\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:49:19.365857 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:19.365859 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ded5c016-60ea-423b-8733-5be68505d94b-maas-api-tls\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:49:19.366035 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:19.365875 2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ded5c016-60ea-423b-8733-5be68505d94b-openshift-service-ca\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:49:20.140962 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:20.140930 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d44bc757-7f6b656979-gd86h" Jul 12 08:49:20.140962 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:20.140948 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d44bc757-7f6b656979-gd86h" event={"ID":"ded5c016-60ea-423b-8733-5be68505d94b","Type":"ContainerDied","Data":"0c6d2822e787ea1616c860c546a4ca722274b6d2fffe885b33beeb4b7f907b26"} Jul 12 08:49:20.141166 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:20.141013 2566 scope.go:117] "RemoveContainer" containerID="015c1de1a9fe5593d4d71bd86c6beacae5098faffcc4efd898efae0501a047c6" Jul 12 08:49:20.166781 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:20.166754 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d44bc757-7f6b656979-gd86h"] Jul 12 08:49:20.171085 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:20.171062 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d44bc757-7f6b656979-gd86h"] Jul 12 08:49:21.026476 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:21.026441 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ded5c016-60ea-423b-8733-5be68505d94b" path="/var/lib/kubelet/pods/ded5c016-60ea-423b-8733-5be68505d94b/volumes" Jul 12 08:49:21.307526 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:21.306667 2566 patch_prober.go:28] interesting pod/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": read tcp 10.132.0.2:46132->10.132.0.54:15021: read: connection reset by peer" start-of-body= Jul 12 08:49:21.307526 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:21.306721 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" podUID="da6163ad-8525-45f2-9f09-7e03c7c97971" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": read tcp 10.132.0.2:46132->10.132.0.54:15021: read: connection reset by peer" Jul 12 08:49:22.153165 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:22.153132 2566 generic.go:356] "Generic (PLEG): container finished" podID="da6163ad-8525-45f2-9f09-7e03c7c97971" containerID="7348c3ed3bf245bd3efeed1f24e96c33a3c870cf6663fcde479d9b4d1749a60f" exitCode=0 Jul 12 08:49:22.153655 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:22.153208 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" event={"ID":"da6163ad-8525-45f2-9f09-7e03c7c97971","Type":"ContainerDied","Data":"7348c3ed3bf245bd3efeed1f24e96c33a3c870cf6663fcde479d9b4d1749a60f"} Jul 12 08:49:22.153734 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:22.153677 2566 scope.go:117] "RemoveContainer" containerID="7348c3ed3bf245bd3efeed1f24e96c33a3c870cf6663fcde479d9b4d1749a60f" Jul 12 08:49:22.154717 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:22.154683 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:49:22.154838 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:22.154761 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:49:22.154838 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:22.154831 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:49:23.159805 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:23.159774 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" event={"ID":"da6163ad-8525-45f2-9f09-7e03c7c97971","Type":"ContainerStarted","Data":"5eec3245d6efa79636156060987406bdccc3d949823c05aafe95ab3c9fd2b766"} Jul 12 08:49:23.180944 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:23.180913 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:23.181204 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:23.181178 2566 patch_prober.go:28] interesting pod/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 12 08:49:23.181310 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:23.181226 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" podUID="da6163ad-8525-45f2-9f09-7e03c7c97971" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 12 08:49:24.181630 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:24.181601 2566 patch_prober.go:28] interesting pod/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 12 08:49:24.182016 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:24.181652 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" podUID="da6163ad-8525-45f2-9f09-7e03c7c97971" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 12 08:49:24.736996 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:24.736964 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:24.798990 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:24.798962 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:24.966412 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:24.966380 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz"] Jul 12 08:49:24.982059 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:24.982028 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:25.169044 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:25.168944 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" podUID="da6163ad-8525-45f2-9f09-7e03c7c97971" containerName="istio-proxy" containerID="cri-o://5eec3245d6efa79636156060987406bdccc3d949823c05aafe95ab3c9fd2b766" gracePeriod=30 Jul 12 08:49:25.328925 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:25.328888 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:25.426760 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:25.426675 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:25.534754 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:25.534715 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:31.011306 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:31.011272 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:31.111839 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:31.111805 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:31.211541 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:31.211505 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:31.348025 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:31.347935 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:32.130725 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.130698 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:32.202973 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.202890 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tk8tr\" (UniqueName: \"kubernetes.io/projected/da6163ad-8525-45f2-9f09-7e03c7c97971-kube-api-access-tk8tr\") pod \"da6163ad-8525-45f2-9f09-7e03c7c97971\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " Jul 12 08:49:32.202973 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.202929 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da6163ad-8525-45f2-9f09-7e03c7c97971-istio-data\") pod \"da6163ad-8525-45f2-9f09-7e03c7c97971\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " Jul 12 08:49:32.203250 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.203023 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da6163ad-8525-45f2-9f09-7e03c7c97971-istio-envoy\") pod \"da6163ad-8525-45f2-9f09-7e03c7c97971\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " Jul 12 08:49:32.203250 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.203051 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da6163ad-8525-45f2-9f09-7e03c7c97971-workload-certs\") pod \"da6163ad-8525-45f2-9f09-7e03c7c97971\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " Jul 12 08:49:32.203250 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.203079 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da6163ad-8525-45f2-9f09-7e03c7c97971-istiod-ca-cert\") pod \"da6163ad-8525-45f2-9f09-7e03c7c97971\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " Jul 12 08:49:32.203250 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.203126 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da6163ad-8525-45f2-9f09-7e03c7c97971-credential-socket\") pod \"da6163ad-8525-45f2-9f09-7e03c7c97971\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " Jul 12 08:49:32.203250 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.203155 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da6163ad-8525-45f2-9f09-7e03c7c97971-workload-socket\") pod \"da6163ad-8525-45f2-9f09-7e03c7c97971\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " Jul 12 08:49:32.203250 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.203187 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da6163ad-8525-45f2-9f09-7e03c7c97971-istio-token\") pod \"da6163ad-8525-45f2-9f09-7e03c7c97971\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " Jul 12 08:49:32.203250 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.203214 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da6163ad-8525-45f2-9f09-7e03c7c97971-istio-podinfo\") pod \"da6163ad-8525-45f2-9f09-7e03c7c97971\" (UID: \"da6163ad-8525-45f2-9f09-7e03c7c97971\") " Jul 12 08:49:32.203635 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.203457 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da6163ad-8525-45f2-9f09-7e03c7c97971-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "da6163ad-8525-45f2-9f09-7e03c7c97971" (UID: "da6163ad-8525-45f2-9f09-7e03c7c97971"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:49:32.203635 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.203537 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da6163ad-8525-45f2-9f09-7e03c7c97971-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "da6163ad-8525-45f2-9f09-7e03c7c97971" (UID: "da6163ad-8525-45f2-9f09-7e03c7c97971"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 08:49:32.203807 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.203743 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da6163ad-8525-45f2-9f09-7e03c7c97971-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "da6163ad-8525-45f2-9f09-7e03c7c97971" (UID: "da6163ad-8525-45f2-9f09-7e03c7c97971"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:49:32.203906 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.203882 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da6163ad-8525-45f2-9f09-7e03c7c97971-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "da6163ad-8525-45f2-9f09-7e03c7c97971" (UID: "da6163ad-8525-45f2-9f09-7e03c7c97971"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:49:32.204031 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.204011 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da6163ad-8525-45f2-9f09-7e03c7c97971-istio-data" (OuterVolumeSpecName: "istio-data") pod "da6163ad-8525-45f2-9f09-7e03c7c97971" (UID: "da6163ad-8525-45f2-9f09-7e03c7c97971"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:49:32.205352 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.205322 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da6163ad-8525-45f2-9f09-7e03c7c97971-kube-api-access-tk8tr" (OuterVolumeSpecName: "kube-api-access-tk8tr") pod "da6163ad-8525-45f2-9f09-7e03c7c97971" (UID: "da6163ad-8525-45f2-9f09-7e03c7c97971"). InnerVolumeSpecName "kube-api-access-tk8tr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:49:32.205479 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.205462 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da6163ad-8525-45f2-9f09-7e03c7c97971-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "da6163ad-8525-45f2-9f09-7e03c7c97971" (UID: "da6163ad-8525-45f2-9f09-7e03c7c97971"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:49:32.205866 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.205848 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da6163ad-8525-45f2-9f09-7e03c7c97971-istio-token" (OuterVolumeSpecName: "istio-token") pod "da6163ad-8525-45f2-9f09-7e03c7c97971" (UID: "da6163ad-8525-45f2-9f09-7e03c7c97971"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:49:32.205942 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.205886 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/da6163ad-8525-45f2-9f09-7e03c7c97971-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "da6163ad-8525-45f2-9f09-7e03c7c97971" (UID: "da6163ad-8525-45f2-9f09-7e03c7c97971"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 08:49:32.210763 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.210739 2566 generic.go:356] "Generic (PLEG): container finished" podID="da6163ad-8525-45f2-9f09-7e03c7c97971" containerID="5eec3245d6efa79636156060987406bdccc3d949823c05aafe95ab3c9fd2b766" exitCode=0 Jul 12 08:49:32.210865 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.210792 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" Jul 12 08:49:32.210865 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.210825 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" event={"ID":"da6163ad-8525-45f2-9f09-7e03c7c97971","Type":"ContainerDied","Data":"5eec3245d6efa79636156060987406bdccc3d949823c05aafe95ab3c9fd2b766"} Jul 12 08:49:32.210980 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.210890 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz" event={"ID":"da6163ad-8525-45f2-9f09-7e03c7c97971","Type":"ContainerDied","Data":"a5bd1c0c38cade4271988067665689bb7767e040567ec752f0b2d413c28845e0"} Jul 12 08:49:32.210980 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.210925 2566 scope.go:117] "RemoveContainer" containerID="5eec3245d6efa79636156060987406bdccc3d949823c05aafe95ab3c9fd2b766" Jul 12 08:49:32.226121 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.226103 2566 scope.go:117] "RemoveContainer" containerID="7348c3ed3bf245bd3efeed1f24e96c33a3c870cf6663fcde479d9b4d1749a60f" Jul 12 08:49:32.234089 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.234059 2566 scope.go:117] "RemoveContainer" containerID="5eec3245d6efa79636156060987406bdccc3d949823c05aafe95ab3c9fd2b766" Jul 12 08:49:32.234389 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:49:32.234368 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5eec3245d6efa79636156060987406bdccc3d949823c05aafe95ab3c9fd2b766\": container with ID starting with 5eec3245d6efa79636156060987406bdccc3d949823c05aafe95ab3c9fd2b766 not found: ID does not exist" containerID="5eec3245d6efa79636156060987406bdccc3d949823c05aafe95ab3c9fd2b766" Jul 12 08:49:32.234486 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.234401 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5eec3245d6efa79636156060987406bdccc3d949823c05aafe95ab3c9fd2b766"} err="failed to get container status \"5eec3245d6efa79636156060987406bdccc3d949823c05aafe95ab3c9fd2b766\": rpc error: code = NotFound desc = could not find container \"5eec3245d6efa79636156060987406bdccc3d949823c05aafe95ab3c9fd2b766\": container with ID starting with 5eec3245d6efa79636156060987406bdccc3d949823c05aafe95ab3c9fd2b766 not found: ID does not exist" Jul 12 08:49:32.234486 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.234428 2566 scope.go:117] "RemoveContainer" containerID="7348c3ed3bf245bd3efeed1f24e96c33a3c870cf6663fcde479d9b4d1749a60f" Jul 12 08:49:32.234725 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:49:32.234700 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7348c3ed3bf245bd3efeed1f24e96c33a3c870cf6663fcde479d9b4d1749a60f\": container with ID starting with 7348c3ed3bf245bd3efeed1f24e96c33a3c870cf6663fcde479d9b4d1749a60f not found: ID does not exist" containerID="7348c3ed3bf245bd3efeed1f24e96c33a3c870cf6663fcde479d9b4d1749a60f" Jul 12 08:49:32.234781 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.234733 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7348c3ed3bf245bd3efeed1f24e96c33a3c870cf6663fcde479d9b4d1749a60f"} err="failed to get container status \"7348c3ed3bf245bd3efeed1f24e96c33a3c870cf6663fcde479d9b4d1749a60f\": rpc error: code = NotFound desc = could not find container \"7348c3ed3bf245bd3efeed1f24e96c33a3c870cf6663fcde479d9b4d1749a60f\": container with ID starting with 7348c3ed3bf245bd3efeed1f24e96c33a3c870cf6663fcde479d9b4d1749a60f not found: ID does not exist" Jul 12 08:49:32.236291 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.236273 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz"] Jul 12 08:49:32.239215 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.239196 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-d44bc757-openshift-default-548d778759-f4qkz"] Jul 12 08:49:32.304798 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.304769 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da6163ad-8525-45f2-9f09-7e03c7c97971-istio-envoy\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:49:32.304798 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.304795 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da6163ad-8525-45f2-9f09-7e03c7c97971-workload-certs\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:49:32.304798 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.304804 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da6163ad-8525-45f2-9f09-7e03c7c97971-istiod-ca-cert\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:49:32.304998 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.304812 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da6163ad-8525-45f2-9f09-7e03c7c97971-credential-socket\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:49:32.304998 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.304820 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da6163ad-8525-45f2-9f09-7e03c7c97971-workload-socket\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:49:32.304998 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.304830 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da6163ad-8525-45f2-9f09-7e03c7c97971-istio-token\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:49:32.304998 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.304838 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da6163ad-8525-45f2-9f09-7e03c7c97971-istio-podinfo\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:49:32.304998 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.304845 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tk8tr\" (UniqueName: \"kubernetes.io/projected/da6163ad-8525-45f2-9f09-7e03c7c97971-kube-api-access-tk8tr\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:49:32.304998 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:32.304854 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da6163ad-8525-45f2-9f09-7e03c7c97971-istio-data\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:49:33.028049 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:33.028017 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da6163ad-8525-45f2-9f09-7e03c7c97971" path="/var/lib/kubelet/pods/da6163ad-8525-45f2-9f09-7e03c7c97971/volumes" Jul 12 08:49:36.992578 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:36.992520 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:37.093473 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:37.093437 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:43.801409 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:43.801374 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:44.113865 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.113776 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt"] Jul 12 08:49:44.114303 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.114285 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da6163ad-8525-45f2-9f09-7e03c7c97971" containerName="istio-proxy" Jul 12 08:49:44.114368 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.114305 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="da6163ad-8525-45f2-9f09-7e03c7c97971" containerName="istio-proxy" Jul 12 08:49:44.114368 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.114341 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ded5c016-60ea-423b-8733-5be68505d94b" containerName="maas-api" Jul 12 08:49:44.114368 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.114347 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ded5c016-60ea-423b-8733-5be68505d94b" containerName="maas-api" Jul 12 08:49:44.114466 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.114404 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="da6163ad-8525-45f2-9f09-7e03c7c97971" containerName="istio-proxy" Jul 12 08:49:44.114466 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.114414 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ded5c016-60ea-423b-8733-5be68505d94b" containerName="maas-api" Jul 12 08:49:44.119117 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.119101 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:49:44.121364 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.121339 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 12 08:49:44.121453 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.121341 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-2828b0e9-openshift-default-dockercfg-6jb9v\"" Jul 12 08:49:44.127178 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.126960 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt"] Jul 12 08:49:44.133031 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.132996 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:44.209991 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.209957 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32b94059-edac-4aaa-8880-ec812dc7a67c-credential-socket\") pod \"e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:49:44.210164 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.210012 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32b94059-edac-4aaa-8880-ec812dc7a67c-workload-certs\") pod \"e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:49:44.210164 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.210109 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32b94059-edac-4aaa-8880-ec812dc7a67c-istio-envoy\") pod \"e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:49:44.210164 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.210145 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32b94059-edac-4aaa-8880-ec812dc7a67c-istio-data\") pod \"e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:49:44.210288 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.210198 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32b94059-edac-4aaa-8880-ec812dc7a67c-workload-socket\") pod \"e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:49:44.210288 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.210247 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32b94059-edac-4aaa-8880-ec812dc7a67c-istiod-ca-cert\") pod \"e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:49:44.210288 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.210267 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrhtl\" (UniqueName: \"kubernetes.io/projected/32b94059-edac-4aaa-8880-ec812dc7a67c-kube-api-access-xrhtl\") pod \"e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:49:44.210425 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.210287 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32b94059-edac-4aaa-8880-ec812dc7a67c-istio-podinfo\") pod \"e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:49:44.210425 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.210304 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32b94059-edac-4aaa-8880-ec812dc7a67c-istio-token\") pod \"e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:49:44.311260 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.311216 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32b94059-edac-4aaa-8880-ec812dc7a67c-istiod-ca-cert\") pod \"e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:49:44.311510 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.311490 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrhtl\" (UniqueName: \"kubernetes.io/projected/32b94059-edac-4aaa-8880-ec812dc7a67c-kube-api-access-xrhtl\") pod \"e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:49:44.312063 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.312043 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32b94059-edac-4aaa-8880-ec812dc7a67c-istio-podinfo\") pod \"e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:49:44.312215 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.312200 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32b94059-edac-4aaa-8880-ec812dc7a67c-istio-token\") pod \"e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:49:44.312403 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.312253 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32b94059-edac-4aaa-8880-ec812dc7a67c-istiod-ca-cert\") pod \"e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:49:44.312504 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.312491 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32b94059-edac-4aaa-8880-ec812dc7a67c-credential-socket\") pod \"e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:49:44.312671 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.312657 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32b94059-edac-4aaa-8880-ec812dc7a67c-workload-certs\") pod \"e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:49:44.312805 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.312792 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32b94059-edac-4aaa-8880-ec812dc7a67c-istio-envoy\") pod \"e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:49:44.312891 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.312866 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32b94059-edac-4aaa-8880-ec812dc7a67c-credential-socket\") pod \"e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:49:44.312968 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.312885 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32b94059-edac-4aaa-8880-ec812dc7a67c-istio-data\") pod \"e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:49:44.313068 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.313056 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32b94059-edac-4aaa-8880-ec812dc7a67c-workload-socket\") pod \"e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:49:44.313180 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.313099 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32b94059-edac-4aaa-8880-ec812dc7a67c-workload-certs\") pod \"e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:49:44.313524 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.313506 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32b94059-edac-4aaa-8880-ec812dc7a67c-istio-data\") pod \"e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:49:44.313659 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.313572 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32b94059-edac-4aaa-8880-ec812dc7a67c-workload-socket\") pod \"e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:49:44.315856 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.315832 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32b94059-edac-4aaa-8880-ec812dc7a67c-istio-envoy\") pod \"e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:49:44.319045 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.319025 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32b94059-edac-4aaa-8880-ec812dc7a67c-istio-podinfo\") pod \"e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:49:44.321227 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.321197 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrhtl\" (UniqueName: \"kubernetes.io/projected/32b94059-edac-4aaa-8880-ec812dc7a67c-kube-api-access-xrhtl\") pod \"e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:49:44.322571 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.322463 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32b94059-edac-4aaa-8880-ec812dc7a67c-istio-token\") pod \"e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:49:44.431095 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.431007 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:49:44.481001 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.480965 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:44.584620 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.584591 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt"] Jul 12 08:49:44.587455 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:49:44.587426 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32b94059_edac_4aaa_8880_ec812dc7a67c.slice/crio-3f9d747191c0a039f8c4fca4a281fe55f8118d70866b17134bc177809a1af342 WatchSource:0}: Error finding container 3f9d747191c0a039f8c4fca4a281fe55f8118d70866b17134bc177809a1af342: Status 404 returned error can't find the container with id 3f9d747191c0a039f8c4fca4a281fe55f8118d70866b17134bc177809a1af342 Jul 12 08:49:44.589665 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.589635 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:49:44.589762 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.589711 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:49:44.589824 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:44.589795 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:49:45.283447 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:45.283410 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" event={"ID":"32b94059-edac-4aaa-8880-ec812dc7a67c","Type":"ContainerStarted","Data":"d2f68d46f99930fb87accde7ac4751df5a3e5fcfa423419d09c773243ea31190"} Jul 12 08:49:45.283850 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:45.283462 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" event={"ID":"32b94059-edac-4aaa-8880-ec812dc7a67c","Type":"ContainerStarted","Data":"3f9d747191c0a039f8c4fca4a281fe55f8118d70866b17134bc177809a1af342"} Jul 12 08:49:45.302570 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:45.302501 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" podStartSLOduration=1.302484576 podStartE2EDuration="1.302484576s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:49:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:49:45.300312805 +0000 UTC m=+1708.961108458" watchObservedRunningTime="2026-07-12 08:49:45.302484576 +0000 UTC m=+1708.963280238" Jul 12 08:49:45.431867 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:45.431832 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:49:45.436785 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:45.436761 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:49:46.288940 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:46.288910 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:49:46.290049 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:46.290031 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:49:47.184702 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:47.184671 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:47.266012 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:47.265956 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:47.333013 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:47.332987 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:47.741622 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:47.741587 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:47.840364 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:47.840331 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:49:51.660237 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:49:51.660188 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:50:01.290098 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:50:01.290062 2566 patch_prober.go:28] interesting pod/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 12 08:50:01.290502 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:50:01.290131 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" podUID="32b94059-edac-4aaa-8880-ec812dc7a67c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 12 08:50:01.367883 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:50:01.367848 2566 generic.go:356] "Generic (PLEG): container finished" podID="32b94059-edac-4aaa-8880-ec812dc7a67c" containerID="d2f68d46f99930fb87accde7ac4751df5a3e5fcfa423419d09c773243ea31190" exitCode=0 Jul 12 08:50:01.368025 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:50:01.367912 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" event={"ID":"32b94059-edac-4aaa-8880-ec812dc7a67c","Type":"ContainerDied","Data":"d2f68d46f99930fb87accde7ac4751df5a3e5fcfa423419d09c773243ea31190"} Jul 12 08:50:01.368282 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:50:01.368266 2566 scope.go:117] "RemoveContainer" containerID="d2f68d46f99930fb87accde7ac4751df5a3e5fcfa423419d09c773243ea31190" Jul 12 08:50:01.369240 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:50:01.369209 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:50:01.369338 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:50:01.369282 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:50:01.369338 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:50:01.369323 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:50:02.374894 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:50:02.374859 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" event={"ID":"32b94059-edac-4aaa-8880-ec812dc7a67c","Type":"ContainerStarted","Data":"4980fffda9000488c0ee9f9e6f2436a51ef748fb194ab84446fb1a4d8a1b512c"} Jul 12 08:50:02.431610 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:50:02.431567 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:50:02.431786 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:50:02.431757 2566 patch_prober.go:28] interesting pod/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 12 08:50:02.431837 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:50:02.431806 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" podUID="32b94059-edac-4aaa-8880-ec812dc7a67c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 12 08:50:03.431835 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:50:03.431796 2566 patch_prober.go:28] interesting pod/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 12 08:50:03.432407 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:50:03.432377 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" podUID="32b94059-edac-4aaa-8880-ec812dc7a67c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 12 08:50:04.431761 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:50:04.431720 2566 patch_prober.go:28] interesting pod/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 12 08:50:04.432435 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:50:04.432404 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" podUID="32b94059-edac-4aaa-8880-ec812dc7a67c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 12 08:50:05.432052 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:50:05.431604 2566 patch_prober.go:28] interesting pod/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 12 08:50:05.432052 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:50:05.431678 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" podUID="32b94059-edac-4aaa-8880-ec812dc7a67c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 12 08:50:06.431515 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:50:06.431470 2566 patch_prober.go:28] interesting pod/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 12 08:50:06.432016 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:50:06.431537 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" podUID="32b94059-edac-4aaa-8880-ec812dc7a67c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 12 08:50:07.435774 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:50:07.435743 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:50:07.436171 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:50:07.436001 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:50:07.436743 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:50:07.436722 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:50:12.150633 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:50:12.150600 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:50:28.088388 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:50:28.088346 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:50:28.201333 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:50:28.201301 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:50:34.430332 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:50:34.430299 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:50:34.858134 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:50:34.858103 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:50:34.958953 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:50:34.958918 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:50:35.123959 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:50:35.123878 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:50:39.743014 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:50:39.742980 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:50:40.487982 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:50:40.487952 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:50:45.568494 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:50:45.568463 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:51:16.888622 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:16.888593 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-k2ltl_bc3f8e97-c9ab-4370-b437-8a47514c869c/console-operator/1.log" Jul 12 08:51:16.894379 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:16.894347 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lfbsq_abe468eb-f6d3-4af4-bf5e-a3c029a404ab/ovn-acl-logging/0.log" Jul 12 08:51:16.896747 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:16.896731 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-k2ltl_bc3f8e97-c9ab-4370-b437-8a47514c869c/console-operator/1.log" Jul 12 08:51:16.902000 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:16.901983 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lfbsq_abe468eb-f6d3-4af4-bf5e-a3c029a404ab/ovn-acl-logging/0.log" Jul 12 08:51:21.875397 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:21.875366 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:51:22.548761 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:22.548723 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:51:27.942842 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:27.942813 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:51:28.020663 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:28.020627 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:51:28.185428 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:28.185388 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt"] Jul 12 08:51:28.185734 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:28.185702 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" podUID="32b94059-edac-4aaa-8880-ec812dc7a67c" containerName="istio-proxy" containerID="cri-o://4980fffda9000488c0ee9f9e6f2436a51ef748fb194ab84446fb1a4d8a1b512c" gracePeriod=30 Jul 12 08:51:28.202889 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:28.202826 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:51:29.433974 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:29.433936 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:51:29.549192 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:29.549158 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:51:29.657937 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:29.657904 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:51:30.344282 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:30.344248 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:51:30.439049 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:30.439017 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:51:30.539024 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:30.538990 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:51:31.913433 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:31.913400 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:51:33.443837 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.443817 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:51:33.560026 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.559987 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32b94059-edac-4aaa-8880-ec812dc7a67c-workload-certs\") pod \"32b94059-edac-4aaa-8880-ec812dc7a67c\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " Jul 12 08:51:33.560221 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.560041 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32b94059-edac-4aaa-8880-ec812dc7a67c-istio-envoy\") pod \"32b94059-edac-4aaa-8880-ec812dc7a67c\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " Jul 12 08:51:33.560221 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.560063 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32b94059-edac-4aaa-8880-ec812dc7a67c-istio-data\") pod \"32b94059-edac-4aaa-8880-ec812dc7a67c\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " Jul 12 08:51:33.560221 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.560084 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32b94059-edac-4aaa-8880-ec812dc7a67c-istio-token\") pod \"32b94059-edac-4aaa-8880-ec812dc7a67c\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " Jul 12 08:51:33.560221 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.560106 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32b94059-edac-4aaa-8880-ec812dc7a67c-istio-podinfo\") pod \"32b94059-edac-4aaa-8880-ec812dc7a67c\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " Jul 12 08:51:33.560221 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.560179 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32b94059-edac-4aaa-8880-ec812dc7a67c-workload-socket\") pod \"32b94059-edac-4aaa-8880-ec812dc7a67c\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " Jul 12 08:51:33.560497 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.560235 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32b94059-edac-4aaa-8880-ec812dc7a67c-credential-socket\") pod \"32b94059-edac-4aaa-8880-ec812dc7a67c\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " Jul 12 08:51:33.560497 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.560261 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xrhtl\" (UniqueName: \"kubernetes.io/projected/32b94059-edac-4aaa-8880-ec812dc7a67c-kube-api-access-xrhtl\") pod \"32b94059-edac-4aaa-8880-ec812dc7a67c\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " Jul 12 08:51:33.560497 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.560283 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32b94059-edac-4aaa-8880-ec812dc7a67c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "32b94059-edac-4aaa-8880-ec812dc7a67c" (UID: "32b94059-edac-4aaa-8880-ec812dc7a67c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:51:33.560497 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.560319 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32b94059-edac-4aaa-8880-ec812dc7a67c-istiod-ca-cert\") pod \"32b94059-edac-4aaa-8880-ec812dc7a67c\" (UID: \"32b94059-edac-4aaa-8880-ec812dc7a67c\") " Jul 12 08:51:33.560737 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.560489 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32b94059-edac-4aaa-8880-ec812dc7a67c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "32b94059-edac-4aaa-8880-ec812dc7a67c" (UID: "32b94059-edac-4aaa-8880-ec812dc7a67c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:51:33.560737 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.560636 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32b94059-edac-4aaa-8880-ec812dc7a67c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "32b94059-edac-4aaa-8880-ec812dc7a67c" (UID: "32b94059-edac-4aaa-8880-ec812dc7a67c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:51:33.560737 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.560679 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32b94059-edac-4aaa-8880-ec812dc7a67c-credential-socket\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:51:33.560737 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.560698 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32b94059-edac-4aaa-8880-ec812dc7a67c-workload-certs\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:51:33.560919 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.560890 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32b94059-edac-4aaa-8880-ec812dc7a67c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "32b94059-edac-4aaa-8880-ec812dc7a67c" (UID: "32b94059-edac-4aaa-8880-ec812dc7a67c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 08:51:33.560977 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.560923 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32b94059-edac-4aaa-8880-ec812dc7a67c-istio-data" (OuterVolumeSpecName: "istio-data") pod "32b94059-edac-4aaa-8880-ec812dc7a67c" (UID: "32b94059-edac-4aaa-8880-ec812dc7a67c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:51:33.562346 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.562322 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32b94059-edac-4aaa-8880-ec812dc7a67c-istio-token" (OuterVolumeSpecName: "istio-token") pod "32b94059-edac-4aaa-8880-ec812dc7a67c" (UID: "32b94059-edac-4aaa-8880-ec812dc7a67c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:51:33.562606 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.562583 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/32b94059-edac-4aaa-8880-ec812dc7a67c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "32b94059-edac-4aaa-8880-ec812dc7a67c" (UID: "32b94059-edac-4aaa-8880-ec812dc7a67c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 08:51:33.562791 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.562774 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32b94059-edac-4aaa-8880-ec812dc7a67c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "32b94059-edac-4aaa-8880-ec812dc7a67c" (UID: "32b94059-edac-4aaa-8880-ec812dc7a67c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:51:33.562860 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.562788 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32b94059-edac-4aaa-8880-ec812dc7a67c-kube-api-access-xrhtl" (OuterVolumeSpecName: "kube-api-access-xrhtl") pod "32b94059-edac-4aaa-8880-ec812dc7a67c" (UID: "32b94059-edac-4aaa-8880-ec812dc7a67c"). InnerVolumeSpecName "kube-api-access-xrhtl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:51:33.661783 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.661752 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrhtl\" (UniqueName: \"kubernetes.io/projected/32b94059-edac-4aaa-8880-ec812dc7a67c-kube-api-access-xrhtl\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:51:33.661783 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.661777 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32b94059-edac-4aaa-8880-ec812dc7a67c-istiod-ca-cert\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:51:33.661783 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.661787 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32b94059-edac-4aaa-8880-ec812dc7a67c-istio-envoy\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:51:33.661783 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.661795 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32b94059-edac-4aaa-8880-ec812dc7a67c-istio-data\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:51:33.662019 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.661803 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32b94059-edac-4aaa-8880-ec812dc7a67c-istio-token\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:51:33.662019 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.661811 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32b94059-edac-4aaa-8880-ec812dc7a67c-istio-podinfo\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:51:33.662019 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.661820 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32b94059-edac-4aaa-8880-ec812dc7a67c-workload-socket\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:51:33.816280 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.816188 2566 generic.go:356] "Generic (PLEG): container finished" podID="32b94059-edac-4aaa-8880-ec812dc7a67c" containerID="4980fffda9000488c0ee9f9e6f2436a51ef748fb194ab84446fb1a4d8a1b512c" exitCode=0 Jul 12 08:51:33.816280 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.816243 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" Jul 12 08:51:33.816512 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.816268 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" event={"ID":"32b94059-edac-4aaa-8880-ec812dc7a67c","Type":"ContainerDied","Data":"4980fffda9000488c0ee9f9e6f2436a51ef748fb194ab84446fb1a4d8a1b512c"} Jul 12 08:51:33.816512 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.816365 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt" event={"ID":"32b94059-edac-4aaa-8880-ec812dc7a67c","Type":"ContainerDied","Data":"3f9d747191c0a039f8c4fca4a281fe55f8118d70866b17134bc177809a1af342"} Jul 12 08:51:33.816512 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.816399 2566 scope.go:117] "RemoveContainer" containerID="4980fffda9000488c0ee9f9e6f2436a51ef748fb194ab84446fb1a4d8a1b512c" Jul 12 08:51:33.829831 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.829812 2566 scope.go:117] "RemoveContainer" containerID="d2f68d46f99930fb87accde7ac4751df5a3e5fcfa423419d09c773243ea31190" Jul 12 08:51:33.838053 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.838033 2566 scope.go:117] "RemoveContainer" containerID="4980fffda9000488c0ee9f9e6f2436a51ef748fb194ab84446fb1a4d8a1b512c" Jul 12 08:51:33.838334 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:51:33.838311 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4980fffda9000488c0ee9f9e6f2436a51ef748fb194ab84446fb1a4d8a1b512c\": container with ID starting with 4980fffda9000488c0ee9f9e6f2436a51ef748fb194ab84446fb1a4d8a1b512c not found: ID does not exist" containerID="4980fffda9000488c0ee9f9e6f2436a51ef748fb194ab84446fb1a4d8a1b512c" Jul 12 08:51:33.838412 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.838346 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4980fffda9000488c0ee9f9e6f2436a51ef748fb194ab84446fb1a4d8a1b512c"} err="failed to get container status \"4980fffda9000488c0ee9f9e6f2436a51ef748fb194ab84446fb1a4d8a1b512c\": rpc error: code = NotFound desc = could not find container \"4980fffda9000488c0ee9f9e6f2436a51ef748fb194ab84446fb1a4d8a1b512c\": container with ID starting with 4980fffda9000488c0ee9f9e6f2436a51ef748fb194ab84446fb1a4d8a1b512c not found: ID does not exist" Jul 12 08:51:33.838412 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.838369 2566 scope.go:117] "RemoveContainer" containerID="d2f68d46f99930fb87accde7ac4751df5a3e5fcfa423419d09c773243ea31190" Jul 12 08:51:33.838638 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:51:33.838618 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2f68d46f99930fb87accde7ac4751df5a3e5fcfa423419d09c773243ea31190\": container with ID starting with d2f68d46f99930fb87accde7ac4751df5a3e5fcfa423419d09c773243ea31190 not found: ID does not exist" containerID="d2f68d46f99930fb87accde7ac4751df5a3e5fcfa423419d09c773243ea31190" Jul 12 08:51:33.838704 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.838645 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2f68d46f99930fb87accde7ac4751df5a3e5fcfa423419d09c773243ea31190"} err="failed to get container status \"d2f68d46f99930fb87accde7ac4751df5a3e5fcfa423419d09c773243ea31190\": rpc error: code = NotFound desc = could not find container \"d2f68d46f99930fb87accde7ac4751df5a3e5fcfa423419d09c773243ea31190\": container with ID starting with d2f68d46f99930fb87accde7ac4751df5a3e5fcfa423419d09c773243ea31190 not found: ID does not exist" Jul 12 08:51:33.839823 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.839801 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt"] Jul 12 08:51:33.842893 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:33.842870 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-2828b0e9-openshift-default-7b76bd7975-cnkrt"] Jul 12 08:51:35.028161 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:35.028130 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32b94059-edac-4aaa-8880-ec812dc7a67c" path="/var/lib/kubelet/pods/32b94059-edac-4aaa-8880-ec812dc7a67c/volumes" Jul 12 08:51:37.442301 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:37.442264 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:51:37.542001 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:37.541964 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:51:38.130830 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:38.130799 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:51:43.739492 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:43.739444 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:51:43.841102 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:43.841073 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:51:43.942738 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:43.942706 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:51:54.480330 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:54.480294 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:51:54.816702 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:51:54.816672 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:52:15.750647 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:15.750612 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:52:21.343142 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:21.343105 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:52:21.443332 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:21.443297 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:52:21.688360 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:21.688273 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:52:21.976868 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:21.976783 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k"] Jul 12 08:52:21.977173 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:21.977160 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32b94059-edac-4aaa-8880-ec812dc7a67c" containerName="istio-proxy" Jul 12 08:52:21.977236 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:21.977173 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="32b94059-edac-4aaa-8880-ec812dc7a67c" containerName="istio-proxy" Jul 12 08:52:21.977236 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:21.977185 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32b94059-edac-4aaa-8880-ec812dc7a67c" containerName="istio-proxy" Jul 12 08:52:21.977236 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:21.977191 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="32b94059-edac-4aaa-8880-ec812dc7a67c" containerName="istio-proxy" Jul 12 08:52:21.977236 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:21.977213 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da6163ad-8525-45f2-9f09-7e03c7c97971" containerName="istio-proxy" Jul 12 08:52:21.977236 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:21.977219 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="da6163ad-8525-45f2-9f09-7e03c7c97971" containerName="istio-proxy" Jul 12 08:52:21.977480 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:21.977292 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="32b94059-edac-4aaa-8880-ec812dc7a67c" containerName="istio-proxy" Jul 12 08:52:21.977480 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:21.977300 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="da6163ad-8525-45f2-9f09-7e03c7c97971" containerName="istio-proxy" Jul 12 08:52:21.977480 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:21.977305 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="32b94059-edac-4aaa-8880-ec812dc7a67c" containerName="istio-proxy" Jul 12 08:52:21.980268 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:21.980252 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:21.982286 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:21.982267 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 12 08:52:21.982286 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:21.982273 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-d8ca53-openshift-default-dockercfg-x7ltp\"" Jul 12 08:52:21.991007 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:21.990983 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k"] Jul 12 08:52:22.001077 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:22.001051 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:52:22.057806 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:22.057782 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:52:22.157308 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:22.157274 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7kzr\" (UniqueName: \"kubernetes.io/projected/70616a20-0d42-433e-9e54-e8516c4ce886-kube-api-access-g7kzr\") pod \"e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:22.157467 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:22.157315 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70616a20-0d42-433e-9e54-e8516c4ce886-istio-envoy\") pod \"e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:22.157467 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:22.157348 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70616a20-0d42-433e-9e54-e8516c4ce886-credential-socket\") pod \"e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:22.157467 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:22.157396 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70616a20-0d42-433e-9e54-e8516c4ce886-workload-socket\") pod \"e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:22.157467 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:22.157419 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70616a20-0d42-433e-9e54-e8516c4ce886-istio-data\") pod \"e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:22.157717 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:22.157481 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70616a20-0d42-433e-9e54-e8516c4ce886-workload-certs\") pod \"e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:22.157717 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:22.157504 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70616a20-0d42-433e-9e54-e8516c4ce886-istiod-ca-cert\") pod \"e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:22.157717 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:22.157531 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70616a20-0d42-433e-9e54-e8516c4ce886-istio-podinfo\") pod \"e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:22.157717 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:22.157583 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70616a20-0d42-433e-9e54-e8516c4ce886-istio-token\") pod \"e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:22.258899 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:22.258866 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7kzr\" (UniqueName: \"kubernetes.io/projected/70616a20-0d42-433e-9e54-e8516c4ce886-kube-api-access-g7kzr\") pod \"e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:22.258899 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:22.258901 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70616a20-0d42-433e-9e54-e8516c4ce886-istio-envoy\") pod \"e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:22.259158 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:22.258927 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70616a20-0d42-433e-9e54-e8516c4ce886-credential-socket\") pod \"e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:22.259158 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:22.258963 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70616a20-0d42-433e-9e54-e8516c4ce886-workload-socket\") pod \"e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:22.259158 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:22.258983 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70616a20-0d42-433e-9e54-e8516c4ce886-istio-data\") pod \"e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:22.259158 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:22.259021 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70616a20-0d42-433e-9e54-e8516c4ce886-workload-certs\") pod \"e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:22.259158 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:22.259043 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70616a20-0d42-433e-9e54-e8516c4ce886-istiod-ca-cert\") pod \"e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:22.259158 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:22.259070 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70616a20-0d42-433e-9e54-e8516c4ce886-istio-podinfo\") pod \"e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:22.259158 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:22.259086 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70616a20-0d42-433e-9e54-e8516c4ce886-istio-token\") pod \"e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:22.259541 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:22.259401 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70616a20-0d42-433e-9e54-e8516c4ce886-workload-socket\") pod \"e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:22.259541 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:22.259421 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70616a20-0d42-433e-9e54-e8516c4ce886-credential-socket\") pod \"e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:22.259541 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:22.259499 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70616a20-0d42-433e-9e54-e8516c4ce886-istio-data\") pod \"e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:22.259737 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:22.259651 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70616a20-0d42-433e-9e54-e8516c4ce886-workload-certs\") pod \"e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:22.259836 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:22.259815 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70616a20-0d42-433e-9e54-e8516c4ce886-istiod-ca-cert\") pod \"e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:22.261272 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:22.261249 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70616a20-0d42-433e-9e54-e8516c4ce886-istio-envoy\") pod \"e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:22.261747 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:22.261726 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70616a20-0d42-433e-9e54-e8516c4ce886-istio-podinfo\") pod \"e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:22.266131 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:22.266109 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70616a20-0d42-433e-9e54-e8516c4ce886-istio-token\") pod \"e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:22.266499 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:22.266479 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7kzr\" (UniqueName: \"kubernetes.io/projected/70616a20-0d42-433e-9e54-e8516c4ce886-kube-api-access-g7kzr\") pod \"e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:22.291062 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:22.291035 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:22.430788 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:22.430760 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k"] Jul 12 08:52:22.432479 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:52:22.432453 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70616a20_0d42_433e_9e54_e8516c4ce886.slice/crio-f1a9a87a09122a99deef883867190c58ff0c52beae30cf8d5c57fc0d2251e3e2 WatchSource:0}: Error finding container f1a9a87a09122a99deef883867190c58ff0c52beae30cf8d5c57fc0d2251e3e2: Status 404 returned error can't find the container with id f1a9a87a09122a99deef883867190c58ff0c52beae30cf8d5c57fc0d2251e3e2 Jul 12 08:52:22.434736 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:22.434699 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:52:22.434821 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:22.434766 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:52:22.434821 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:22.434800 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:52:23.058098 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:23.058064 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" event={"ID":"70616a20-0d42-433e-9e54-e8516c4ce886","Type":"ContainerStarted","Data":"7d3908e75ddce9176c02426ebd8a3a3d74f8b705add547cc99ef78d02aa3dd3d"} Jul 12 08:52:23.058281 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:23.058114 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" event={"ID":"70616a20-0d42-433e-9e54-e8516c4ce886","Type":"ContainerStarted","Data":"f1a9a87a09122a99deef883867190c58ff0c52beae30cf8d5c57fc0d2251e3e2"} Jul 12 08:52:23.078018 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:23.077971 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" podStartSLOduration=2.077956511 podStartE2EDuration="2.077956511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:52:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:52:23.076167816 +0000 UTC m=+1866.736963479" watchObservedRunningTime="2026-07-12 08:52:23.077956511 +0000 UTC m=+1866.738752172" Jul 12 08:52:23.291383 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:23.291346 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:23.296182 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:23.296161 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:24.063418 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:24.063386 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:24.064306 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:24.064285 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:24.960817 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:24.960788 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:52:25.035352 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:25.035321 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:52:25.101665 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:25.101633 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:52:25.220466 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:25.220402 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:52:25.635206 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:25.635177 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:52:25.731697 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:25.731664 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:52:29.469354 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:29.469318 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:52:29.792485 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:29.792437 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:52:30.063966 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.063887 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2"] Jul 12 08:52:30.067611 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.067589 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:30.069929 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.069909 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-b534df-openshift-default-dockercfg-x4k8v\"" Jul 12 08:52:30.078763 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.078743 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2"] Jul 12 08:52:30.096051 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.096026 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:52:30.223649 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.223609 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12cdaa3d-49a5-439f-8502-ec498e23017c-istio-podinfo\") pod \"e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:30.223649 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.223648 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12cdaa3d-49a5-439f-8502-ec498e23017c-istiod-ca-cert\") pod \"e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:30.223886 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.223691 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12cdaa3d-49a5-439f-8502-ec498e23017c-istio-token\") pod \"e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:30.223886 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.223720 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12cdaa3d-49a5-439f-8502-ec498e23017c-istio-envoy\") pod \"e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:30.223886 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.223764 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12cdaa3d-49a5-439f-8502-ec498e23017c-workload-certs\") pod \"e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:30.223886 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.223809 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qhcm\" (UniqueName: \"kubernetes.io/projected/12cdaa3d-49a5-439f-8502-ec498e23017c-kube-api-access-5qhcm\") pod \"e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:30.223886 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.223874 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12cdaa3d-49a5-439f-8502-ec498e23017c-workload-socket\") pod \"e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:30.224082 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.223900 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12cdaa3d-49a5-439f-8502-ec498e23017c-istio-data\") pod \"e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:30.224082 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.223933 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12cdaa3d-49a5-439f-8502-ec498e23017c-credential-socket\") pod \"e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:30.324576 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.324480 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12cdaa3d-49a5-439f-8502-ec498e23017c-workload-certs\") pod \"e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:30.324576 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.324526 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qhcm\" (UniqueName: \"kubernetes.io/projected/12cdaa3d-49a5-439f-8502-ec498e23017c-kube-api-access-5qhcm\") pod \"e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:30.324576 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.324577 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12cdaa3d-49a5-439f-8502-ec498e23017c-workload-socket\") pod \"e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:30.324865 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.324698 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12cdaa3d-49a5-439f-8502-ec498e23017c-istio-data\") pod \"e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:30.324865 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.324761 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12cdaa3d-49a5-439f-8502-ec498e23017c-credential-socket\") pod \"e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:30.324865 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.324804 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12cdaa3d-49a5-439f-8502-ec498e23017c-istio-podinfo\") pod \"e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:30.324865 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.324829 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12cdaa3d-49a5-439f-8502-ec498e23017c-istiod-ca-cert\") pod \"e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:30.325066 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.324874 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12cdaa3d-49a5-439f-8502-ec498e23017c-istio-token\") pod \"e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:30.325066 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.324908 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12cdaa3d-49a5-439f-8502-ec498e23017c-istio-envoy\") pod \"e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:30.325066 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.324940 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12cdaa3d-49a5-439f-8502-ec498e23017c-workload-socket\") pod \"e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:30.325270 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.325063 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12cdaa3d-49a5-439f-8502-ec498e23017c-istio-data\") pod \"e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:30.325270 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.325096 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12cdaa3d-49a5-439f-8502-ec498e23017c-workload-certs\") pod \"e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:30.325270 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.325167 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12cdaa3d-49a5-439f-8502-ec498e23017c-credential-socket\") pod \"e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:30.325634 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.325611 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12cdaa3d-49a5-439f-8502-ec498e23017c-istiod-ca-cert\") pod \"e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:30.327113 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.327095 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12cdaa3d-49a5-439f-8502-ec498e23017c-istio-envoy\") pod \"e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:30.327247 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.327227 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12cdaa3d-49a5-439f-8502-ec498e23017c-istio-podinfo\") pod \"e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:30.331855 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.331836 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qhcm\" (UniqueName: \"kubernetes.io/projected/12cdaa3d-49a5-439f-8502-ec498e23017c-kube-api-access-5qhcm\") pod \"e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:30.332000 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.331980 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12cdaa3d-49a5-439f-8502-ec498e23017c-istio-token\") pod \"e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:30.378854 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.378824 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:30.522809 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.522781 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2"] Jul 12 08:52:30.524246 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:52:30.524221 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12cdaa3d_49a5_439f_8502_ec498e23017c.slice/crio-f7999b04ee8bdef7da36b928112f73610bab40be67d9943c5e95d7d43962be05 WatchSource:0}: Error finding container f7999b04ee8bdef7da36b928112f73610bab40be67d9943c5e95d7d43962be05: Status 404 returned error can't find the container with id f7999b04ee8bdef7da36b928112f73610bab40be67d9943c5e95d7d43962be05 Jul 12 08:52:30.526319 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.526286 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:52:30.526401 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.526361 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:52:30.526443 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.526397 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:52:30.893633 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:30.893532 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:52:31.097754 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:31.097722 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" event={"ID":"12cdaa3d-49a5-439f-8502-ec498e23017c","Type":"ContainerStarted","Data":"2d2348b8b718d8ea7ba0038c6cd650e50e36a60de4b1e4e6a0d566ee51ca11c5"} Jul 12 08:52:31.097956 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:31.097771 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" event={"ID":"12cdaa3d-49a5-439f-8502-ec498e23017c","Type":"ContainerStarted","Data":"f7999b04ee8bdef7da36b928112f73610bab40be67d9943c5e95d7d43962be05"} Jul 12 08:52:31.117810 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:31.117753 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" podStartSLOduration=1.117734529 podStartE2EDuration="1.117734529s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:52:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:52:31.114729094 +0000 UTC m=+1874.775524769" watchObservedRunningTime="2026-07-12 08:52:31.117734529 +0000 UTC m=+1874.778530194" Jul 12 08:52:31.379772 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:31.379745 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:31.384192 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:31.384170 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:32.102703 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:32.102667 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:32.103544 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:32.103524 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:32.991775 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:32.991742 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:52:33.066817 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:33.066769 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:52:33.137870 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:33.137834 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:52:33.643308 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:33.643274 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:52:33.742296 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:33.742263 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:52:33.841083 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:33.841056 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:52:37.477701 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:37.477669 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:52:40.104785 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:40.104698 2566 patch_prober.go:28] interesting pod/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 12 08:52:40.105109 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:40.104784 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" podUID="70616a20-0d42-433e-9e54-e8516c4ce886" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 12 08:52:42.160611 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:42.160577 2566 generic.go:356] "Generic (PLEG): container finished" podID="70616a20-0d42-433e-9e54-e8516c4ce886" containerID="7d3908e75ddce9176c02426ebd8a3a3d74f8b705add547cc99ef78d02aa3dd3d" exitCode=0 Jul 12 08:52:42.161026 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:42.160644 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" event={"ID":"70616a20-0d42-433e-9e54-e8516c4ce886","Type":"ContainerDied","Data":"7d3908e75ddce9176c02426ebd8a3a3d74f8b705add547cc99ef78d02aa3dd3d"} Jul 12 08:52:42.161070 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:42.161049 2566 scope.go:117] "RemoveContainer" containerID="7d3908e75ddce9176c02426ebd8a3a3d74f8b705add547cc99ef78d02aa3dd3d" Jul 12 08:52:42.161996 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:42.161972 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:52:42.162065 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:42.162036 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:52:42.162107 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:42.162067 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:52:43.168569 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:43.168511 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" event={"ID":"70616a20-0d42-433e-9e54-e8516c4ce886","Type":"ContainerStarted","Data":"6a3aeae03371199ab085fcf38bea44986cb8692fdc82091ba526bb13553f0f22"} Jul 12 08:52:43.291842 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:43.291806 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:43.292134 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:43.292111 2566 patch_prober.go:28] interesting pod/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 12 08:52:43.292188 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:43.292158 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" podUID="70616a20-0d42-433e-9e54-e8516c4ce886" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 12 08:52:44.292395 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:44.292363 2566 patch_prober.go:28] interesting pod/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 12 08:52:44.292822 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:44.292418 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" podUID="70616a20-0d42-433e-9e54-e8516c4ce886" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 12 08:52:45.291912 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:45.291877 2566 patch_prober.go:28] interesting pod/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 12 08:52:45.292067 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:45.291955 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" podUID="70616a20-0d42-433e-9e54-e8516c4ce886" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 12 08:52:46.291755 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:46.291716 2566 patch_prober.go:28] interesting pod/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 12 08:52:46.292137 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:46.291790 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" podUID="70616a20-0d42-433e-9e54-e8516c4ce886" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 12 08:52:46.332026 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:46.331982 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:52:46.436411 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:46.436327 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:52:47.295189 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:47.295152 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:47.295537 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:47.295442 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:47.296190 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:47.296171 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:52:48.039718 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:48.039029 2566 patch_prober.go:28] interesting pod/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": read tcp 10.132.0.2:40286->10.132.0.58:15021: read: connection reset by peer" start-of-body= Jul 12 08:52:48.039718 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:48.039111 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" podUID="12cdaa3d-49a5-439f-8502-ec498e23017c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": read tcp 10.132.0.2:40286->10.132.0.58:15021: read: connection reset by peer" Jul 12 08:52:48.195393 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:48.195357 2566 generic.go:356] "Generic (PLEG): container finished" podID="12cdaa3d-49a5-439f-8502-ec498e23017c" containerID="2d2348b8b718d8ea7ba0038c6cd650e50e36a60de4b1e4e6a0d566ee51ca11c5" exitCode=0 Jul 12 08:52:48.195537 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:48.195430 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" event={"ID":"12cdaa3d-49a5-439f-8502-ec498e23017c","Type":"ContainerDied","Data":"2d2348b8b718d8ea7ba0038c6cd650e50e36a60de4b1e4e6a0d566ee51ca11c5"} Jul 12 08:52:48.195941 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:48.195921 2566 scope.go:117] "RemoveContainer" containerID="2d2348b8b718d8ea7ba0038c6cd650e50e36a60de4b1e4e6a0d566ee51ca11c5" Jul 12 08:52:48.196895 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:48.196867 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:52:48.196950 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:48.196938 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:52:48.196986 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:48.196971 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:52:49.203238 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:49.203204 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" event={"ID":"12cdaa3d-49a5-439f-8502-ec498e23017c","Type":"ContainerStarted","Data":"02321a619cc2511fe07d61122f1ef4aa93b171bb5b614f21801671de10b32972"} Jul 12 08:52:49.379118 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:49.379080 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:49.379318 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:49.379285 2566 patch_prober.go:28] interesting pod/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 12 08:52:49.379399 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:49.379338 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" podUID="12cdaa3d-49a5-439f-8502-ec498e23017c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 12 08:52:50.379175 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:50.379141 2566 patch_prober.go:28] interesting pod/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 12 08:52:50.379600 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:50.379198 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" podUID="12cdaa3d-49a5-439f-8502-ec498e23017c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 12 08:52:51.379648 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:51.379611 2566 patch_prober.go:28] interesting pod/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 12 08:52:51.380063 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:51.379668 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" podUID="12cdaa3d-49a5-439f-8502-ec498e23017c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 12 08:52:52.379592 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:52.379559 2566 patch_prober.go:28] interesting pod/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 12 08:52:52.379760 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:52.379624 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" podUID="12cdaa3d-49a5-439f-8502-ec498e23017c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 12 08:52:53.382998 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:53.382964 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:53.383437 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:53.383227 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:53.383830 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:53.383816 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:52:54.332998 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:54.332964 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:52:54.432915 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:54.432883 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:52:56.820540 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:52:56.820500 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:53:23.429658 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:53:23.429619 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:53:34.872600 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:53:34.872564 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:53:40.961521 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:53:40.961491 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:53:41.939844 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:53:41.939808 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:53:42.189104 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:53:42.189076 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:53:42.594089 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:53:42.594054 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:53:42.696326 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:53:42.696278 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:53:42.795114 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:53:42.795079 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:53:42.959344 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:53:42.959257 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:53:47.670019 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:53:47.669966 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:54:24.175074 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:24.174982 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:54:24.420718 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:24.420684 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k"] Jul 12 08:54:24.420969 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:24.420938 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" podUID="70616a20-0d42-433e-9e54-e8516c4ce886" containerName="istio-proxy" containerID="cri-o://6a3aeae03371199ab085fcf38bea44986cb8692fdc82091ba526bb13553f0f22" gracePeriod=30 Jul 12 08:54:25.502594 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:25.502528 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:54:25.591675 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:25.591642 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:54:29.672440 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.672410 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:54:29.693797 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.693726 2566 generic.go:356] "Generic (PLEG): container finished" podID="70616a20-0d42-433e-9e54-e8516c4ce886" containerID="6a3aeae03371199ab085fcf38bea44986cb8692fdc82091ba526bb13553f0f22" exitCode=0 Jul 12 08:54:29.693797 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.693782 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" event={"ID":"70616a20-0d42-433e-9e54-e8516c4ce886","Type":"ContainerDied","Data":"6a3aeae03371199ab085fcf38bea44986cb8692fdc82091ba526bb13553f0f22"} Jul 12 08:54:29.693797 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.693787 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" Jul 12 08:54:29.694017 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.693831 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k" event={"ID":"70616a20-0d42-433e-9e54-e8516c4ce886","Type":"ContainerDied","Data":"f1a9a87a09122a99deef883867190c58ff0c52beae30cf8d5c57fc0d2251e3e2"} Jul 12 08:54:29.694017 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.693860 2566 scope.go:117] "RemoveContainer" containerID="6a3aeae03371199ab085fcf38bea44986cb8692fdc82091ba526bb13553f0f22" Jul 12 08:54:29.703776 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.703754 2566 scope.go:117] "RemoveContainer" containerID="7d3908e75ddce9176c02426ebd8a3a3d74f8b705add547cc99ef78d02aa3dd3d" Jul 12 08:54:29.712306 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.712286 2566 scope.go:117] "RemoveContainer" containerID="6a3aeae03371199ab085fcf38bea44986cb8692fdc82091ba526bb13553f0f22" Jul 12 08:54:29.712590 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:54:29.712560 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a3aeae03371199ab085fcf38bea44986cb8692fdc82091ba526bb13553f0f22\": container with ID starting with 6a3aeae03371199ab085fcf38bea44986cb8692fdc82091ba526bb13553f0f22 not found: ID does not exist" containerID="6a3aeae03371199ab085fcf38bea44986cb8692fdc82091ba526bb13553f0f22" Jul 12 08:54:29.712691 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.712600 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a3aeae03371199ab085fcf38bea44986cb8692fdc82091ba526bb13553f0f22"} err="failed to get container status \"6a3aeae03371199ab085fcf38bea44986cb8692fdc82091ba526bb13553f0f22\": rpc error: code = NotFound desc = could not find container \"6a3aeae03371199ab085fcf38bea44986cb8692fdc82091ba526bb13553f0f22\": container with ID starting with 6a3aeae03371199ab085fcf38bea44986cb8692fdc82091ba526bb13553f0f22 not found: ID does not exist" Jul 12 08:54:29.712691 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.712625 2566 scope.go:117] "RemoveContainer" containerID="7d3908e75ddce9176c02426ebd8a3a3d74f8b705add547cc99ef78d02aa3dd3d" Jul 12 08:54:29.712882 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:54:29.712866 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d3908e75ddce9176c02426ebd8a3a3d74f8b705add547cc99ef78d02aa3dd3d\": container with ID starting with 7d3908e75ddce9176c02426ebd8a3a3d74f8b705add547cc99ef78d02aa3dd3d not found: ID does not exist" containerID="7d3908e75ddce9176c02426ebd8a3a3d74f8b705add547cc99ef78d02aa3dd3d" Jul 12 08:54:29.712941 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.712897 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d3908e75ddce9176c02426ebd8a3a3d74f8b705add547cc99ef78d02aa3dd3d"} err="failed to get container status \"7d3908e75ddce9176c02426ebd8a3a3d74f8b705add547cc99ef78d02aa3dd3d\": rpc error: code = NotFound desc = could not find container \"7d3908e75ddce9176c02426ebd8a3a3d74f8b705add547cc99ef78d02aa3dd3d\": container with ID starting with 7d3908e75ddce9176c02426ebd8a3a3d74f8b705add547cc99ef78d02aa3dd3d not found: ID does not exist" Jul 12 08:54:29.776417 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.776381 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70616a20-0d42-433e-9e54-e8516c4ce886-istio-podinfo\") pod \"70616a20-0d42-433e-9e54-e8516c4ce886\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " Jul 12 08:54:29.776621 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.776448 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g7kzr\" (UniqueName: \"kubernetes.io/projected/70616a20-0d42-433e-9e54-e8516c4ce886-kube-api-access-g7kzr\") pod \"70616a20-0d42-433e-9e54-e8516c4ce886\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " Jul 12 08:54:29.776621 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.776479 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70616a20-0d42-433e-9e54-e8516c4ce886-istio-envoy\") pod \"70616a20-0d42-433e-9e54-e8516c4ce886\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " Jul 12 08:54:29.776621 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.776597 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70616a20-0d42-433e-9e54-e8516c4ce886-workload-socket\") pod \"70616a20-0d42-433e-9e54-e8516c4ce886\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " Jul 12 08:54:29.776807 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.776639 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70616a20-0d42-433e-9e54-e8516c4ce886-istio-token\") pod \"70616a20-0d42-433e-9e54-e8516c4ce886\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " Jul 12 08:54:29.776807 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.776665 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70616a20-0d42-433e-9e54-e8516c4ce886-istio-data\") pod \"70616a20-0d42-433e-9e54-e8516c4ce886\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " Jul 12 08:54:29.776807 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.776698 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70616a20-0d42-433e-9e54-e8516c4ce886-workload-certs\") pod \"70616a20-0d42-433e-9e54-e8516c4ce886\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " Jul 12 08:54:29.776807 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.776792 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70616a20-0d42-433e-9e54-e8516c4ce886-credential-socket\") pod \"70616a20-0d42-433e-9e54-e8516c4ce886\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " Jul 12 08:54:29.777015 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.776822 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70616a20-0d42-433e-9e54-e8516c4ce886-istiod-ca-cert\") pod \"70616a20-0d42-433e-9e54-e8516c4ce886\" (UID: \"70616a20-0d42-433e-9e54-e8516c4ce886\") " Jul 12 08:54:29.777385 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.777111 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70616a20-0d42-433e-9e54-e8516c4ce886-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "70616a20-0d42-433e-9e54-e8516c4ce886" (UID: "70616a20-0d42-433e-9e54-e8516c4ce886"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:54:29.777385 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.777325 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70616a20-0d42-433e-9e54-e8516c4ce886-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "70616a20-0d42-433e-9e54-e8516c4ce886" (UID: "70616a20-0d42-433e-9e54-e8516c4ce886"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:54:29.777385 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.777379 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70616a20-0d42-433e-9e54-e8516c4ce886-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "70616a20-0d42-433e-9e54-e8516c4ce886" (UID: "70616a20-0d42-433e-9e54-e8516c4ce886"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 08:54:29.777672 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.777581 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70616a20-0d42-433e-9e54-e8516c4ce886-istio-data" (OuterVolumeSpecName: "istio-data") pod "70616a20-0d42-433e-9e54-e8516c4ce886" (UID: "70616a20-0d42-433e-9e54-e8516c4ce886"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:54:29.777672 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.777661 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70616a20-0d42-433e-9e54-e8516c4ce886-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "70616a20-0d42-433e-9e54-e8516c4ce886" (UID: "70616a20-0d42-433e-9e54-e8516c4ce886"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:54:29.778807 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.778782 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/70616a20-0d42-433e-9e54-e8516c4ce886-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "70616a20-0d42-433e-9e54-e8516c4ce886" (UID: "70616a20-0d42-433e-9e54-e8516c4ce886"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 08:54:29.778924 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.778811 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70616a20-0d42-433e-9e54-e8516c4ce886-kube-api-access-g7kzr" (OuterVolumeSpecName: "kube-api-access-g7kzr") pod "70616a20-0d42-433e-9e54-e8516c4ce886" (UID: "70616a20-0d42-433e-9e54-e8516c4ce886"). InnerVolumeSpecName "kube-api-access-g7kzr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:54:29.779029 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.779014 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70616a20-0d42-433e-9e54-e8516c4ce886-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "70616a20-0d42-433e-9e54-e8516c4ce886" (UID: "70616a20-0d42-433e-9e54-e8516c4ce886"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:54:29.779029 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.779018 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70616a20-0d42-433e-9e54-e8516c4ce886-istio-token" (OuterVolumeSpecName: "istio-token") pod "70616a20-0d42-433e-9e54-e8516c4ce886" (UID: "70616a20-0d42-433e-9e54-e8516c4ce886"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:54:29.878226 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.878194 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70616a20-0d42-433e-9e54-e8516c4ce886-credential-socket\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:54:29.878226 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.878221 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70616a20-0d42-433e-9e54-e8516c4ce886-istiod-ca-cert\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:54:29.878226 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.878233 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70616a20-0d42-433e-9e54-e8516c4ce886-istio-podinfo\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:54:29.878435 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.878242 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7kzr\" (UniqueName: \"kubernetes.io/projected/70616a20-0d42-433e-9e54-e8516c4ce886-kube-api-access-g7kzr\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:54:29.878435 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.878251 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70616a20-0d42-433e-9e54-e8516c4ce886-istio-envoy\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:54:29.878435 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.878259 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70616a20-0d42-433e-9e54-e8516c4ce886-workload-socket\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:54:29.878435 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.878267 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70616a20-0d42-433e-9e54-e8516c4ce886-istio-token\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:54:29.878435 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.878275 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70616a20-0d42-433e-9e54-e8516c4ce886-istio-data\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:54:29.878435 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:29.878282 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70616a20-0d42-433e-9e54-e8516c4ce886-workload-certs\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:54:30.019530 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:30.019496 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k"] Jul 12 08:54:30.023284 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:30.023251 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-d8ca53-openshift-default-85dd6b89b5-bxh6k"] Jul 12 08:54:30.884519 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:30.884487 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:54:31.028690 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:31.028655 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70616a20-0d42-433e-9e54-e8516c4ce886" path="/var/lib/kubelet/pods/70616a20-0d42-433e-9e54-e8516c4ce886/volumes" Jul 12 08:54:31.054096 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:31.054066 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:54:31.166844 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:31.166757 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2"] Jul 12 08:54:31.167010 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:31.166984 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" podUID="12cdaa3d-49a5-439f-8502-ec498e23017c" containerName="istio-proxy" containerID="cri-o://02321a619cc2511fe07d61122f1ef4aa93b171bb5b614f21801671de10b32972" gracePeriod=30 Jul 12 08:54:31.192001 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:31.191975 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:54:31.754130 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:31.754098 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:54:32.019793 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.019711 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w"] Jul 12 08:54:32.020220 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.020109 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70616a20-0d42-433e-9e54-e8516c4ce886" containerName="istio-proxy" Jul 12 08:54:32.020220 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.020125 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="70616a20-0d42-433e-9e54-e8516c4ce886" containerName="istio-proxy" Jul 12 08:54:32.020220 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.020147 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70616a20-0d42-433e-9e54-e8516c4ce886" containerName="istio-proxy" Jul 12 08:54:32.020220 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.020156 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="70616a20-0d42-433e-9e54-e8516c4ce886" containerName="istio-proxy" Jul 12 08:54:32.020442 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.020271 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="70616a20-0d42-433e-9e54-e8516c4ce886" containerName="istio-proxy" Jul 12 08:54:32.024873 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.024849 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:32.027018 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.026996 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-91c269-openshift-default-dockercfg-xbvmp\"" Jul 12 08:54:32.034887 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.034865 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w"] Jul 12 08:54:32.045099 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.045067 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:54:32.097556 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.097531 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9976363d-ffec-421d-a2ca-4421e522ad98-workload-socket\") pod \"e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:32.097674 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.097582 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j58x\" (UniqueName: \"kubernetes.io/projected/9976363d-ffec-421d-a2ca-4421e522ad98-kube-api-access-7j58x\") pod \"e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:32.097737 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.097666 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9976363d-ffec-421d-a2ca-4421e522ad98-istio-podinfo\") pod \"e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:32.097737 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.097710 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9976363d-ffec-421d-a2ca-4421e522ad98-credential-socket\") pod \"e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:32.097838 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.097821 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9976363d-ffec-421d-a2ca-4421e522ad98-workload-certs\") pod \"e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:32.097879 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.097853 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9976363d-ffec-421d-a2ca-4421e522ad98-istio-envoy\") pod \"e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:32.097929 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.097915 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9976363d-ffec-421d-a2ca-4421e522ad98-istio-data\") pod \"e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:32.097976 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.097937 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9976363d-ffec-421d-a2ca-4421e522ad98-istiod-ca-cert\") pod \"e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:32.098057 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.098034 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9976363d-ffec-421d-a2ca-4421e522ad98-istio-token\") pod \"e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:32.199413 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.199375 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9976363d-ffec-421d-a2ca-4421e522ad98-istio-data\") pod \"e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:32.199574 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.199437 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9976363d-ffec-421d-a2ca-4421e522ad98-istiod-ca-cert\") pod \"e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:32.199574 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.199498 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9976363d-ffec-421d-a2ca-4421e522ad98-istio-token\") pod \"e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:32.199574 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.199563 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9976363d-ffec-421d-a2ca-4421e522ad98-workload-socket\") pod \"e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:32.199740 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.199589 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j58x\" (UniqueName: \"kubernetes.io/projected/9976363d-ffec-421d-a2ca-4421e522ad98-kube-api-access-7j58x\") pod \"e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:32.199740 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.199637 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9976363d-ffec-421d-a2ca-4421e522ad98-istio-podinfo\") pod \"e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:32.199740 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.199665 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9976363d-ffec-421d-a2ca-4421e522ad98-credential-socket\") pod \"e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:32.200243 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.200181 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9976363d-ffec-421d-a2ca-4421e522ad98-istio-data\") pod \"e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:32.200431 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.200403 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9976363d-ffec-421d-a2ca-4421e522ad98-workload-certs\") pod \"e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:32.201353 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.200541 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9976363d-ffec-421d-a2ca-4421e522ad98-workload-certs\") pod \"e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:32.201353 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.200581 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9976363d-ffec-421d-a2ca-4421e522ad98-istio-envoy\") pod \"e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:32.201353 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.200503 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9976363d-ffec-421d-a2ca-4421e522ad98-workload-socket\") pod \"e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:32.201353 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.200788 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9976363d-ffec-421d-a2ca-4421e522ad98-credential-socket\") pod \"e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:32.201353 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.200956 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9976363d-ffec-421d-a2ca-4421e522ad98-istiod-ca-cert\") pod \"e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:32.205046 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.203797 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9976363d-ffec-421d-a2ca-4421e522ad98-istio-podinfo\") pod \"e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:32.205046 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.203811 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9976363d-ffec-421d-a2ca-4421e522ad98-istio-envoy\") pod \"e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:32.211697 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.211665 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9976363d-ffec-421d-a2ca-4421e522ad98-istio-token\") pod \"e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:32.211914 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.211897 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j58x\" (UniqueName: \"kubernetes.io/projected/9976363d-ffec-421d-a2ca-4421e522ad98-kube-api-access-7j58x\") pod \"e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:32.276034 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.275967 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:54:32.336887 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.336848 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:32.473056 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.473030 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w"] Jul 12 08:54:32.474447 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:54:32.474421 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9976363d_ffec_421d_a2ca_4421e522ad98.slice/crio-45bfe6c9c0c874435a16c0f7e289f5507d3b079f3a57a110d0e36a5606986bfd WatchSource:0}: Error finding container 45bfe6c9c0c874435a16c0f7e289f5507d3b079f3a57a110d0e36a5606986bfd: Status 404 returned error can't find the container with id 45bfe6c9c0c874435a16c0f7e289f5507d3b079f3a57a110d0e36a5606986bfd Jul 12 08:54:32.476558 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.476513 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:54:32.476647 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.476601 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:54:32.476647 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.476631 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:54:32.712952 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.712909 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" event={"ID":"9976363d-ffec-421d-a2ca-4421e522ad98","Type":"ContainerStarted","Data":"e8d7d1ce42aca4e946fca1a1386b8c59ffaf6dc22e8ae8f355757d15f4fd5aa4"} Jul 12 08:54:32.713103 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.712976 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" event={"ID":"9976363d-ffec-421d-a2ca-4421e522ad98","Type":"ContainerStarted","Data":"45bfe6c9c0c874435a16c0f7e289f5507d3b079f3a57a110d0e36a5606986bfd"} Jul 12 08:54:32.732713 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:32.732663 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" podStartSLOduration=0.732648354 podStartE2EDuration="732.648354ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:54:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:54:32.731449605 +0000 UTC m=+1996.392245268" watchObservedRunningTime="2026-07-12 08:54:32.732648354 +0000 UTC m=+1996.393444015" Jul 12 08:54:33.337923 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:33.337889 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:33.343009 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:33.342978 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:33.718478 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:33.718398 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:33.719370 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:33.719347 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:34.637012 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:34.636979 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:54:34.707694 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:34.707657 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:54:34.778085 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:34.778035 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:54:34.878374 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:34.878346 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:54:35.167131 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:35.167099 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:54:35.268214 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:35.268181 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:54:36.426746 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.426683 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:54:36.543032 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.542999 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12cdaa3d-49a5-439f-8502-ec498e23017c-credential-socket\") pod \"12cdaa3d-49a5-439f-8502-ec498e23017c\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " Jul 12 08:54:36.543204 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.543073 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qhcm\" (UniqueName: \"kubernetes.io/projected/12cdaa3d-49a5-439f-8502-ec498e23017c-kube-api-access-5qhcm\") pod \"12cdaa3d-49a5-439f-8502-ec498e23017c\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " Jul 12 08:54:36.543204 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.543102 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12cdaa3d-49a5-439f-8502-ec498e23017c-istio-podinfo\") pod \"12cdaa3d-49a5-439f-8502-ec498e23017c\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " Jul 12 08:54:36.543204 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.543141 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12cdaa3d-49a5-439f-8502-ec498e23017c-istio-token\") pod \"12cdaa3d-49a5-439f-8502-ec498e23017c\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " Jul 12 08:54:36.543204 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.543166 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12cdaa3d-49a5-439f-8502-ec498e23017c-workload-certs\") pod \"12cdaa3d-49a5-439f-8502-ec498e23017c\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " Jul 12 08:54:36.543388 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.543341 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12cdaa3d-49a5-439f-8502-ec498e23017c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "12cdaa3d-49a5-439f-8502-ec498e23017c" (UID: "12cdaa3d-49a5-439f-8502-ec498e23017c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:54:36.543436 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.543353 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12cdaa3d-49a5-439f-8502-ec498e23017c-istiod-ca-cert\") pod \"12cdaa3d-49a5-439f-8502-ec498e23017c\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " Jul 12 08:54:36.543478 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.543430 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12cdaa3d-49a5-439f-8502-ec498e23017c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "12cdaa3d-49a5-439f-8502-ec498e23017c" (UID: "12cdaa3d-49a5-439f-8502-ec498e23017c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:54:36.543478 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.543459 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12cdaa3d-49a5-439f-8502-ec498e23017c-workload-socket\") pod \"12cdaa3d-49a5-439f-8502-ec498e23017c\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " Jul 12 08:54:36.543604 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.543491 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12cdaa3d-49a5-439f-8502-ec498e23017c-istio-data\") pod \"12cdaa3d-49a5-439f-8502-ec498e23017c\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " Jul 12 08:54:36.543604 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.543526 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12cdaa3d-49a5-439f-8502-ec498e23017c-istio-envoy\") pod \"12cdaa3d-49a5-439f-8502-ec498e23017c\" (UID: \"12cdaa3d-49a5-439f-8502-ec498e23017c\") " Jul 12 08:54:36.543734 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.543700 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12cdaa3d-49a5-439f-8502-ec498e23017c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "12cdaa3d-49a5-439f-8502-ec498e23017c" (UID: "12cdaa3d-49a5-439f-8502-ec498e23017c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 08:54:36.543811 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.543755 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12cdaa3d-49a5-439f-8502-ec498e23017c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "12cdaa3d-49a5-439f-8502-ec498e23017c" (UID: "12cdaa3d-49a5-439f-8502-ec498e23017c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:54:36.543927 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.543911 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12cdaa3d-49a5-439f-8502-ec498e23017c-workload-certs\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:54:36.543998 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.543932 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12cdaa3d-49a5-439f-8502-ec498e23017c-istiod-ca-cert\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:54:36.543998 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.543975 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12cdaa3d-49a5-439f-8502-ec498e23017c-workload-socket\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:54:36.543998 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.543991 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12cdaa3d-49a5-439f-8502-ec498e23017c-credential-socket\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:54:36.544750 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.544725 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12cdaa3d-49a5-439f-8502-ec498e23017c-istio-data" (OuterVolumeSpecName: "istio-data") pod "12cdaa3d-49a5-439f-8502-ec498e23017c" (UID: "12cdaa3d-49a5-439f-8502-ec498e23017c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:54:36.545580 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.545533 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12cdaa3d-49a5-439f-8502-ec498e23017c-kube-api-access-5qhcm" (OuterVolumeSpecName: "kube-api-access-5qhcm") pod "12cdaa3d-49a5-439f-8502-ec498e23017c" (UID: "12cdaa3d-49a5-439f-8502-ec498e23017c"). InnerVolumeSpecName "kube-api-access-5qhcm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:54:36.545689 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.545606 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12cdaa3d-49a5-439f-8502-ec498e23017c-istio-token" (OuterVolumeSpecName: "istio-token") pod "12cdaa3d-49a5-439f-8502-ec498e23017c" (UID: "12cdaa3d-49a5-439f-8502-ec498e23017c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:54:36.545761 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.545746 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/12cdaa3d-49a5-439f-8502-ec498e23017c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "12cdaa3d-49a5-439f-8502-ec498e23017c" (UID: "12cdaa3d-49a5-439f-8502-ec498e23017c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 08:54:36.545831 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.545816 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12cdaa3d-49a5-439f-8502-ec498e23017c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "12cdaa3d-49a5-439f-8502-ec498e23017c" (UID: "12cdaa3d-49a5-439f-8502-ec498e23017c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:54:36.644911 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.644873 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12cdaa3d-49a5-439f-8502-ec498e23017c-istio-data\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:54:36.644911 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.644904 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12cdaa3d-49a5-439f-8502-ec498e23017c-istio-envoy\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:54:36.644911 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.644915 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qhcm\" (UniqueName: \"kubernetes.io/projected/12cdaa3d-49a5-439f-8502-ec498e23017c-kube-api-access-5qhcm\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:54:36.645165 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.644925 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12cdaa3d-49a5-439f-8502-ec498e23017c-istio-podinfo\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:54:36.645165 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.644934 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12cdaa3d-49a5-439f-8502-ec498e23017c-istio-token\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:54:36.734654 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.734620 2566 generic.go:356] "Generic (PLEG): container finished" podID="12cdaa3d-49a5-439f-8502-ec498e23017c" containerID="02321a619cc2511fe07d61122f1ef4aa93b171bb5b614f21801671de10b32972" exitCode=0 Jul 12 08:54:36.734831 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.734669 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" Jul 12 08:54:36.734831 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.734701 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" event={"ID":"12cdaa3d-49a5-439f-8502-ec498e23017c","Type":"ContainerDied","Data":"02321a619cc2511fe07d61122f1ef4aa93b171bb5b614f21801671de10b32972"} Jul 12 08:54:36.734831 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.734753 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2" event={"ID":"12cdaa3d-49a5-439f-8502-ec498e23017c","Type":"ContainerDied","Data":"f7999b04ee8bdef7da36b928112f73610bab40be67d9943c5e95d7d43962be05"} Jul 12 08:54:36.734831 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.734776 2566 scope.go:117] "RemoveContainer" containerID="02321a619cc2511fe07d61122f1ef4aa93b171bb5b614f21801671de10b32972" Jul 12 08:54:36.744344 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.744323 2566 scope.go:117] "RemoveContainer" containerID="2d2348b8b718d8ea7ba0038c6cd650e50e36a60de4b1e4e6a0d566ee51ca11c5" Jul 12 08:54:36.756803 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.756773 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2"] Jul 12 08:54:36.757429 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.757393 2566 scope.go:117] "RemoveContainer" containerID="02321a619cc2511fe07d61122f1ef4aa93b171bb5b614f21801671de10b32972" Jul 12 08:54:36.758005 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:54:36.757876 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02321a619cc2511fe07d61122f1ef4aa93b171bb5b614f21801671de10b32972\": container with ID starting with 02321a619cc2511fe07d61122f1ef4aa93b171bb5b614f21801671de10b32972 not found: ID does not exist" containerID="02321a619cc2511fe07d61122f1ef4aa93b171bb5b614f21801671de10b32972" Jul 12 08:54:36.758005 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.757914 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02321a619cc2511fe07d61122f1ef4aa93b171bb5b614f21801671de10b32972"} err="failed to get container status \"02321a619cc2511fe07d61122f1ef4aa93b171bb5b614f21801671de10b32972\": rpc error: code = NotFound desc = could not find container \"02321a619cc2511fe07d61122f1ef4aa93b171bb5b614f21801671de10b32972\": container with ID starting with 02321a619cc2511fe07d61122f1ef4aa93b171bb5b614f21801671de10b32972 not found: ID does not exist" Jul 12 08:54:36.758005 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.757938 2566 scope.go:117] "RemoveContainer" containerID="2d2348b8b718d8ea7ba0038c6cd650e50e36a60de4b1e4e6a0d566ee51ca11c5" Jul 12 08:54:36.758258 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:54:36.758231 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d2348b8b718d8ea7ba0038c6cd650e50e36a60de4b1e4e6a0d566ee51ca11c5\": container with ID starting with 2d2348b8b718d8ea7ba0038c6cd650e50e36a60de4b1e4e6a0d566ee51ca11c5 not found: ID does not exist" containerID="2d2348b8b718d8ea7ba0038c6cd650e50e36a60de4b1e4e6a0d566ee51ca11c5" Jul 12 08:54:36.758374 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.758258 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d2348b8b718d8ea7ba0038c6cd650e50e36a60de4b1e4e6a0d566ee51ca11c5"} err="failed to get container status \"2d2348b8b718d8ea7ba0038c6cd650e50e36a60de4b1e4e6a0d566ee51ca11c5\": rpc error: code = NotFound desc = could not find container \"2d2348b8b718d8ea7ba0038c6cd650e50e36a60de4b1e4e6a0d566ee51ca11c5\": container with ID starting with 2d2348b8b718d8ea7ba0038c6cd650e50e36a60de4b1e4e6a0d566ee51ca11c5 not found: ID does not exist" Jul 12 08:54:36.759924 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:36.759903 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-b534df-openshift-default-6bbfd67587-h6pn2"] Jul 12 08:54:37.026432 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:37.026353 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12cdaa3d-49a5-439f-8502-ec498e23017c" path="/var/lib/kubelet/pods/12cdaa3d-49a5-439f-8502-ec498e23017c/volumes" Jul 12 08:54:39.151704 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:39.151665 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:54:39.612706 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:39.612673 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:54:39.876059 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:39.875983 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t"] Jul 12 08:54:39.876493 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:39.876475 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12cdaa3d-49a5-439f-8502-ec498e23017c" containerName="istio-proxy" Jul 12 08:54:39.876583 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:39.876495 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="12cdaa3d-49a5-439f-8502-ec498e23017c" containerName="istio-proxy" Jul 12 08:54:39.876583 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:39.876506 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12cdaa3d-49a5-439f-8502-ec498e23017c" containerName="istio-proxy" Jul 12 08:54:39.876583 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:39.876512 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="12cdaa3d-49a5-439f-8502-ec498e23017c" containerName="istio-proxy" Jul 12 08:54:39.876749 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:39.876611 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="12cdaa3d-49a5-439f-8502-ec498e23017c" containerName="istio-proxy" Jul 12 08:54:39.876749 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:39.876625 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="70616a20-0d42-433e-9e54-e8516c4ce886" containerName="istio-proxy" Jul 12 08:54:39.881240 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:39.881221 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:54:39.883267 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:39.883249 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-ad9984-openshift-default-dockercfg-gg599\"" Jul 12 08:54:39.891256 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:39.891234 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t"] Jul 12 08:54:39.922047 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:39.922003 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:54:39.975028 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:39.974989 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7232660a-9177-4c0a-80f5-785ed909df2c-istiod-ca-cert\") pod \"e2e-api-b-ad9984-openshift-default-5975dd9979-2247t\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:54:39.975211 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:39.975045 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7232660a-9177-4c0a-80f5-785ed909df2c-workload-certs\") pod \"e2e-api-b-ad9984-openshift-default-5975dd9979-2247t\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:54:39.975211 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:39.975076 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7232660a-9177-4c0a-80f5-785ed909df2c-istio-token\") pod \"e2e-api-b-ad9984-openshift-default-5975dd9979-2247t\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:54:39.975211 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:39.975107 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7232660a-9177-4c0a-80f5-785ed909df2c-istio-podinfo\") pod \"e2e-api-b-ad9984-openshift-default-5975dd9979-2247t\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:54:39.975211 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:39.975150 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7d6j\" (UniqueName: \"kubernetes.io/projected/7232660a-9177-4c0a-80f5-785ed909df2c-kube-api-access-l7d6j\") pod \"e2e-api-b-ad9984-openshift-default-5975dd9979-2247t\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:54:39.975495 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:39.975238 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7232660a-9177-4c0a-80f5-785ed909df2c-credential-socket\") pod \"e2e-api-b-ad9984-openshift-default-5975dd9979-2247t\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:54:39.975495 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:39.975282 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7232660a-9177-4c0a-80f5-785ed909df2c-workload-socket\") pod \"e2e-api-b-ad9984-openshift-default-5975dd9979-2247t\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:54:39.975495 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:39.975313 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7232660a-9177-4c0a-80f5-785ed909df2c-istio-data\") pod \"e2e-api-b-ad9984-openshift-default-5975dd9979-2247t\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:54:39.975495 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:39.975342 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7232660a-9177-4c0a-80f5-785ed909df2c-istio-envoy\") pod \"e2e-api-b-ad9984-openshift-default-5975dd9979-2247t\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:54:40.075924 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:40.075885 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7232660a-9177-4c0a-80f5-785ed909df2c-istio-envoy\") pod \"e2e-api-b-ad9984-openshift-default-5975dd9979-2247t\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:54:40.075924 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:40.075931 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7232660a-9177-4c0a-80f5-785ed909df2c-istiod-ca-cert\") pod \"e2e-api-b-ad9984-openshift-default-5975dd9979-2247t\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:54:40.076153 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:40.075962 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7232660a-9177-4c0a-80f5-785ed909df2c-workload-certs\") pod \"e2e-api-b-ad9984-openshift-default-5975dd9979-2247t\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:54:40.076153 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:40.075983 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7232660a-9177-4c0a-80f5-785ed909df2c-istio-token\") pod \"e2e-api-b-ad9984-openshift-default-5975dd9979-2247t\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:54:40.076153 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:40.076042 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7232660a-9177-4c0a-80f5-785ed909df2c-istio-podinfo\") pod \"e2e-api-b-ad9984-openshift-default-5975dd9979-2247t\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:54:40.076153 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:40.076083 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7d6j\" (UniqueName: \"kubernetes.io/projected/7232660a-9177-4c0a-80f5-785ed909df2c-kube-api-access-l7d6j\") pod \"e2e-api-b-ad9984-openshift-default-5975dd9979-2247t\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:54:40.076368 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:40.076176 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7232660a-9177-4c0a-80f5-785ed909df2c-credential-socket\") pod \"e2e-api-b-ad9984-openshift-default-5975dd9979-2247t\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:54:40.076368 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:40.076211 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7232660a-9177-4c0a-80f5-785ed909df2c-workload-socket\") pod \"e2e-api-b-ad9984-openshift-default-5975dd9979-2247t\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:54:40.076368 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:40.076252 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7232660a-9177-4c0a-80f5-785ed909df2c-istio-data\") pod \"e2e-api-b-ad9984-openshift-default-5975dd9979-2247t\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:54:40.076520 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:40.076465 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7232660a-9177-4c0a-80f5-785ed909df2c-workload-certs\") pod \"e2e-api-b-ad9984-openshift-default-5975dd9979-2247t\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:54:40.076672 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:40.076656 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7232660a-9177-4c0a-80f5-785ed909df2c-istio-data\") pod \"e2e-api-b-ad9984-openshift-default-5975dd9979-2247t\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:54:40.076736 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:40.076713 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7232660a-9177-4c0a-80f5-785ed909df2c-credential-socket\") pod \"e2e-api-b-ad9984-openshift-default-5975dd9979-2247t\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:54:40.076794 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:40.076722 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7232660a-9177-4c0a-80f5-785ed909df2c-workload-socket\") pod \"e2e-api-b-ad9984-openshift-default-5975dd9979-2247t\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:54:40.076935 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:40.076912 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7232660a-9177-4c0a-80f5-785ed909df2c-istiod-ca-cert\") pod \"e2e-api-b-ad9984-openshift-default-5975dd9979-2247t\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:54:40.078366 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:40.078345 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7232660a-9177-4c0a-80f5-785ed909df2c-istio-envoy\") pod \"e2e-api-b-ad9984-openshift-default-5975dd9979-2247t\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:54:40.078982 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:40.078956 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7232660a-9177-4c0a-80f5-785ed909df2c-istio-podinfo\") pod \"e2e-api-b-ad9984-openshift-default-5975dd9979-2247t\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:54:40.084135 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:40.084114 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7232660a-9177-4c0a-80f5-785ed909df2c-istio-token\") pod \"e2e-api-b-ad9984-openshift-default-5975dd9979-2247t\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:54:40.084468 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:40.084446 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7d6j\" (UniqueName: \"kubernetes.io/projected/7232660a-9177-4c0a-80f5-785ed909df2c-kube-api-access-l7d6j\") pod \"e2e-api-b-ad9984-openshift-default-5975dd9979-2247t\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:54:40.193530 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:40.193437 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:54:40.333708 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:40.333681 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t"] Jul 12 08:54:40.335972 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:54:40.335939 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7232660a_9177_4c0a_80f5_785ed909df2c.slice/crio-736ef038795c0de022ad84a72789f6ed79f131a44c95799765505b1db60ee288 WatchSource:0}: Error finding container 736ef038795c0de022ad84a72789f6ed79f131a44c95799765505b1db60ee288: Status 404 returned error can't find the container with id 736ef038795c0de022ad84a72789f6ed79f131a44c95799765505b1db60ee288 Jul 12 08:54:40.338214 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:40.338181 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:54:40.338308 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:40.338246 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:54:40.338308 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:40.338272 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:54:40.563462 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:40.563435 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:54:40.759246 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:40.759208 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" event={"ID":"7232660a-9177-4c0a-80f5-785ed909df2c","Type":"ContainerStarted","Data":"a1dffb9761e66da4fd0a89c734e9d6b476960f5839e3932c41e31eb1e9f4bcf4"} Jul 12 08:54:40.759441 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:40.759273 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" event={"ID":"7232660a-9177-4c0a-80f5-785ed909df2c","Type":"ContainerStarted","Data":"736ef038795c0de022ad84a72789f6ed79f131a44c95799765505b1db60ee288"} Jul 12 08:54:40.778209 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:40.778150 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" podStartSLOduration=1.7781326530000001 podStartE2EDuration="1.778132653s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:54:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:54:40.776779641 +0000 UTC m=+2004.437575324" watchObservedRunningTime="2026-07-12 08:54:40.778132653 +0000 UTC m=+2004.438928315" Jul 12 08:54:41.194117 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:41.194087 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:54:41.199122 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:41.199099 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:54:41.764178 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:41.764141 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:54:41.764926 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:41.764907 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:54:42.676668 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:42.676624 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:54:42.749791 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:42.749759 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:54:42.819292 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:42.819251 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:54:42.919616 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:42.919573 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:54:43.231105 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:43.231070 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:54:43.331530 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:43.331496 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:54:47.189769 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:47.189738 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:54:49.759764 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:49.759721 2566 patch_prober.go:28] interesting pod/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 12 08:54:49.760143 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:49.759819 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" podUID="9976363d-ffec-421d-a2ca-4421e522ad98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 12 08:54:51.823000 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:51.822955 2566 generic.go:356] "Generic (PLEG): container finished" podID="9976363d-ffec-421d-a2ca-4421e522ad98" containerID="e8d7d1ce42aca4e946fca1a1386b8c59ffaf6dc22e8ae8f355757d15f4fd5aa4" exitCode=0 Jul 12 08:54:51.823566 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:51.823106 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" event={"ID":"9976363d-ffec-421d-a2ca-4421e522ad98","Type":"ContainerDied","Data":"e8d7d1ce42aca4e946fca1a1386b8c59ffaf6dc22e8ae8f355757d15f4fd5aa4"} Jul 12 08:54:51.823662 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:51.823611 2566 scope.go:117] "RemoveContainer" containerID="e8d7d1ce42aca4e946fca1a1386b8c59ffaf6dc22e8ae8f355757d15f4fd5aa4" Jul 12 08:54:51.825124 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:51.825079 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:54:51.825231 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:51.825182 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:54:51.825297 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:51.825226 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:54:52.833789 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:52.833743 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" event={"ID":"9976363d-ffec-421d-a2ca-4421e522ad98","Type":"ContainerStarted","Data":"1f1af4db51add342abca192c40580fbeb8fa917e22497e1925819924a9f82321"} Jul 12 08:54:53.337572 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:53.337516 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:53.337981 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:53.337890 2566 patch_prober.go:28] interesting pod/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 12 08:54:53.338154 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:53.338127 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" podUID="9976363d-ffec-421d-a2ca-4421e522ad98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 12 08:54:54.338017 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:54.337975 2566 patch_prober.go:28] interesting pod/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 12 08:54:54.338636 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:54.338603 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" podUID="9976363d-ffec-421d-a2ca-4421e522ad98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 12 08:54:55.301399 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:55.301366 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:54:55.338195 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:55.338156 2566 patch_prober.go:28] interesting pod/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 12 08:54:55.338623 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:55.338215 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" podUID="9976363d-ffec-421d-a2ca-4421e522ad98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 12 08:54:55.671664 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:55.671631 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:54:55.772301 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:55.772256 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:54:55.873082 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:55.873050 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:54:56.055423 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:56.055388 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:54:56.337542 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:56.337451 2566 patch_prober.go:28] interesting pod/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 12 08:54:56.337542 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:56.337525 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" podUID="9976363d-ffec-421d-a2ca-4421e522ad98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 12 08:54:57.341740 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:57.341709 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:57.342131 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:57.342099 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:57.342836 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:57.342816 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:54:57.806720 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:57.806681 2566 patch_prober.go:28] interesting pod/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 12 08:54:57.806912 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:57.806750 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" podUID="7232660a-9177-4c0a-80f5-785ed909df2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 12 08:54:58.873956 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:58.873920 2566 generic.go:356] "Generic (PLEG): container finished" podID="7232660a-9177-4c0a-80f5-785ed909df2c" containerID="a1dffb9761e66da4fd0a89c734e9d6b476960f5839e3932c41e31eb1e9f4bcf4" exitCode=0 Jul 12 08:54:58.874421 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:58.873999 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" event={"ID":"7232660a-9177-4c0a-80f5-785ed909df2c","Type":"ContainerDied","Data":"a1dffb9761e66da4fd0a89c734e9d6b476960f5839e3932c41e31eb1e9f4bcf4"} Jul 12 08:54:58.874588 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:58.874571 2566 scope.go:117] "RemoveContainer" containerID="a1dffb9761e66da4fd0a89c734e9d6b476960f5839e3932c41e31eb1e9f4bcf4" Jul 12 08:54:58.875511 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:58.875488 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:54:58.875586 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:58.875566 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:54:58.875640 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:58.875597 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:54:59.881792 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:54:59.881751 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" event={"ID":"7232660a-9177-4c0a-80f5-785ed909df2c","Type":"ContainerStarted","Data":"b1b0cd3e4d998f7314afa0af9a9f133d0d511c77effd83201b9ff46fa8e8d506"} Jul 12 08:55:00.193765 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:00.193682 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:55:00.193931 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:00.193909 2566 patch_prober.go:28] interesting pod/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 12 08:55:00.193986 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:00.193969 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" podUID="7232660a-9177-4c0a-80f5-785ed909df2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 12 08:55:01.190252 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:01.190217 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:01.194269 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:01.194245 2566 patch_prober.go:28] interesting pod/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 12 08:55:01.194330 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:01.194307 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" podUID="7232660a-9177-4c0a-80f5-785ed909df2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 12 08:55:01.290686 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:01.290647 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:01.450470 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:01.450381 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w"] Jul 12 08:55:01.450682 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:01.450655 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" podUID="9976363d-ffec-421d-a2ca-4421e522ad98" containerName="istio-proxy" containerID="cri-o://1f1af4db51add342abca192c40580fbeb8fa917e22497e1925819924a9f82321" gracePeriod=30 Jul 12 08:55:01.472472 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:01.472441 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:01.842705 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:01.842671 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:02.194478 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:02.194394 2566 patch_prober.go:28] interesting pod/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 12 08:55:02.194478 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:02.194460 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" podUID="7232660a-9177-4c0a-80f5-785ed909df2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 12 08:55:02.229677 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:02.229612 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:02.525251 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:02.525196 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:02.619135 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:02.619099 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:02.730574 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:02.730509 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:03.194701 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:03.194666 2566 patch_prober.go:28] interesting pod/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 12 08:55:03.195071 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:03.194719 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" podUID="7232660a-9177-4c0a-80f5-785ed909df2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 12 08:55:04.198071 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:04.198046 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:55:04.198435 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:04.198358 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:55:04.199008 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:04.198987 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:55:06.709132 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.709106 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:55:06.747135 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.747104 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9976363d-ffec-421d-a2ca-4421e522ad98-istio-envoy\") pod \"9976363d-ffec-421d-a2ca-4421e522ad98\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " Jul 12 08:55:06.747300 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.747190 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9976363d-ffec-421d-a2ca-4421e522ad98-workload-socket\") pod \"9976363d-ffec-421d-a2ca-4421e522ad98\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " Jul 12 08:55:06.747300 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.747236 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9976363d-ffec-421d-a2ca-4421e522ad98-istio-podinfo\") pod \"9976363d-ffec-421d-a2ca-4421e522ad98\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " Jul 12 08:55:06.747300 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.747260 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9976363d-ffec-421d-a2ca-4421e522ad98-istio-data\") pod \"9976363d-ffec-421d-a2ca-4421e522ad98\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " Jul 12 08:55:06.747300 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.747296 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9976363d-ffec-421d-a2ca-4421e522ad98-credential-socket\") pod \"9976363d-ffec-421d-a2ca-4421e522ad98\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " Jul 12 08:55:06.747513 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.747340 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7j58x\" (UniqueName: \"kubernetes.io/projected/9976363d-ffec-421d-a2ca-4421e522ad98-kube-api-access-7j58x\") pod \"9976363d-ffec-421d-a2ca-4421e522ad98\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " Jul 12 08:55:06.747513 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.747366 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9976363d-ffec-421d-a2ca-4421e522ad98-workload-certs\") pod \"9976363d-ffec-421d-a2ca-4421e522ad98\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " Jul 12 08:55:06.747513 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.747389 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9976363d-ffec-421d-a2ca-4421e522ad98-istio-token\") pod \"9976363d-ffec-421d-a2ca-4421e522ad98\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " Jul 12 08:55:06.747513 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.747426 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9976363d-ffec-421d-a2ca-4421e522ad98-istiod-ca-cert\") pod \"9976363d-ffec-421d-a2ca-4421e522ad98\" (UID: \"9976363d-ffec-421d-a2ca-4421e522ad98\") " Jul 12 08:55:06.747513 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.747493 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9976363d-ffec-421d-a2ca-4421e522ad98-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9976363d-ffec-421d-a2ca-4421e522ad98" (UID: "9976363d-ffec-421d-a2ca-4421e522ad98"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:55:06.747804 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.747773 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9976363d-ffec-421d-a2ca-4421e522ad98-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9976363d-ffec-421d-a2ca-4421e522ad98" (UID: "9976363d-ffec-421d-a2ca-4421e522ad98"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:55:06.747863 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.747790 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9976363d-ffec-421d-a2ca-4421e522ad98-workload-socket\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:55:06.747921 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.747887 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9976363d-ffec-421d-a2ca-4421e522ad98-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9976363d-ffec-421d-a2ca-4421e522ad98" (UID: "9976363d-ffec-421d-a2ca-4421e522ad98"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 08:55:06.748116 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.748087 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9976363d-ffec-421d-a2ca-4421e522ad98-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9976363d-ffec-421d-a2ca-4421e522ad98" (UID: "9976363d-ffec-421d-a2ca-4421e522ad98"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:55:06.748906 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.748865 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9976363d-ffec-421d-a2ca-4421e522ad98-istio-data" (OuterVolumeSpecName: "istio-data") pod "9976363d-ffec-421d-a2ca-4421e522ad98" (UID: "9976363d-ffec-421d-a2ca-4421e522ad98"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:55:06.749804 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.749779 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9976363d-ffec-421d-a2ca-4421e522ad98-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9976363d-ffec-421d-a2ca-4421e522ad98" (UID: "9976363d-ffec-421d-a2ca-4421e522ad98"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:55:06.749804 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.749788 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9976363d-ffec-421d-a2ca-4421e522ad98-kube-api-access-7j58x" (OuterVolumeSpecName: "kube-api-access-7j58x") pod "9976363d-ffec-421d-a2ca-4421e522ad98" (UID: "9976363d-ffec-421d-a2ca-4421e522ad98"). InnerVolumeSpecName "kube-api-access-7j58x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:55:06.750177 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.750152 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9976363d-ffec-421d-a2ca-4421e522ad98-istio-token" (OuterVolumeSpecName: "istio-token") pod "9976363d-ffec-421d-a2ca-4421e522ad98" (UID: "9976363d-ffec-421d-a2ca-4421e522ad98"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:55:06.750308 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.750293 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9976363d-ffec-421d-a2ca-4421e522ad98-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9976363d-ffec-421d-a2ca-4421e522ad98" (UID: "9976363d-ffec-421d-a2ca-4421e522ad98"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 08:55:06.848768 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.848719 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9976363d-ffec-421d-a2ca-4421e522ad98-istio-podinfo\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:55:06.848768 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.848762 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9976363d-ffec-421d-a2ca-4421e522ad98-istio-data\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:55:06.848768 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.848772 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9976363d-ffec-421d-a2ca-4421e522ad98-credential-socket\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:55:06.848768 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.848782 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7j58x\" (UniqueName: \"kubernetes.io/projected/9976363d-ffec-421d-a2ca-4421e522ad98-kube-api-access-7j58x\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:55:06.849040 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.848791 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9976363d-ffec-421d-a2ca-4421e522ad98-workload-certs\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:55:06.849040 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.848799 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9976363d-ffec-421d-a2ca-4421e522ad98-istio-token\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:55:06.849040 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.848807 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9976363d-ffec-421d-a2ca-4421e522ad98-istiod-ca-cert\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:55:06.849040 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.848814 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9976363d-ffec-421d-a2ca-4421e522ad98-istio-envoy\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:55:06.917045 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.917011 2566 generic.go:356] "Generic (PLEG): container finished" podID="9976363d-ffec-421d-a2ca-4421e522ad98" containerID="1f1af4db51add342abca192c40580fbeb8fa917e22497e1925819924a9f82321" exitCode=0 Jul 12 08:55:06.917181 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.917062 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" Jul 12 08:55:06.917181 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.917100 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" event={"ID":"9976363d-ffec-421d-a2ca-4421e522ad98","Type":"ContainerDied","Data":"1f1af4db51add342abca192c40580fbeb8fa917e22497e1925819924a9f82321"} Jul 12 08:55:06.917181 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.917158 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w" event={"ID":"9976363d-ffec-421d-a2ca-4421e522ad98","Type":"ContainerDied","Data":"45bfe6c9c0c874435a16c0f7e289f5507d3b079f3a57a110d0e36a5606986bfd"} Jul 12 08:55:06.917303 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.917188 2566 scope.go:117] "RemoveContainer" containerID="1f1af4db51add342abca192c40580fbeb8fa917e22497e1925819924a9f82321" Jul 12 08:55:06.926801 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.926784 2566 scope.go:117] "RemoveContainer" containerID="e8d7d1ce42aca4e946fca1a1386b8c59ffaf6dc22e8ae8f355757d15f4fd5aa4" Jul 12 08:55:06.935402 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.935385 2566 scope.go:117] "RemoveContainer" containerID="1f1af4db51add342abca192c40580fbeb8fa917e22497e1925819924a9f82321" Jul 12 08:55:06.935766 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:55:06.935721 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f1af4db51add342abca192c40580fbeb8fa917e22497e1925819924a9f82321\": container with ID starting with 1f1af4db51add342abca192c40580fbeb8fa917e22497e1925819924a9f82321 not found: ID does not exist" containerID="1f1af4db51add342abca192c40580fbeb8fa917e22497e1925819924a9f82321" Jul 12 08:55:06.935866 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.935761 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f1af4db51add342abca192c40580fbeb8fa917e22497e1925819924a9f82321"} err="failed to get container status \"1f1af4db51add342abca192c40580fbeb8fa917e22497e1925819924a9f82321\": rpc error: code = NotFound desc = could not find container \"1f1af4db51add342abca192c40580fbeb8fa917e22497e1925819924a9f82321\": container with ID starting with 1f1af4db51add342abca192c40580fbeb8fa917e22497e1925819924a9f82321 not found: ID does not exist" Jul 12 08:55:06.935866 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.935785 2566 scope.go:117] "RemoveContainer" containerID="e8d7d1ce42aca4e946fca1a1386b8c59ffaf6dc22e8ae8f355757d15f4fd5aa4" Jul 12 08:55:06.936072 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:55:06.936049 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8d7d1ce42aca4e946fca1a1386b8c59ffaf6dc22e8ae8f355757d15f4fd5aa4\": container with ID starting with e8d7d1ce42aca4e946fca1a1386b8c59ffaf6dc22e8ae8f355757d15f4fd5aa4 not found: ID does not exist" containerID="e8d7d1ce42aca4e946fca1a1386b8c59ffaf6dc22e8ae8f355757d15f4fd5aa4" Jul 12 08:55:06.936170 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.936074 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8d7d1ce42aca4e946fca1a1386b8c59ffaf6dc22e8ae8f355757d15f4fd5aa4"} err="failed to get container status \"e8d7d1ce42aca4e946fca1a1386b8c59ffaf6dc22e8ae8f355757d15f4fd5aa4\": rpc error: code = NotFound desc = could not find container \"e8d7d1ce42aca4e946fca1a1386b8c59ffaf6dc22e8ae8f355757d15f4fd5aa4\": container with ID starting with e8d7d1ce42aca4e946fca1a1386b8c59ffaf6dc22e8ae8f355757d15f4fd5aa4 not found: ID does not exist" Jul 12 08:55:06.937863 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.937842 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w"] Jul 12 08:55:06.940708 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:06.940687 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-91c269-openshift-default-6d76c64ff5-vbr4w"] Jul 12 08:55:07.026929 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:07.026852 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9976363d-ffec-421d-a2ca-4421e522ad98" path="/var/lib/kubelet/pods/9976363d-ffec-421d-a2ca-4421e522ad98/volumes" Jul 12 08:55:07.757366 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:07.757332 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:07.858487 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:07.858454 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:08.002220 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.002187 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t"] Jul 12 08:55:08.002439 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.002395 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" podUID="7232660a-9177-4c0a-80f5-785ed909df2c" containerName="istio-proxy" containerID="cri-o://b1b0cd3e4d998f7314afa0af9a9f133d0d511c77effd83201b9ff46fa8e8d506" gracePeriod=30 Jul 12 08:55:08.022614 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.022521 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:08.551755 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.551726 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:08.829161 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.829082 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6"] Jul 12 08:55:08.829688 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.829654 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9976363d-ffec-421d-a2ca-4421e522ad98" containerName="istio-proxy" Jul 12 08:55:08.829688 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.829680 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9976363d-ffec-421d-a2ca-4421e522ad98" containerName="istio-proxy" Jul 12 08:55:08.829906 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.829717 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9976363d-ffec-421d-a2ca-4421e522ad98" containerName="istio-proxy" Jul 12 08:55:08.829906 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.829728 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9976363d-ffec-421d-a2ca-4421e522ad98" containerName="istio-proxy" Jul 12 08:55:08.829906 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.829814 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="12cdaa3d-49a5-439f-8502-ec498e23017c" containerName="istio-proxy" Jul 12 08:55:08.829906 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.829824 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9976363d-ffec-421d-a2ca-4421e522ad98" containerName="istio-proxy" Jul 12 08:55:08.829906 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.829833 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9976363d-ffec-421d-a2ca-4421e522ad98" containerName="istio-proxy" Jul 12 08:55:08.832990 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.832968 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:08.835067 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.835048 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-a67649-openshift-default-dockercfg-nz986\"" Jul 12 08:55:08.842199 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.842175 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6"] Jul 12 08:55:08.854732 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.854704 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:08.865978 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.865953 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18ff97ac-a784-43fd-9f8c-613aad9cb754-istiod-ca-cert\") pod \"e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:08.866119 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.865997 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18ff97ac-a784-43fd-9f8c-613aad9cb754-workload-certs\") pod \"e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:08.866119 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.866069 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18ff97ac-a784-43fd-9f8c-613aad9cb754-workload-socket\") pod \"e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:08.866119 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.866091 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18ff97ac-a784-43fd-9f8c-613aad9cb754-istio-token\") pod \"e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:08.866286 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.866142 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18ff97ac-a784-43fd-9f8c-613aad9cb754-credential-socket\") pod \"e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:08.866286 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.866168 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18ff97ac-a784-43fd-9f8c-613aad9cb754-istio-envoy\") pod \"e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:08.866286 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.866199 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18ff97ac-a784-43fd-9f8c-613aad9cb754-istio-data\") pod \"e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:08.866286 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.866230 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qvzs\" (UniqueName: \"kubernetes.io/projected/18ff97ac-a784-43fd-9f8c-613aad9cb754-kube-api-access-6qvzs\") pod \"e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:08.866455 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.866316 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18ff97ac-a784-43fd-9f8c-613aad9cb754-istio-podinfo\") pod \"e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:08.967282 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.967247 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18ff97ac-a784-43fd-9f8c-613aad9cb754-workload-certs\") pod \"e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:08.967744 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.967714 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18ff97ac-a784-43fd-9f8c-613aad9cb754-workload-socket\") pod \"e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:08.967837 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.967770 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18ff97ac-a784-43fd-9f8c-613aad9cb754-istio-token\") pod \"e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:08.967987 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.967955 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18ff97ac-a784-43fd-9f8c-613aad9cb754-workload-certs\") pod \"e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:08.968237 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.968217 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18ff97ac-a784-43fd-9f8c-613aad9cb754-credential-socket\") pod \"e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:08.968333 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.968239 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18ff97ac-a784-43fd-9f8c-613aad9cb754-workload-socket\") pod \"e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:08.968333 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.968259 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18ff97ac-a784-43fd-9f8c-613aad9cb754-istio-envoy\") pod \"e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:08.968333 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.968309 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18ff97ac-a784-43fd-9f8c-613aad9cb754-istio-data\") pod \"e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:08.968500 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.968362 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qvzs\" (UniqueName: \"kubernetes.io/projected/18ff97ac-a784-43fd-9f8c-613aad9cb754-kube-api-access-6qvzs\") pod \"e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:08.968500 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.968440 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18ff97ac-a784-43fd-9f8c-613aad9cb754-istio-podinfo\") pod \"e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:08.968500 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.968485 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18ff97ac-a784-43fd-9f8c-613aad9cb754-credential-socket\") pod \"e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:08.968676 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.968528 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18ff97ac-a784-43fd-9f8c-613aad9cb754-istiod-ca-cert\") pod \"e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:08.975579 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.968759 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18ff97ac-a784-43fd-9f8c-613aad9cb754-istio-data\") pod \"e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:08.975579 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.969300 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18ff97ac-a784-43fd-9f8c-613aad9cb754-istiod-ca-cert\") pod \"e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:08.977709 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.977680 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18ff97ac-a784-43fd-9f8c-613aad9cb754-istio-podinfo\") pod \"e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:08.979011 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.978986 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18ff97ac-a784-43fd-9f8c-613aad9cb754-istio-envoy\") pod \"e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:08.979133 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.979115 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qvzs\" (UniqueName: \"kubernetes.io/projected/18ff97ac-a784-43fd-9f8c-613aad9cb754-kube-api-access-6qvzs\") pod \"e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:08.979435 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:08.979417 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18ff97ac-a784-43fd-9f8c-613aad9cb754-istio-token\") pod \"e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:09.144038 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:09.143954 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:09.279884 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:09.279860 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6"] Jul 12 08:55:09.281870 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:55:09.281844 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18ff97ac_a784_43fd_9f8c_613aad9cb754.slice/crio-df8e806b22cc7c096126e83e7143ceb2fc87421a6a87e1fefaaef973bbae67d0 WatchSource:0}: Error finding container df8e806b22cc7c096126e83e7143ceb2fc87421a6a87e1fefaaef973bbae67d0: Status 404 returned error can't find the container with id df8e806b22cc7c096126e83e7143ceb2fc87421a6a87e1fefaaef973bbae67d0 Jul 12 08:55:09.283932 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:09.283899 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:55:09.284007 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:09.283963 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:55:09.284007 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:09.283992 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:55:09.344515 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:09.344483 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:09.935803 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:09.935767 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" event={"ID":"18ff97ac-a784-43fd-9f8c-613aad9cb754","Type":"ContainerStarted","Data":"76d14b39ca53fb7a89cdaeb7fa4b8d9a6ff5f84ce8e817c52b59f46f04681800"} Jul 12 08:55:09.936239 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:09.935823 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" event={"ID":"18ff97ac-a784-43fd-9f8c-613aad9cb754","Type":"ContainerStarted","Data":"df8e806b22cc7c096126e83e7143ceb2fc87421a6a87e1fefaaef973bbae67d0"} Jul 12 08:55:09.954825 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:09.954779 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" podStartSLOduration=1.954730916 podStartE2EDuration="1.954730916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:55:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:55:09.952699165 +0000 UTC m=+2033.613494839" watchObservedRunningTime="2026-07-12 08:55:09.954730916 +0000 UTC m=+2033.615526578" Jul 12 08:55:10.144608 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:10.144575 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:10.149264 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:10.149238 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:10.942680 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:10.942646 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:10.943484 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:10.943467 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:11.851892 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:11.851856 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:11.944245 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:11.944203 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:12.096234 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:12.096202 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:12.525736 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:12.525707 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:12.626306 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:12.626271 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:12.919809 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:12.919717 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:13.262745 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.262724 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:55:13.307182 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.307149 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7232660a-9177-4c0a-80f5-785ed909df2c-istio-envoy\") pod \"7232660a-9177-4c0a-80f5-785ed909df2c\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " Jul 12 08:55:13.307397 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.307189 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7232660a-9177-4c0a-80f5-785ed909df2c-istio-podinfo\") pod \"7232660a-9177-4c0a-80f5-785ed909df2c\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " Jul 12 08:55:13.307397 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.307207 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7232660a-9177-4c0a-80f5-785ed909df2c-workload-socket\") pod \"7232660a-9177-4c0a-80f5-785ed909df2c\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " Jul 12 08:55:13.307397 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.307235 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7232660a-9177-4c0a-80f5-785ed909df2c-istiod-ca-cert\") pod \"7232660a-9177-4c0a-80f5-785ed909df2c\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " Jul 12 08:55:13.307397 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.307261 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7232660a-9177-4c0a-80f5-785ed909df2c-workload-certs\") pod \"7232660a-9177-4c0a-80f5-785ed909df2c\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " Jul 12 08:55:13.307397 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.307327 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7232660a-9177-4c0a-80f5-785ed909df2c-istio-token\") pod \"7232660a-9177-4c0a-80f5-785ed909df2c\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " Jul 12 08:55:13.307397 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.307355 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7232660a-9177-4c0a-80f5-785ed909df2c-credential-socket\") pod \"7232660a-9177-4c0a-80f5-785ed909df2c\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " Jul 12 08:55:13.307795 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.307416 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7d6j\" (UniqueName: \"kubernetes.io/projected/7232660a-9177-4c0a-80f5-785ed909df2c-kube-api-access-l7d6j\") pod \"7232660a-9177-4c0a-80f5-785ed909df2c\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " Jul 12 08:55:13.307795 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.307438 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7232660a-9177-4c0a-80f5-785ed909df2c-istio-data\") pod \"7232660a-9177-4c0a-80f5-785ed909df2c\" (UID: \"7232660a-9177-4c0a-80f5-785ed909df2c\") " Jul 12 08:55:13.307795 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.307510 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7232660a-9177-4c0a-80f5-785ed909df2c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7232660a-9177-4c0a-80f5-785ed909df2c" (UID: "7232660a-9177-4c0a-80f5-785ed909df2c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:55:13.307795 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.307742 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7232660a-9177-4c0a-80f5-785ed909df2c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7232660a-9177-4c0a-80f5-785ed909df2c" (UID: "7232660a-9177-4c0a-80f5-785ed909df2c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:55:13.308013 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.307805 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7232660a-9177-4c0a-80f5-785ed909df2c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7232660a-9177-4c0a-80f5-785ed909df2c" (UID: "7232660a-9177-4c0a-80f5-785ed909df2c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 08:55:13.308013 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.307824 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7232660a-9177-4c0a-80f5-785ed909df2c-workload-socket\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:55:13.308013 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.307858 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7232660a-9177-4c0a-80f5-785ed909df2c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7232660a-9177-4c0a-80f5-785ed909df2c" (UID: "7232660a-9177-4c0a-80f5-785ed909df2c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:55:13.308629 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.308601 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7232660a-9177-4c0a-80f5-785ed909df2c-istio-data" (OuterVolumeSpecName: "istio-data") pod "7232660a-9177-4c0a-80f5-785ed909df2c" (UID: "7232660a-9177-4c0a-80f5-785ed909df2c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:55:13.309587 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.309540 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7232660a-9177-4c0a-80f5-785ed909df2c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7232660a-9177-4c0a-80f5-785ed909df2c" (UID: "7232660a-9177-4c0a-80f5-785ed909df2c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 08:55:13.310080 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.310054 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7232660a-9177-4c0a-80f5-785ed909df2c-kube-api-access-l7d6j" (OuterVolumeSpecName: "kube-api-access-l7d6j") pod "7232660a-9177-4c0a-80f5-785ed909df2c" (UID: "7232660a-9177-4c0a-80f5-785ed909df2c"). InnerVolumeSpecName "kube-api-access-l7d6j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:55:13.310080 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.310060 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7232660a-9177-4c0a-80f5-785ed909df2c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7232660a-9177-4c0a-80f5-785ed909df2c" (UID: "7232660a-9177-4c0a-80f5-785ed909df2c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:55:13.310196 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.310139 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7232660a-9177-4c0a-80f5-785ed909df2c-istio-token" (OuterVolumeSpecName: "istio-token") pod "7232660a-9177-4c0a-80f5-785ed909df2c" (UID: "7232660a-9177-4c0a-80f5-785ed909df2c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:55:13.408804 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.408773 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7232660a-9177-4c0a-80f5-785ed909df2c-istio-envoy\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:55:13.408804 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.408800 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7232660a-9177-4c0a-80f5-785ed909df2c-istio-podinfo\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:55:13.408983 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.408814 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7232660a-9177-4c0a-80f5-785ed909df2c-istiod-ca-cert\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:55:13.408983 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.408827 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7232660a-9177-4c0a-80f5-785ed909df2c-workload-certs\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:55:13.408983 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.408839 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7232660a-9177-4c0a-80f5-785ed909df2c-istio-token\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:55:13.408983 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.408850 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7232660a-9177-4c0a-80f5-785ed909df2c-credential-socket\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:55:13.408983 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.408863 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7d6j\" (UniqueName: \"kubernetes.io/projected/7232660a-9177-4c0a-80f5-785ed909df2c-kube-api-access-l7d6j\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:55:13.408983 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.408874 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7232660a-9177-4c0a-80f5-785ed909df2c-istio-data\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:55:13.960388 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.960352 2566 generic.go:356] "Generic (PLEG): container finished" podID="7232660a-9177-4c0a-80f5-785ed909df2c" containerID="b1b0cd3e4d998f7314afa0af9a9f133d0d511c77effd83201b9ff46fa8e8d506" exitCode=0 Jul 12 08:55:13.960593 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.960399 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" Jul 12 08:55:13.960593 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.960435 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" event={"ID":"7232660a-9177-4c0a-80f5-785ed909df2c","Type":"ContainerDied","Data":"b1b0cd3e4d998f7314afa0af9a9f133d0d511c77effd83201b9ff46fa8e8d506"} Jul 12 08:55:13.960593 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.960489 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t" event={"ID":"7232660a-9177-4c0a-80f5-785ed909df2c","Type":"ContainerDied","Data":"736ef038795c0de022ad84a72789f6ed79f131a44c95799765505b1db60ee288"} Jul 12 08:55:13.960593 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.960512 2566 scope.go:117] "RemoveContainer" containerID="b1b0cd3e4d998f7314afa0af9a9f133d0d511c77effd83201b9ff46fa8e8d506" Jul 12 08:55:13.969408 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.969393 2566 scope.go:117] "RemoveContainer" containerID="a1dffb9761e66da4fd0a89c734e9d6b476960f5839e3932c41e31eb1e9f4bcf4" Jul 12 08:55:13.977019 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.977001 2566 scope.go:117] "RemoveContainer" containerID="b1b0cd3e4d998f7314afa0af9a9f133d0d511c77effd83201b9ff46fa8e8d506" Jul 12 08:55:13.977269 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:55:13.977253 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1b0cd3e4d998f7314afa0af9a9f133d0d511c77effd83201b9ff46fa8e8d506\": container with ID starting with b1b0cd3e4d998f7314afa0af9a9f133d0d511c77effd83201b9ff46fa8e8d506 not found: ID does not exist" containerID="b1b0cd3e4d998f7314afa0af9a9f133d0d511c77effd83201b9ff46fa8e8d506" Jul 12 08:55:13.977322 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.977277 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1b0cd3e4d998f7314afa0af9a9f133d0d511c77effd83201b9ff46fa8e8d506"} err="failed to get container status \"b1b0cd3e4d998f7314afa0af9a9f133d0d511c77effd83201b9ff46fa8e8d506\": rpc error: code = NotFound desc = could not find container \"b1b0cd3e4d998f7314afa0af9a9f133d0d511c77effd83201b9ff46fa8e8d506\": container with ID starting with b1b0cd3e4d998f7314afa0af9a9f133d0d511c77effd83201b9ff46fa8e8d506 not found: ID does not exist" Jul 12 08:55:13.977322 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.977293 2566 scope.go:117] "RemoveContainer" containerID="a1dffb9761e66da4fd0a89c734e9d6b476960f5839e3932c41e31eb1e9f4bcf4" Jul 12 08:55:13.977516 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:55:13.977500 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1dffb9761e66da4fd0a89c734e9d6b476960f5839e3932c41e31eb1e9f4bcf4\": container with ID starting with a1dffb9761e66da4fd0a89c734e9d6b476960f5839e3932c41e31eb1e9f4bcf4 not found: ID does not exist" containerID="a1dffb9761e66da4fd0a89c734e9d6b476960f5839e3932c41e31eb1e9f4bcf4" Jul 12 08:55:13.977566 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.977520 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1dffb9761e66da4fd0a89c734e9d6b476960f5839e3932c41e31eb1e9f4bcf4"} err="failed to get container status \"a1dffb9761e66da4fd0a89c734e9d6b476960f5839e3932c41e31eb1e9f4bcf4\": rpc error: code = NotFound desc = could not find container \"a1dffb9761e66da4fd0a89c734e9d6b476960f5839e3932c41e31eb1e9f4bcf4\": container with ID starting with a1dffb9761e66da4fd0a89c734e9d6b476960f5839e3932c41e31eb1e9f4bcf4 not found: ID does not exist" Jul 12 08:55:13.981026 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.981004 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t"] Jul 12 08:55:13.986320 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:13.986296 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-ad9984-openshift-default-5975dd9979-2247t"] Jul 12 08:55:15.026606 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:15.026573 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7232660a-9177-4c0a-80f5-785ed909df2c" path="/var/lib/kubelet/pods/7232660a-9177-4c0a-80f5-785ed909df2c/volumes" Jul 12 08:55:16.362288 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:16.362253 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:16.689134 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:16.689045 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:16.968470 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:16.968396 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz"] Jul 12 08:55:16.968899 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:16.968875 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7232660a-9177-4c0a-80f5-785ed909df2c" containerName="istio-proxy" Jul 12 08:55:16.969036 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:16.968945 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7232660a-9177-4c0a-80f5-785ed909df2c" containerName="istio-proxy" Jul 12 08:55:16.969185 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:16.969164 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7232660a-9177-4c0a-80f5-785ed909df2c" containerName="istio-proxy" Jul 12 08:55:16.969286 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:16.969187 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7232660a-9177-4c0a-80f5-785ed909df2c" containerName="istio-proxy" Jul 12 08:55:16.969353 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:16.969290 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7232660a-9177-4c0a-80f5-785ed909df2c" containerName="istio-proxy" Jul 12 08:55:16.969353 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:16.969301 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7232660a-9177-4c0a-80f5-785ed909df2c" containerName="istio-proxy" Jul 12 08:55:16.973485 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:16.973466 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:16.975784 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:16.975763 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-a7f934-openshift-default-dockercfg-z6wln\"" Jul 12 08:55:16.982807 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:16.982781 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz"] Jul 12 08:55:16.996158 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:16.996133 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:17.041352 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:17.041318 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-istio-envoy\") pod \"e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:17.041415 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:17.041367 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-workload-certs\") pod \"e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:17.041415 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:17.041397 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-istio-token\") pod \"e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:17.041575 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:17.041442 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-istio-podinfo\") pod \"e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:17.041575 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:17.041486 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-workload-socket\") pod \"e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:17.041575 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:17.041510 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8xtb\" (UniqueName: \"kubernetes.io/projected/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-kube-api-access-k8xtb\") pod \"e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:17.041575 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:17.041537 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-istiod-ca-cert\") pod \"e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:17.041790 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:17.041592 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-credential-socket\") pod \"e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:17.041790 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:17.041610 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-istio-data\") pod \"e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:17.142321 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:17.142290 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-istio-envoy\") pod \"e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:17.142321 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:17.142331 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-workload-certs\") pod \"e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:17.142604 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:17.142353 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-istio-token\") pod \"e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:17.142604 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:17.142395 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-istio-podinfo\") pod \"e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:17.142604 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:17.142424 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-workload-socket\") pod \"e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:17.142604 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:17.142455 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8xtb\" (UniqueName: \"kubernetes.io/projected/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-kube-api-access-k8xtb\") pod \"e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:17.142604 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:17.142479 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-istiod-ca-cert\") pod \"e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:17.142604 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:17.142524 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-credential-socket\") pod \"e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:17.142604 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:17.142573 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-istio-data\") pod \"e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:17.142972 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:17.142781 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-workload-certs\") pod \"e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:17.142972 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:17.142829 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-workload-socket\") pod \"e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:17.142972 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:17.142895 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-istio-data\") pod \"e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:17.143133 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:17.143037 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-credential-socket\") pod \"e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:17.143345 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:17.143321 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-istiod-ca-cert\") pod \"e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:17.145003 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:17.144979 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-istio-envoy\") pod \"e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:17.145315 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:17.145293 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-istio-podinfo\") pod \"e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:17.149530 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:17.149510 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-istio-token\") pod \"e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:17.149759 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:17.149743 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8xtb\" (UniqueName: \"kubernetes.io/projected/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-kube-api-access-k8xtb\") pod \"e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:17.220532 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:17.220417 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:17.285510 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:17.285486 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:17.422981 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:17.422956 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz"] Jul 12 08:55:17.424987 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:55:17.424961 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e88a4c2_fd91_4d99_8107_0af2e2eb8b7a.slice/crio-c5bd74cebae6c8b902f34c136a80baf41eeb040010ff76f4f17707188f629043 WatchSource:0}: Error finding container c5bd74cebae6c8b902f34c136a80baf41eeb040010ff76f4f17707188f629043: Status 404 returned error can't find the container with id c5bd74cebae6c8b902f34c136a80baf41eeb040010ff76f4f17707188f629043 Jul 12 08:55:17.427074 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:17.427046 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:55:17.427170 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:17.427106 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:55:17.427170 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:17.427133 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:55:17.984197 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:17.984161 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" event={"ID":"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a","Type":"ContainerStarted","Data":"11c026d0308b804683b96eeac88dab80fd8abeeb5c5cdd33c5181f9d1582a8fa"} Jul 12 08:55:17.984391 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:17.984214 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" event={"ID":"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a","Type":"ContainerStarted","Data":"c5bd74cebae6c8b902f34c136a80baf41eeb040010ff76f4f17707188f629043"} Jul 12 08:55:18.003112 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:18.003067 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" podStartSLOduration=2.003050583 podStartE2EDuration="2.003050583s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 08:55:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:55:18.001626013 +0000 UTC m=+2041.662421675" watchObservedRunningTime="2026-07-12 08:55:18.003050583 +0000 UTC m=+2041.663846245" Jul 12 08:55:18.286360 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:18.286276 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:18.290881 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:18.290855 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:18.988806 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:18.988774 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:18.989793 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:18.989770 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:19.906542 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:19.906495 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:20.005052 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:20.005023 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:20.149321 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:20.149279 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:20.282086 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:20.282054 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:20.664181 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:20.664090 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:20.763955 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:20.763925 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:24.414894 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:24.414820 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:26.984725 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:26.984677 2566 patch_prober.go:28] interesting pod/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 12 08:55:26.985202 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:26.984768 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" podUID="18ff97ac-a784-43fd-9f8c-613aad9cb754" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 12 08:55:28.037823 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:28.037786 2566 generic.go:356] "Generic (PLEG): container finished" podID="18ff97ac-a784-43fd-9f8c-613aad9cb754" containerID="76d14b39ca53fb7a89cdaeb7fa4b8d9a6ff5f84ce8e817c52b59f46f04681800" exitCode=0 Jul 12 08:55:28.038276 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:28.037865 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" event={"ID":"18ff97ac-a784-43fd-9f8c-613aad9cb754","Type":"ContainerDied","Data":"76d14b39ca53fb7a89cdaeb7fa4b8d9a6ff5f84ce8e817c52b59f46f04681800"} Jul 12 08:55:28.038388 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:28.038371 2566 scope.go:117] "RemoveContainer" containerID="76d14b39ca53fb7a89cdaeb7fa4b8d9a6ff5f84ce8e817c52b59f46f04681800" Jul 12 08:55:28.039539 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:28.039508 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:55:28.039659 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:28.039621 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:55:28.039659 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:28.039654 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:55:28.907228 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:28.907200 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:29.007385 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:29.007343 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:29.049110 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:29.049063 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" event={"ID":"18ff97ac-a784-43fd-9f8c-613aad9cb754","Type":"ContainerStarted","Data":"992f4293c0bb2dbc291a28c4e8c316a2dc72675f98c801062ca608ea69401a69"} Jul 12 08:55:29.144229 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:29.144195 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:29.144491 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:29.144459 2566 patch_prober.go:28] interesting pod/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 12 08:55:29.144598 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:29.144538 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" podUID="18ff97ac-a784-43fd-9f8c-613aad9cb754" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 12 08:55:30.144806 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:30.144756 2566 patch_prober.go:28] interesting pod/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 12 08:55:30.145341 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:30.144833 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" podUID="18ff97ac-a784-43fd-9f8c-613aad9cb754" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 12 08:55:31.144970 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:31.144918 2566 patch_prober.go:28] interesting pod/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 12 08:55:31.145495 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:31.144995 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" podUID="18ff97ac-a784-43fd-9f8c-613aad9cb754" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 12 08:55:32.145328 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:32.145288 2566 patch_prober.go:28] interesting pod/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 12 08:55:32.145758 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:32.145350 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" podUID="18ff97ac-a784-43fd-9f8c-613aad9cb754" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 12 08:55:33.148135 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:33.148100 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:33.148589 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:33.148494 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:33.149160 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:33.149144 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:55:34.237363 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:34.237321 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:34.338457 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:34.338421 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:34.535103 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:34.534768 2566 patch_prober.go:28] interesting pod/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": read tcp 10.132.0.2:45324->10.132.0.62:15021: read: connection reset by peer" start-of-body= Jul 12 08:55:34.535103 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:34.534848 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" podUID="3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": read tcp 10.132.0.2:45324->10.132.0.62:15021: read: connection reset by peer" Jul 12 08:55:35.086418 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:35.086383 2566 generic.go:356] "Generic (PLEG): container finished" podID="3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a" containerID="11c026d0308b804683b96eeac88dab80fd8abeeb5c5cdd33c5181f9d1582a8fa" exitCode=0 Jul 12 08:55:35.086632 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:35.086457 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" event={"ID":"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a","Type":"ContainerDied","Data":"11c026d0308b804683b96eeac88dab80fd8abeeb5c5cdd33c5181f9d1582a8fa"} Jul 12 08:55:35.086984 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:35.086967 2566 scope.go:117] "RemoveContainer" containerID="11c026d0308b804683b96eeac88dab80fd8abeeb5c5cdd33c5181f9d1582a8fa" Jul 12 08:55:35.087923 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:35.087899 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:55:35.088003 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:35.087963 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:55:35.088003 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:35.087994 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 08:55:36.094473 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:36.094436 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" event={"ID":"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a","Type":"ContainerStarted","Data":"70a3987c5142b6e461f9ea5089f7d3185648e175e418503f47f742c28b5499fa"} Jul 12 08:55:36.286791 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:36.286754 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:36.286999 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:36.286977 2566 patch_prober.go:28] interesting pod/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 12 08:55:36.287061 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:36.287034 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" podUID="3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 12 08:55:37.286892 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:37.286853 2566 patch_prober.go:28] interesting pod/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 12 08:55:37.287289 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:37.286913 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" podUID="3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 12 08:55:38.286278 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:38.286236 2566 patch_prober.go:28] interesting pod/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 12 08:55:38.286434 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:38.286309 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" podUID="3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 12 08:55:39.286529 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:39.286452 2566 patch_prober.go:28] interesting pod/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 12 08:55:39.286529 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:39.286512 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" podUID="3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 12 08:55:40.289330 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:40.289300 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:40.289716 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:40.289662 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:40.290287 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:40.290270 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:55:41.380974 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:41.380939 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:55:41.481054 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:55:41.481019 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:56:16.919077 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:56:16.919049 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-k2ltl_bc3f8e97-c9ab-4370-b437-8a47514c869c/console-operator/1.log" Jul 12 08:56:16.924620 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:56:16.924599 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lfbsq_abe468eb-f6d3-4af4-bf5e-a3c029a404ab/ovn-acl-logging/0.log" Jul 12 08:56:16.927734 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:56:16.927714 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-k2ltl_bc3f8e97-c9ab-4370-b437-8a47514c869c/console-operator/1.log" Jul 12 08:56:16.932966 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:56:16.932951 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lfbsq_abe468eb-f6d3-4af4-bf5e-a3c029a404ab/ovn-acl-logging/0.log" Jul 12 08:56:35.087635 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:56:35.087601 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:56:35.135089 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:56:35.135059 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:56:47.217886 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:56:47.217833 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:57:12.674465 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:57:12.674380 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:57:12.727362 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:57:12.727328 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:58:12.180437 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:12.180400 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:58:29.470668 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:29.470637 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:58:29.911013 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:29.910981 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:58:30.011410 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:30.011376 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:58:30.111742 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:30.111712 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:58:30.233636 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:30.233609 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:58:35.696153 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:35.696115 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:58:35.770124 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:35.770088 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:58:35.922628 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:35.922593 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6"] Jul 12 08:58:35.922853 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:35.922826 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" podUID="18ff97ac-a784-43fd-9f8c-613aad9cb754" containerName="istio-proxy" containerID="cri-o://992f4293c0bb2dbc291a28c4e8c316a2dc72675f98c801062ca608ea69401a69" gracePeriod=30 Jul 12 08:58:35.941705 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:35.941671 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:58:36.342231 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:36.342193 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:58:36.777805 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:36.777766 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:58:36.929781 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:36.929748 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:58:37.032256 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:37.032173 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:58:37.258173 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:37.258137 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:58:41.180869 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:41.180813 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:58:41.252177 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:41.252148 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18ff97ac-a784-43fd-9f8c-613aad9cb754-istiod-ca-cert\") pod \"18ff97ac-a784-43fd-9f8c-613aad9cb754\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " Jul 12 08:58:41.252345 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:41.252196 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18ff97ac-a784-43fd-9f8c-613aad9cb754-credential-socket\") pod \"18ff97ac-a784-43fd-9f8c-613aad9cb754\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " Jul 12 08:58:41.252345 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:41.252239 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18ff97ac-a784-43fd-9f8c-613aad9cb754-istio-token\") pod \"18ff97ac-a784-43fd-9f8c-613aad9cb754\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " Jul 12 08:58:41.252345 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:41.252259 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18ff97ac-a784-43fd-9f8c-613aad9cb754-istio-data\") pod \"18ff97ac-a784-43fd-9f8c-613aad9cb754\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " Jul 12 08:58:41.252345 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:41.252292 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18ff97ac-a784-43fd-9f8c-613aad9cb754-istio-envoy\") pod \"18ff97ac-a784-43fd-9f8c-613aad9cb754\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " Jul 12 08:58:41.252345 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:41.252321 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18ff97ac-a784-43fd-9f8c-613aad9cb754-workload-certs\") pod \"18ff97ac-a784-43fd-9f8c-613aad9cb754\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " Jul 12 08:58:41.252656 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:41.252350 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qvzs\" (UniqueName: \"kubernetes.io/projected/18ff97ac-a784-43fd-9f8c-613aad9cb754-kube-api-access-6qvzs\") pod \"18ff97ac-a784-43fd-9f8c-613aad9cb754\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " Jul 12 08:58:41.252656 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:41.252406 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18ff97ac-a784-43fd-9f8c-613aad9cb754-istio-podinfo\") pod \"18ff97ac-a784-43fd-9f8c-613aad9cb754\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " Jul 12 08:58:41.252656 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:41.252461 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18ff97ac-a784-43fd-9f8c-613aad9cb754-workload-socket\") pod \"18ff97ac-a784-43fd-9f8c-613aad9cb754\" (UID: \"18ff97ac-a784-43fd-9f8c-613aad9cb754\") " Jul 12 08:58:41.252656 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:41.252586 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18ff97ac-a784-43fd-9f8c-613aad9cb754-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "18ff97ac-a784-43fd-9f8c-613aad9cb754" (UID: "18ff97ac-a784-43fd-9f8c-613aad9cb754"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:58:41.252656 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:41.252613 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18ff97ac-a784-43fd-9f8c-613aad9cb754-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "18ff97ac-a784-43fd-9f8c-613aad9cb754" (UID: "18ff97ac-a784-43fd-9f8c-613aad9cb754"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:58:41.252656 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:41.252649 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18ff97ac-a784-43fd-9f8c-613aad9cb754-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "18ff97ac-a784-43fd-9f8c-613aad9cb754" (UID: "18ff97ac-a784-43fd-9f8c-613aad9cb754"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 08:58:41.252991 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:41.252809 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18ff97ac-a784-43fd-9f8c-613aad9cb754-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "18ff97ac-a784-43fd-9f8c-613aad9cb754" (UID: "18ff97ac-a784-43fd-9f8c-613aad9cb754"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:58:41.252991 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:41.252831 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18ff97ac-a784-43fd-9f8c-613aad9cb754-credential-socket\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:58:41.252991 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:41.252845 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18ff97ac-a784-43fd-9f8c-613aad9cb754-workload-certs\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:58:41.252991 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:41.252857 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18ff97ac-a784-43fd-9f8c-613aad9cb754-istiod-ca-cert\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:58:41.253372 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:41.253346 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18ff97ac-a784-43fd-9f8c-613aad9cb754-istio-data" (OuterVolumeSpecName: "istio-data") pod "18ff97ac-a784-43fd-9f8c-613aad9cb754" (UID: "18ff97ac-a784-43fd-9f8c-613aad9cb754"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:58:41.254509 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:41.254484 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18ff97ac-a784-43fd-9f8c-613aad9cb754-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "18ff97ac-a784-43fd-9f8c-613aad9cb754" (UID: "18ff97ac-a784-43fd-9f8c-613aad9cb754"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:58:41.254626 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:41.254608 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18ff97ac-a784-43fd-9f8c-613aad9cb754-kube-api-access-6qvzs" (OuterVolumeSpecName: "kube-api-access-6qvzs") pod "18ff97ac-a784-43fd-9f8c-613aad9cb754" (UID: "18ff97ac-a784-43fd-9f8c-613aad9cb754"). InnerVolumeSpecName "kube-api-access-6qvzs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:58:41.254999 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:41.254978 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18ff97ac-a784-43fd-9f8c-613aad9cb754-istio-token" (OuterVolumeSpecName: "istio-token") pod "18ff97ac-a784-43fd-9f8c-613aad9cb754" (UID: "18ff97ac-a784-43fd-9f8c-613aad9cb754"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:58:41.255063 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:41.254982 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/18ff97ac-a784-43fd-9f8c-613aad9cb754-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "18ff97ac-a784-43fd-9f8c-613aad9cb754" (UID: "18ff97ac-a784-43fd-9f8c-613aad9cb754"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 08:58:41.353874 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:41.353848 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18ff97ac-a784-43fd-9f8c-613aad9cb754-istio-token\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:58:41.353874 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:41.353872 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18ff97ac-a784-43fd-9f8c-613aad9cb754-istio-data\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:58:41.354025 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:41.353881 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18ff97ac-a784-43fd-9f8c-613aad9cb754-istio-envoy\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:58:41.354025 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:41.353890 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qvzs\" (UniqueName: \"kubernetes.io/projected/18ff97ac-a784-43fd-9f8c-613aad9cb754-kube-api-access-6qvzs\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:58:41.354025 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:41.353899 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18ff97ac-a784-43fd-9f8c-613aad9cb754-istio-podinfo\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:58:41.354025 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:41.353907 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18ff97ac-a784-43fd-9f8c-613aad9cb754-workload-socket\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:58:42.024302 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:42.024255 2566 generic.go:356] "Generic (PLEG): container finished" podID="18ff97ac-a784-43fd-9f8c-613aad9cb754" containerID="992f4293c0bb2dbc291a28c4e8c316a2dc72675f98c801062ca608ea69401a69" exitCode=0 Jul 12 08:58:42.024468 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:42.024301 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" event={"ID":"18ff97ac-a784-43fd-9f8c-613aad9cb754","Type":"ContainerDied","Data":"992f4293c0bb2dbc291a28c4e8c316a2dc72675f98c801062ca608ea69401a69"} Jul 12 08:58:42.024468 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:42.024355 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" event={"ID":"18ff97ac-a784-43fd-9f8c-613aad9cb754","Type":"ContainerDied","Data":"df8e806b22cc7c096126e83e7143ceb2fc87421a6a87e1fefaaef973bbae67d0"} Jul 12 08:58:42.024468 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:42.024377 2566 scope.go:117] "RemoveContainer" containerID="992f4293c0bb2dbc291a28c4e8c316a2dc72675f98c801062ca608ea69401a69" Jul 12 08:58:42.024468 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:42.024315 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6" Jul 12 08:58:42.033495 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:42.033480 2566 scope.go:117] "RemoveContainer" containerID="76d14b39ca53fb7a89cdaeb7fa4b8d9a6ff5f84ce8e817c52b59f46f04681800" Jul 12 08:58:42.041745 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:42.041512 2566 scope.go:117] "RemoveContainer" containerID="992f4293c0bb2dbc291a28c4e8c316a2dc72675f98c801062ca608ea69401a69" Jul 12 08:58:42.041817 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:58:42.041799 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"992f4293c0bb2dbc291a28c4e8c316a2dc72675f98c801062ca608ea69401a69\": container with ID starting with 992f4293c0bb2dbc291a28c4e8c316a2dc72675f98c801062ca608ea69401a69 not found: ID does not exist" containerID="992f4293c0bb2dbc291a28c4e8c316a2dc72675f98c801062ca608ea69401a69" Jul 12 08:58:42.041864 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:42.041824 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"992f4293c0bb2dbc291a28c4e8c316a2dc72675f98c801062ca608ea69401a69"} err="failed to get container status \"992f4293c0bb2dbc291a28c4e8c316a2dc72675f98c801062ca608ea69401a69\": rpc error: code = NotFound desc = could not find container \"992f4293c0bb2dbc291a28c4e8c316a2dc72675f98c801062ca608ea69401a69\": container with ID starting with 992f4293c0bb2dbc291a28c4e8c316a2dc72675f98c801062ca608ea69401a69 not found: ID does not exist" Jul 12 08:58:42.041864 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:42.041839 2566 scope.go:117] "RemoveContainer" containerID="76d14b39ca53fb7a89cdaeb7fa4b8d9a6ff5f84ce8e817c52b59f46f04681800" Jul 12 08:58:42.042037 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:58:42.042022 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76d14b39ca53fb7a89cdaeb7fa4b8d9a6ff5f84ce8e817c52b59f46f04681800\": container with ID starting with 76d14b39ca53fb7a89cdaeb7fa4b8d9a6ff5f84ce8e817c52b59f46f04681800 not found: ID does not exist" containerID="76d14b39ca53fb7a89cdaeb7fa4b8d9a6ff5f84ce8e817c52b59f46f04681800" Jul 12 08:58:42.042080 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:42.042042 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76d14b39ca53fb7a89cdaeb7fa4b8d9a6ff5f84ce8e817c52b59f46f04681800"} err="failed to get container status \"76d14b39ca53fb7a89cdaeb7fa4b8d9a6ff5f84ce8e817c52b59f46f04681800\": rpc error: code = NotFound desc = could not find container \"76d14b39ca53fb7a89cdaeb7fa4b8d9a6ff5f84ce8e817c52b59f46f04681800\": container with ID starting with 76d14b39ca53fb7a89cdaeb7fa4b8d9a6ff5f84ce8e817c52b59f46f04681800 not found: ID does not exist" Jul 12 08:58:42.045935 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:42.045905 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6"] Jul 12 08:58:42.047820 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:42.047798 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-a67649-openshift-default-57f494bd75-lq6s6"] Jul 12 08:58:42.807594 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:42.807537 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:58:42.908423 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:42.908390 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:58:43.032224 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:43.032185 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18ff97ac-a784-43fd-9f8c-613aad9cb754" path="/var/lib/kubelet/pods/18ff97ac-a784-43fd-9f8c-613aad9cb754/volumes" Jul 12 08:58:43.047530 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:43.047496 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz"] Jul 12 08:58:43.047817 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:43.047790 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" podUID="3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a" containerName="istio-proxy" containerID="cri-o://70a3987c5142b6e461f9ea5089f7d3185648e175e418503f47f742c28b5499fa" gracePeriod=30 Jul 12 08:58:43.062215 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:43.062157 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:58:43.821681 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:43.821646 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-d67dd4f5c-748fl_61815d5f-206f-43d9-abde-cee35b61f923/authorino/0.log" Jul 12 08:58:48.310063 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:48.310001 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:58:48.419188 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:48.419152 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-istio-envoy\") pod \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " Jul 12 08:58:48.419188 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:48.419185 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-istio-token\") pod \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " Jul 12 08:58:48.419444 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:48.419206 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-workload-socket\") pod \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " Jul 12 08:58:48.419444 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:48.419224 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-istio-data\") pod \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " Jul 12 08:58:48.419444 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:48.419262 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-credential-socket\") pod \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " Jul 12 08:58:48.419444 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:48.419303 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-istio-podinfo\") pod \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " Jul 12 08:58:48.419444 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:48.419347 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8xtb\" (UniqueName: \"kubernetes.io/projected/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-kube-api-access-k8xtb\") pod \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " Jul 12 08:58:48.419739 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:48.419492 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-workload-certs\") pod \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " Jul 12 08:58:48.419739 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:48.419495 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a" (UID: "3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:58:48.419739 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:48.419591 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-istiod-ca-cert\") pod \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\" (UID: \"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a\") " Jul 12 08:58:48.419739 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:48.419693 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a" (UID: "3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:58:48.419932 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:48.419834 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a" (UID: "3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:58:48.420006 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:48.419982 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-credential-socket\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:58:48.420058 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:48.420008 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-workload-certs\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:58:48.420058 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:48.420009 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a" (UID: "3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 08:58:48.420058 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:48.420024 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-workload-socket\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:58:48.420186 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:48.420150 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-istio-data" (OuterVolumeSpecName: "istio-data") pod "3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a" (UID: "3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:58:48.421490 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:48.421466 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-istio-token" (OuterVolumeSpecName: "istio-token") pod "3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a" (UID: "3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:58:48.421709 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:48.421682 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a" (UID: "3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 08:58:48.422010 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:48.421985 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-kube-api-access-k8xtb" (OuterVolumeSpecName: "kube-api-access-k8xtb") pod "3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a" (UID: "3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a"). InnerVolumeSpecName "kube-api-access-k8xtb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 08:58:48.422094 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:48.422074 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a" (UID: "3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 08:58:48.520772 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:48.520737 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-istiod-ca-cert\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:58:48.520772 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:48.520766 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-istio-envoy\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:58:48.520772 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:48.520777 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-istio-token\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:58:48.521001 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:48.520785 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-istio-data\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:58:48.521001 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:48.520796 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-istio-podinfo\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:58:48.521001 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:48.520809 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8xtb\" (UniqueName: \"kubernetes.io/projected/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a-kube-api-access-k8xtb\") on node \"ip-10-0-136-138.ec2.internal\" DevicePath \"\"" Jul 12 08:58:48.599667 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:48.599594 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-69948768-xzlj6_745fa8c6-2e93-43bf-b960-97c09b42ced2/manager/0.log" Jul 12 08:58:49.066386 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:49.066358 2566 generic.go:356] "Generic (PLEG): container finished" podID="3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a" containerID="70a3987c5142b6e461f9ea5089f7d3185648e175e418503f47f742c28b5499fa" exitCode=0 Jul 12 08:58:49.066520 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:49.066404 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" Jul 12 08:58:49.066520 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:49.066437 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" event={"ID":"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a","Type":"ContainerDied","Data":"70a3987c5142b6e461f9ea5089f7d3185648e175e418503f47f742c28b5499fa"} Jul 12 08:58:49.066520 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:49.066491 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz" event={"ID":"3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a","Type":"ContainerDied","Data":"c5bd74cebae6c8b902f34c136a80baf41eeb040010ff76f4f17707188f629043"} Jul 12 08:58:49.066520 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:49.066516 2566 scope.go:117] "RemoveContainer" containerID="70a3987c5142b6e461f9ea5089f7d3185648e175e418503f47f742c28b5499fa" Jul 12 08:58:49.075609 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:49.075592 2566 scope.go:117] "RemoveContainer" containerID="11c026d0308b804683b96eeac88dab80fd8abeeb5c5cdd33c5181f9d1582a8fa" Jul 12 08:58:49.082894 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:49.082874 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz"] Jul 12 08:58:49.083805 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:49.083789 2566 scope.go:117] "RemoveContainer" containerID="70a3987c5142b6e461f9ea5089f7d3185648e175e418503f47f742c28b5499fa" Jul 12 08:58:49.084076 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:58:49.084058 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70a3987c5142b6e461f9ea5089f7d3185648e175e418503f47f742c28b5499fa\": container with ID starting with 70a3987c5142b6e461f9ea5089f7d3185648e175e418503f47f742c28b5499fa not found: ID does not exist" containerID="70a3987c5142b6e461f9ea5089f7d3185648e175e418503f47f742c28b5499fa" Jul 12 08:58:49.084149 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:49.084087 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70a3987c5142b6e461f9ea5089f7d3185648e175e418503f47f742c28b5499fa"} err="failed to get container status \"70a3987c5142b6e461f9ea5089f7d3185648e175e418503f47f742c28b5499fa\": rpc error: code = NotFound desc = could not find container \"70a3987c5142b6e461f9ea5089f7d3185648e175e418503f47f742c28b5499fa\": container with ID starting with 70a3987c5142b6e461f9ea5089f7d3185648e175e418503f47f742c28b5499fa not found: ID does not exist" Jul 12 08:58:49.084149 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:49.084110 2566 scope.go:117] "RemoveContainer" containerID="11c026d0308b804683b96eeac88dab80fd8abeeb5c5cdd33c5181f9d1582a8fa" Jul 12 08:58:49.084375 ip-10-0-136-138 kubenswrapper[2566]: E0712 08:58:49.084359 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11c026d0308b804683b96eeac88dab80fd8abeeb5c5cdd33c5181f9d1582a8fa\": container with ID starting with 11c026d0308b804683b96eeac88dab80fd8abeeb5c5cdd33c5181f9d1582a8fa not found: ID does not exist" containerID="11c026d0308b804683b96eeac88dab80fd8abeeb5c5cdd33c5181f9d1582a8fa" Jul 12 08:58:49.084432 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:49.084383 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11c026d0308b804683b96eeac88dab80fd8abeeb5c5cdd33c5181f9d1582a8fa"} err="failed to get container status \"11c026d0308b804683b96eeac88dab80fd8abeeb5c5cdd33c5181f9d1582a8fa\": rpc error: code = NotFound desc = could not find container \"11c026d0308b804683b96eeac88dab80fd8abeeb5c5cdd33c5181f9d1582a8fa\": container with ID starting with 11c026d0308b804683b96eeac88dab80fd8abeeb5c5cdd33c5181f9d1582a8fa not found: ID does not exist" Jul 12 08:58:49.087277 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:49.087258 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-a7f934-openshift-default-67d6bd6959-jnctz"] Jul 12 08:58:49.201313 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:49.201278 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4_19bc5347-542e-4e2d-8de3-1a2c1cb298c0/copy-manifests/0.log" Jul 12 08:58:49.207923 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:49.207898 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4_19bc5347-542e-4e2d-8de3-1a2c1cb298c0/manager/0.log" Jul 12 08:58:50.496758 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:50.496729 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-d67dd4f5c-748fl_61815d5f-206f-43d9-abde-cee35b61f923/authorino/0.log" Jul 12 08:58:50.616340 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:50.616312 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-t49ds_1394821b-2a26-4cf2-9f53-33432f999f4c/manager/0.log" Jul 12 08:58:50.725033 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:50.725007 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-4k8lq_c6d21d48-2a6e-4ecf-bcf4-3fc979d11e2a/manager/0.log" Jul 12 08:58:50.940004 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:50.939931 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-4nvqv_3898a1d5-9053-4ddf-9a3f-b1c27dafc611/registry-server/0.log" Jul 12 08:58:51.027064 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:51.027027 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a" path="/var/lib/kubelet/pods/3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a/volumes" Jul 12 08:58:52.229230 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:52.229202 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-r8g9d_f325a226-e2c4-44e0-ab4c-81b481a5427f/payload-pre-processing/0.log" Jul 12 08:58:52.341089 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:52.341061 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-f6sp4_76d96201-18a0-4e3d-bfa8-3b34e6f52117/payload-processing/0.log" Jul 12 08:58:52.454349 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:52.454325 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-b8989f8c6-v5dgq_1bd2626f-14a5-450e-ae9a-6968c5aa9055/router/0.log" Jul 12 08:58:53.034363 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:53.034334 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-5mzk6_1965a474-d47b-4dcb-af7d-f6e332f4c45a/main/0.log" Jul 12 08:58:57.456317 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:57.456276 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:58:57.556536 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:58:57.556504 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" limit=1024 Jul 12 08:59:00.353142 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:00.353111 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-w4cfw_0f993375-bf43-40a2-b8c6-558a97dcba75/global-pull-secret-syncer/0.log" Jul 12 08:59:00.401336 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:00.401309 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8mwdg_37e6726a-f4aa-431c-8d02-961f53c45038/konnectivity-agent/0.log" Jul 12 08:59:00.500520 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:00.500491 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-138.ec2.internal_4f7c17a66495e6fef40567703d162ecc/haproxy/0.log" Jul 12 08:59:05.429421 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:05.429371 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-d67dd4f5c-748fl_61815d5f-206f-43d9-abde-cee35b61f923/authorino/0.log" Jul 12 08:59:05.458752 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:05.458727 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-t49ds_1394821b-2a26-4cf2-9f53-33432f999f4c/manager/0.log" Jul 12 08:59:05.484596 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:05.484574 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-4k8lq_c6d21d48-2a6e-4ecf-bcf4-3fc979d11e2a/manager/0.log" Jul 12 08:59:05.551748 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:05.551720 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-4nvqv_3898a1d5-9053-4ddf-9a3f-b1c27dafc611/registry-server/0.log" Jul 12 08:59:07.628409 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:07.628380 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-kxpkk_f8037f69-5405-41e4-9473-f2e22adbbb17/cluster-monitoring-operator/0.log" Jul 12 08:59:07.889832 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:07.889808 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5gtms_f2a2abd2-70f9-42e1-b68d-7026bc1be2d2/node-exporter/0.log" Jul 12 08:59:07.911187 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:07.911164 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5gtms_f2a2abd2-70f9-42e1-b68d-7026bc1be2d2/kube-rbac-proxy/0.log" Jul 12 08:59:07.930440 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:07.930423 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5gtms_f2a2abd2-70f9-42e1-b68d-7026bc1be2d2/init-textfile/0.log" Jul 12 08:59:08.325441 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:08.325407 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-bxvgb_5e9e1ae0-3d7c-4a0f-bdf1-694141807cc5/prometheus-operator-admission-webhook/0.log" Jul 12 08:59:09.740420 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:09.740391 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-8t6pv_96c69912-dd41-420f-9c38-a7b64b14d50f/networking-console-plugin/0.log" Jul 12 08:59:10.327057 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:10.327029 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-k2ltl_bc3f8e97-c9ab-4370-b437-8a47514c869c/console-operator/1.log" Jul 12 08:59:10.332087 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:10.332064 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-k2ltl_bc3f8e97-c9ab-4370-b437-8a47514c869c/console-operator/2.log" Jul 12 08:59:10.481682 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:10.481649 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-zjv2n/perf-node-gather-daemonset-c4wp8"] Jul 12 08:59:10.482010 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:10.481998 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a" containerName="istio-proxy" Jul 12 08:59:10.482055 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:10.482012 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a" containerName="istio-proxy" Jul 12 08:59:10.482055 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:10.482025 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18ff97ac-a784-43fd-9f8c-613aad9cb754" containerName="istio-proxy" Jul 12 08:59:10.482055 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:10.482031 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="18ff97ac-a784-43fd-9f8c-613aad9cb754" containerName="istio-proxy" Jul 12 08:59:10.482150 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:10.482107 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="18ff97ac-a784-43fd-9f8c-613aad9cb754" containerName="istio-proxy" Jul 12 08:59:10.482150 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:10.482122 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e88a4c2-fd91-4d99-8107-0af2e2eb8b7a" containerName="istio-proxy" Jul 12 08:59:10.485331 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:10.485312 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zjv2n/perf-node-gather-daemonset-c4wp8" Jul 12 08:59:10.487460 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:10.487444 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zjv2n\"/\"kube-root-ca.crt\"" Jul 12 08:59:10.488198 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:10.488184 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-zjv2n\"/\"default-dockercfg-rz8s7\"" Jul 12 08:59:10.488257 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:10.488211 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zjv2n\"/\"openshift-service-ca.crt\"" Jul 12 08:59:10.494818 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:10.494796 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zjv2n/perf-node-gather-daemonset-c4wp8"] Jul 12 08:59:10.608000 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:10.607926 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3b212433-c5fd-4f4c-86de-201eb4ec3672-lib-modules\") pod \"perf-node-gather-daemonset-c4wp8\" (UID: \"3b212433-c5fd-4f4c-86de-201eb4ec3672\") " pod="openshift-must-gather-zjv2n/perf-node-gather-daemonset-c4wp8" Jul 12 08:59:10.608000 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:10.607964 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3b212433-c5fd-4f4c-86de-201eb4ec3672-podres\") pod \"perf-node-gather-daemonset-c4wp8\" (UID: \"3b212433-c5fd-4f4c-86de-201eb4ec3672\") " pod="openshift-must-gather-zjv2n/perf-node-gather-daemonset-c4wp8" Jul 12 08:59:10.608190 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:10.608041 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3b212433-c5fd-4f4c-86de-201eb4ec3672-proc\") pod \"perf-node-gather-daemonset-c4wp8\" (UID: \"3b212433-c5fd-4f4c-86de-201eb4ec3672\") " pod="openshift-must-gather-zjv2n/perf-node-gather-daemonset-c4wp8" Jul 12 08:59:10.608190 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:10.608084 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3b212433-c5fd-4f4c-86de-201eb4ec3672-sys\") pod \"perf-node-gather-daemonset-c4wp8\" (UID: \"3b212433-c5fd-4f4c-86de-201eb4ec3672\") " pod="openshift-must-gather-zjv2n/perf-node-gather-daemonset-c4wp8" Jul 12 08:59:10.608190 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:10.608118 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkx59\" (UniqueName: \"kubernetes.io/projected/3b212433-c5fd-4f4c-86de-201eb4ec3672-kube-api-access-dkx59\") pod \"perf-node-gather-daemonset-c4wp8\" (UID: \"3b212433-c5fd-4f4c-86de-201eb4ec3672\") " pod="openshift-must-gather-zjv2n/perf-node-gather-daemonset-c4wp8" Jul 12 08:59:10.709071 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:10.709038 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3b212433-c5fd-4f4c-86de-201eb4ec3672-sys\") pod \"perf-node-gather-daemonset-c4wp8\" (UID: \"3b212433-c5fd-4f4c-86de-201eb4ec3672\") " pod="openshift-must-gather-zjv2n/perf-node-gather-daemonset-c4wp8" Jul 12 08:59:10.709218 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:10.709098 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkx59\" (UniqueName: \"kubernetes.io/projected/3b212433-c5fd-4f4c-86de-201eb4ec3672-kube-api-access-dkx59\") pod \"perf-node-gather-daemonset-c4wp8\" (UID: \"3b212433-c5fd-4f4c-86de-201eb4ec3672\") " pod="openshift-must-gather-zjv2n/perf-node-gather-daemonset-c4wp8" Jul 12 08:59:10.709218 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:10.709146 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3b212433-c5fd-4f4c-86de-201eb4ec3672-sys\") pod \"perf-node-gather-daemonset-c4wp8\" (UID: \"3b212433-c5fd-4f4c-86de-201eb4ec3672\") " pod="openshift-must-gather-zjv2n/perf-node-gather-daemonset-c4wp8" Jul 12 08:59:10.709218 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:10.709150 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3b212433-c5fd-4f4c-86de-201eb4ec3672-lib-modules\") pod \"perf-node-gather-daemonset-c4wp8\" (UID: \"3b212433-c5fd-4f4c-86de-201eb4ec3672\") " pod="openshift-must-gather-zjv2n/perf-node-gather-daemonset-c4wp8" Jul 12 08:59:10.709218 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:10.709196 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3b212433-c5fd-4f4c-86de-201eb4ec3672-podres\") pod \"perf-node-gather-daemonset-c4wp8\" (UID: \"3b212433-c5fd-4f4c-86de-201eb4ec3672\") " pod="openshift-must-gather-zjv2n/perf-node-gather-daemonset-c4wp8" Jul 12 08:59:10.709372 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:10.709238 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3b212433-c5fd-4f4c-86de-201eb4ec3672-proc\") pod \"perf-node-gather-daemonset-c4wp8\" (UID: \"3b212433-c5fd-4f4c-86de-201eb4ec3672\") " pod="openshift-must-gather-zjv2n/perf-node-gather-daemonset-c4wp8" Jul 12 08:59:10.709372 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:10.709269 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3b212433-c5fd-4f4c-86de-201eb4ec3672-lib-modules\") pod \"perf-node-gather-daemonset-c4wp8\" (UID: \"3b212433-c5fd-4f4c-86de-201eb4ec3672\") " pod="openshift-must-gather-zjv2n/perf-node-gather-daemonset-c4wp8" Jul 12 08:59:10.709372 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:10.709309 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3b212433-c5fd-4f4c-86de-201eb4ec3672-proc\") pod \"perf-node-gather-daemonset-c4wp8\" (UID: \"3b212433-c5fd-4f4c-86de-201eb4ec3672\") " pod="openshift-must-gather-zjv2n/perf-node-gather-daemonset-c4wp8" Jul 12 08:59:10.709372 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:10.709308 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3b212433-c5fd-4f4c-86de-201eb4ec3672-podres\") pod \"perf-node-gather-daemonset-c4wp8\" (UID: \"3b212433-c5fd-4f4c-86de-201eb4ec3672\") " pod="openshift-must-gather-zjv2n/perf-node-gather-daemonset-c4wp8" Jul 12 08:59:10.716561 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:10.716524 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkx59\" (UniqueName: \"kubernetes.io/projected/3b212433-c5fd-4f4c-86de-201eb4ec3672-kube-api-access-dkx59\") pod \"perf-node-gather-daemonset-c4wp8\" (UID: \"3b212433-c5fd-4f4c-86de-201eb4ec3672\") " pod="openshift-must-gather-zjv2n/perf-node-gather-daemonset-c4wp8" Jul 12 08:59:10.796107 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:10.796082 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zjv2n/perf-node-gather-daemonset-c4wp8" Jul 12 08:59:10.921583 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:10.921536 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-nslt6_360e4c5b-afe0-4bb4-88d7-5e419497559d/download-server/0.log" Jul 12 08:59:10.924781 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:10.924762 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zjv2n/perf-node-gather-daemonset-c4wp8"] Jul 12 08:59:10.927212 ip-10-0-136-138 kubenswrapper[2566]: W0712 08:59:10.927187 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3b212433_c5fd_4f4c_86de_201eb4ec3672.slice/crio-004eed1e6da92b7bdc3fd3fe941722c4e7922de1ada7fc8e098a7356bf472938 WatchSource:0}: Error finding container 004eed1e6da92b7bdc3fd3fe941722c4e7922de1ada7fc8e098a7356bf472938: Status 404 returned error can't find the container with id 004eed1e6da92b7bdc3fd3fe941722c4e7922de1ada7fc8e098a7356bf472938 Jul 12 08:59:11.172997 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:11.172908 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zjv2n/perf-node-gather-daemonset-c4wp8" event={"ID":"3b212433-c5fd-4f4c-86de-201eb4ec3672","Type":"ContainerStarted","Data":"83fb805d44227a386039956c8ad4abd4b2e5a614a62fe47b1e5f8ae5a7958991"} Jul 12 08:59:11.172997 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:11.172958 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zjv2n/perf-node-gather-daemonset-c4wp8" event={"ID":"3b212433-c5fd-4f4c-86de-201eb4ec3672","Type":"ContainerStarted","Data":"004eed1e6da92b7bdc3fd3fe941722c4e7922de1ada7fc8e098a7356bf472938"} Jul 12 08:59:11.172997 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:11.172987 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-zjv2n/perf-node-gather-daemonset-c4wp8" Jul 12 08:59:11.189597 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:11.189539 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-zjv2n/perf-node-gather-daemonset-c4wp8" podStartSLOduration=1.1895229 podStartE2EDuration="1.1895229s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 08:59:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 08:59:11.18874178 +0000 UTC m=+2274.849537443" watchObservedRunningTime="2026-07-12 08:59:11.1895229 +0000 UTC m=+2274.850318564" Jul 12 08:59:11.492472 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:11.492448 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-k9kgv_29040aa3-9ad7-4c27-9ed5-fcc5339a5e28/volume-data-source-validator/0.log" Jul 12 08:59:12.520384 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:12.520354 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b756k_088c2a49-9c89-44f2-b75a-68dbeda985a6/dns/0.log" Jul 12 08:59:12.539889 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:12.539866 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b756k_088c2a49-9c89-44f2-b75a-68dbeda985a6/kube-rbac-proxy/0.log" Jul 12 08:59:12.631071 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:12.631048 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lwln2_ccef3090-87ec-4e72-a24c-5dd65f19c55b/dns-node-resolver/0.log" Jul 12 08:59:13.228697 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:13.228664 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pzgtb_6cd77bb0-5bbe-4d49-9e22-05dad8c650af/node-ca/0.log" Jul 12 08:59:14.395057 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:14.395003 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-r8g9d_f325a226-e2c4-44e0-ab4c-81b481a5427f/payload-pre-processing/0.log" Jul 12 08:59:14.436657 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:14.436634 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-f6sp4_76d96201-18a0-4e3d-bfa8-3b34e6f52117/payload-processing/0.log" Jul 12 08:59:14.460268 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:14.460249 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-b8989f8c6-v5dgq_1bd2626f-14a5-450e-ae9a-6968c5aa9055/router/0.log" Jul 12 08:59:15.133697 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:15.133668 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4p4w4_f5e63ffc-280a-49f6-8861-6b5bcf16ac29/serve-healthcheck-canary/0.log" Jul 12 08:59:15.610949 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:15.610922 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-rvkmd_e142390b-f773-488a-92de-6883d5f4ad70/insights-operator/0.log" Jul 12 08:59:15.611734 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:15.611716 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-rvkmd_e142390b-f773-488a-92de-6883d5f4ad70/insights-operator/1.log" Jul 12 08:59:15.629894 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:15.629864 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-98qmc_636c08ec-7cba-4f5e-a5b5-4db9f35be60a/kube-rbac-proxy/0.log" Jul 12 08:59:15.648523 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:15.648502 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-98qmc_636c08ec-7cba-4f5e-a5b5-4db9f35be60a/exporter/0.log" Jul 12 08:59:15.668610 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:15.668592 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-98qmc_636c08ec-7cba-4f5e-a5b5-4db9f35be60a/extractor/0.log" Jul 12 08:59:17.188440 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:17.188416 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-zjv2n/perf-node-gather-daemonset-c4wp8" Jul 12 08:59:18.055422 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:18.055385 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-69948768-xzlj6_745fa8c6-2e93-43bf-b960-97c09b42ced2/manager/0.log" Jul 12 08:59:18.309023 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:18.308932 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4_19bc5347-542e-4e2d-8de3-1a2c1cb298c0/manager/0.log" Jul 12 08:59:18.327183 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:18.327160 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d4d9c68b7-nwcz4_19bc5347-542e-4e2d-8de3-1a2c1cb298c0/copy-manifests/0.log" Jul 12 08:59:19.404244 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:19.404199 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7cb89b9bb7-b7r6r_7a86bf2f-527f-4468-a37b-3cd9bcd91c6e/manager/0.log" Jul 12 08:59:21.119102 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:21.119021 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-r8g9d_f325a226-e2c4-44e0-ab4c-81b481a5427f/payload-pre-processing/0.log" Jul 12 08:59:21.154042 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:21.154005 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-f6sp4_76d96201-18a0-4e3d-bfa8-3b34e6f52117/payload-processing/0.log" Jul 12 08:59:21.172067 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:21.172043 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-b8989f8c6-v5dgq_1bd2626f-14a5-450e-ae9a-6968c5aa9055/router/0.log" Jul 12 08:59:22.682740 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:22.682711 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-r8g9d_f325a226-e2c4-44e0-ab4c-81b481a5427f/payload-pre-processing/0.log" Jul 12 08:59:22.722710 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:22.722676 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-f6sp4_76d96201-18a0-4e3d-bfa8-3b34e6f52117/payload-processing/0.log" Jul 12 08:59:22.737027 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:22.737004 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-b8989f8c6-v5dgq_1bd2626f-14a5-450e-ae9a-6968c5aa9055/router/0.log" Jul 12 08:59:24.809916 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:24.809886 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-6l8ww_55cb2ce1-ea8b-4fc7-9a14-7280c499cf67/migrator/0.log" Jul 12 08:59:24.829369 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:24.829346 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-6l8ww_55cb2ce1-ea8b-4fc7-9a14-7280c499cf67/graceful-termination/0.log" Jul 12 08:59:25.172054 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:25.171972 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-n9s6q_5cf8c8a8-825a-404d-b968-dfb4f4f466df/kube-storage-version-migrator-operator/1.log" Jul 12 08:59:25.172891 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:25.172871 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-n9s6q_5cf8c8a8-825a-404d-b968-dfb4f4f466df/kube-storage-version-migrator-operator/0.log" Jul 12 08:59:26.465841 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:26.465813 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xvhw4_a69a78ba-31f4-488d-a7a5-0d2a197ef775/kube-multus-additional-cni-plugins/0.log" Jul 12 08:59:26.506250 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:26.506224 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xvhw4_a69a78ba-31f4-488d-a7a5-0d2a197ef775/egress-router-binary-copy/0.log" Jul 12 08:59:26.538691 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:26.538665 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xvhw4_a69a78ba-31f4-488d-a7a5-0d2a197ef775/cni-plugins/0.log" Jul 12 08:59:26.579817 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:26.579783 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xvhw4_a69a78ba-31f4-488d-a7a5-0d2a197ef775/bond-cni-plugin/0.log" Jul 12 08:59:26.614822 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:26.614794 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xvhw4_a69a78ba-31f4-488d-a7a5-0d2a197ef775/routeoverride-cni/0.log" Jul 12 08:59:26.636632 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:26.636609 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xvhw4_a69a78ba-31f4-488d-a7a5-0d2a197ef775/whereabouts-cni-bincopy/0.log" Jul 12 08:59:26.656222 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:26.656203 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xvhw4_a69a78ba-31f4-488d-a7a5-0d2a197ef775/whereabouts-cni/0.log" Jul 12 08:59:26.748765 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:26.748740 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-pfjqv_3dcfbd82-b903-426b-9573-7e68317111b0/kube-multus/0.log" Jul 12 08:59:26.854345 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:26.854322 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9ms85_508b3df5-2cbe-4cd6-a8fd-e6cb29188184/network-metrics-daemon/0.log" Jul 12 08:59:26.871761 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:26.871742 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9ms85_508b3df5-2cbe-4cd6-a8fd-e6cb29188184/kube-rbac-proxy/0.log" Jul 12 08:59:27.861517 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:27.861489 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lfbsq_abe468eb-f6d3-4af4-bf5e-a3c029a404ab/ovn-controller/0.log" Jul 12 08:59:27.883695 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:27.883672 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lfbsq_abe468eb-f6d3-4af4-bf5e-a3c029a404ab/ovn-acl-logging/0.log" Jul 12 08:59:27.883838 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:27.883675 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lfbsq_abe468eb-f6d3-4af4-bf5e-a3c029a404ab/ovn-acl-logging/1.log" Jul 12 08:59:27.899624 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:27.899596 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lfbsq_abe468eb-f6d3-4af4-bf5e-a3c029a404ab/kube-rbac-proxy-node/0.log" Jul 12 08:59:27.917694 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:27.917673 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lfbsq_abe468eb-f6d3-4af4-bf5e-a3c029a404ab/kube-rbac-proxy-ovn-metrics/0.log" Jul 12 08:59:27.936819 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:27.936801 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lfbsq_abe468eb-f6d3-4af4-bf5e-a3c029a404ab/northd/0.log" Jul 12 08:59:27.957539 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:27.957517 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lfbsq_abe468eb-f6d3-4af4-bf5e-a3c029a404ab/nbdb/0.log" Jul 12 08:59:27.978312 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:27.978279 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lfbsq_abe468eb-f6d3-4af4-bf5e-a3c029a404ab/sbdb/0.log" Jul 12 08:59:28.033110 ip-10-0-136-138 kubenswrapper[2566]: I0712 08:59:28.033053 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lfbsq_abe468eb-f6d3-4af4-bf5e-a3c029a404ab/ovnkube-controller/0.log"