Jul 15 12:20:25.612119 ip-10-0-139-188 systemd[1]: Starting Kubernetes Kubelet... Jul 15 12:20:26.083879 ip-10-0-139-188 systemd[1]: Started Kubernetes Kubelet. Jul 15 12:20:26.087524 ip-10-0-139-188 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 12:20:26.087524 ip-10-0-139-188 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 15 12:20:26.087524 ip-10-0-139-188 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 12:20:26.087524 ip-10-0-139-188 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 15 12:20:26.087524 ip-10-0-139-188 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 12:20:26.087524 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.011288 2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 15 12:20:26.087524 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015351 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 12:20:26.087524 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015363 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 12:20:26.087524 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015367 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 12:20:26.087524 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015371 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 12:20:26.087524 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015374 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 12:20:26.087524 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015377 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 12:20:26.087524 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015379 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 12:20:26.087524 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015382 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 12:20:26.088456 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015385 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 12:20:26.088456 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015388 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 12:20:26.088456 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015391 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 12:20:26.088456 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015394 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 12:20:26.088456 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015397 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 12:20:26.088456 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015400 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 12:20:26.088456 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015403 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 12:20:26.088456 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015407 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 12:20:26.088456 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015409 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 12:20:26.088456 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015412 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 12:20:26.088456 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015415 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 12:20:26.088456 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015417 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 12:20:26.088456 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015420 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 12:20:26.088456 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015422 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 12:20:26.088456 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015425 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 12:20:26.088456 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015428 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 12:20:26.088456 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015431 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 12:20:26.088456 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015437 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 12:20:26.088456 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015439 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 12:20:26.088456 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015442 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 12:20:26.089282 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015445 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 12:20:26.089282 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015447 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 12:20:26.089282 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015450 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 12:20:26.089282 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015454 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 12:20:26.089282 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015458 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 12:20:26.089282 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015460 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 12:20:26.089282 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015463 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 12:20:26.089282 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015466 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 12:20:26.089282 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015468 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 12:20:26.089282 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015471 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 12:20:26.089282 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015473 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 12:20:26.089282 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015476 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 12:20:26.089282 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015478 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 12:20:26.089282 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015481 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 12:20:26.089282 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015483 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 12:20:26.089282 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015486 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 12:20:26.089282 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015488 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 12:20:26.089282 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015491 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 12:20:26.089282 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015493 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 12:20:26.089282 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015497 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 15 12:20:26.089942 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015500 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 12:20:26.089942 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015502 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 12:20:26.089942 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015505 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 12:20:26.089942 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015507 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 12:20:26.089942 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015510 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 12:20:26.089942 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015514 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 12:20:26.089942 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015517 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 12:20:26.089942 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015520 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 12:20:26.089942 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015524 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 12:20:26.089942 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015527 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 12:20:26.089942 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015530 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 12:20:26.089942 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015533 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 12:20:26.089942 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015535 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 12:20:26.089942 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015538 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 12:20:26.089942 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015541 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 12:20:26.089942 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015543 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 12:20:26.089942 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015546 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 12:20:26.089942 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015548 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 12:20:26.089942 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015551 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 12:20:26.090991 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015553 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 12:20:26.090991 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015556 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 12:20:26.090991 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015559 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 12:20:26.090991 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015561 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 12:20:26.090991 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015564 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 12:20:26.090991 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015566 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 12:20:26.090991 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015569 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 12:20:26.090991 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015571 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 12:20:26.090991 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015574 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 12:20:26.090991 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015577 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 12:20:26.090991 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015580 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 12:20:26.090991 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015582 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 12:20:26.090991 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015586 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 12:20:26.090991 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015589 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 12:20:26.090991 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015591 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 12:20:26.090991 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015594 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 12:20:26.090991 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015596 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 12:20:26.090991 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015599 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 12:20:26.090991 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015601 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 12:20:26.091791 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015604 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 12:20:26.091791 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015607 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 12:20:26.091791 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015611 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 12:20:26.091791 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015615 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 12:20:26.091791 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015617 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 12:20:26.091791 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015620 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 12:20:26.091791 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015623 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 12:20:26.091791 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015626 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 12:20:26.091791 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015628 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 12:20:26.091791 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015631 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 12:20:26.091791 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015634 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 12:20:26.091791 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015637 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 12:20:26.091791 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.015639 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 12:20:26.091791 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016335 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 12:20:26.091791 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016345 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 12:20:26.091791 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016349 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 12:20:26.091791 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016354 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 12:20:26.091791 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016358 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 12:20:26.091791 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016361 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 12:20:26.091791 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016364 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 12:20:26.092709 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016366 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 12:20:26.092709 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016369 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 12:20:26.092709 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016372 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 12:20:26.092709 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016375 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 12:20:26.092709 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016378 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 12:20:26.092709 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016380 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 12:20:26.092709 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016383 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 12:20:26.092709 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016385 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 12:20:26.092709 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016388 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 12:20:26.092709 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016391 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 12:20:26.092709 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016393 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 12:20:26.092709 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016396 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 12:20:26.092709 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016399 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 12:20:26.092709 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016401 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 12:20:26.092709 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016404 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 12:20:26.092709 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016406 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 12:20:26.092709 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016409 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 12:20:26.092709 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016412 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 12:20:26.092709 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016414 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 12:20:26.092709 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016417 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 12:20:26.093503 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016420 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 12:20:26.093503 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016422 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 12:20:26.093503 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016425 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 12:20:26.093503 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016428 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 12:20:26.093503 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016430 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 12:20:26.093503 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016434 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 12:20:26.093503 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016436 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 12:20:26.093503 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016439 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 12:20:26.093503 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016442 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 12:20:26.093503 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016445 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 12:20:26.093503 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016448 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 12:20:26.093503 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016450 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 12:20:26.093503 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016453 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 15 12:20:26.093503 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016455 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 12:20:26.093503 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016458 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 12:20:26.093503 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016460 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 12:20:26.093503 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016463 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 12:20:26.093503 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016465 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 12:20:26.093503 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016468 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 12:20:26.093503 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016471 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 12:20:26.094120 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016473 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 12:20:26.094120 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016476 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 12:20:26.094120 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016478 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 12:20:26.094120 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016481 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 12:20:26.094120 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016483 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 12:20:26.094120 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016486 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 12:20:26.094120 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016491 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 12:20:26.094120 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016496 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 12:20:26.094120 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016499 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 12:20:26.094120 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016502 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 12:20:26.094120 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016505 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 12:20:26.094120 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016508 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 12:20:26.094120 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016510 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 12:20:26.094120 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016513 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 12:20:26.094120 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016516 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 12:20:26.094120 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016519 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 12:20:26.094120 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016522 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 12:20:26.094120 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016526 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 12:20:26.094120 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016530 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 12:20:26.094910 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016533 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 12:20:26.094910 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016536 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 12:20:26.094910 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016539 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 12:20:26.094910 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016542 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 12:20:26.094910 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016545 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 12:20:26.094910 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016548 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 12:20:26.094910 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016552 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 12:20:26.094910 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016558 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 12:20:26.094910 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016560 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 12:20:26.094910 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016563 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 12:20:26.094910 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016565 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 12:20:26.094910 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016568 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 12:20:26.094910 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016571 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 12:20:26.094910 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016573 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 12:20:26.094910 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016576 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 12:20:26.094910 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016578 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 12:20:26.094910 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016581 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 12:20:26.094910 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016583 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 12:20:26.094910 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016586 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 12:20:26.095600 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016588 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 12:20:26.095600 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016591 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 12:20:26.095600 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016594 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 12:20:26.095600 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016597 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 12:20:26.095600 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016600 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 12:20:26.095600 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016602 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 12:20:26.095600 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016605 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 12:20:26.095600 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016607 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 12:20:26.095600 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016610 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 12:20:26.095600 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016614 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 12:20:26.095600 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016617 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 12:20:26.095600 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016620 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 12:20:26.095600 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016623 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 12:20:26.095600 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016626 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 12:20:26.095600 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.016995 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 12:20:26.095600 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017001 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 12:20:26.095600 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017005 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 12:20:26.095600 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017008 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 12:20:26.095600 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017011 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 12:20:26.095600 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017014 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 12:20:26.096486 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017019 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 12:20:26.096486 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017023 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 12:20:26.096486 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017027 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 12:20:26.096486 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017030 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 12:20:26.096486 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017033 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 12:20:26.096486 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017036 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 12:20:26.096486 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017039 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 12:20:26.096486 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017041 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 12:20:26.096486 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017044 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 12:20:26.096486 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017046 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 12:20:26.096486 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017050 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 12:20:26.096486 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017052 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 12:20:26.096486 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017056 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 12:20:26.096486 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017060 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 12:20:26.096486 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017063 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 12:20:26.096486 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017065 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 12:20:26.096486 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017068 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 12:20:26.096486 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017071 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 12:20:26.096486 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017073 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 12:20:26.097582 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017076 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 12:20:26.097582 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017079 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 12:20:26.097582 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017081 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 12:20:26.097582 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017084 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 12:20:26.097582 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017087 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 12:20:26.097582 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017090 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 12:20:26.097582 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017093 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 12:20:26.097582 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017096 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 12:20:26.097582 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017098 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 12:20:26.097582 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017101 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 12:20:26.097582 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017104 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 12:20:26.097582 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017106 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 12:20:26.097582 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017109 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 12:20:26.097582 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017113 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 12:20:26.097582 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017116 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 12:20:26.097582 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017119 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 12:20:26.097582 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017122 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 12:20:26.097582 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017124 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 12:20:26.097582 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017127 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 12:20:26.098662 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017129 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 12:20:26.098662 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017133 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 12:20:26.098662 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017136 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 12:20:26.098662 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017139 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 12:20:26.098662 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017142 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 12:20:26.098662 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017144 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 12:20:26.098662 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017147 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 12:20:26.098662 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017149 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 12:20:26.098662 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017152 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 12:20:26.098662 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017154 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 12:20:26.098662 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017157 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 12:20:26.098662 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017159 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 12:20:26.098662 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017162 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 12:20:26.098662 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017164 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 15 12:20:26.098662 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017167 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 12:20:26.098662 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017172 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 12:20:26.098662 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017174 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 12:20:26.098662 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017177 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 12:20:26.098662 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017179 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 12:20:26.099756 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017183 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 12:20:26.099756 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017185 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 12:20:26.099756 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017187 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 12:20:26.099756 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017190 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 12:20:26.099756 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017193 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 12:20:26.099756 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017196 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 12:20:26.099756 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017198 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 12:20:26.099756 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017202 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 12:20:26.099756 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017205 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 12:20:26.099756 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017207 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 12:20:26.099756 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017210 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 12:20:26.099756 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017212 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 12:20:26.099756 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017215 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 12:20:26.099756 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017217 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 12:20:26.099756 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017220 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 12:20:26.099756 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017223 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 12:20:26.099756 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017225 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 12:20:26.099756 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017228 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 12:20:26.099756 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017230 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 12:20:26.099756 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017232 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 12:20:26.100834 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017235 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 12:20:26.100834 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017237 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 12:20:26.100834 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017240 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 12:20:26.100834 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017243 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 12:20:26.100834 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017245 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 12:20:26.100834 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017248 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 12:20:26.100834 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017250 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 12:20:26.100834 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017253 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 12:20:26.100834 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017257 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 12:20:26.100834 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017259 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 12:20:26.100834 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017262 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 12:20:26.100834 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017265 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 12:20:26.100834 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017267 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 12:20:26.100834 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017270 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 12:20:26.100834 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017272 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 12:20:26.100834 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017275 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 12:20:26.100834 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017353 2560 flags.go:64] FLAG: --address="0.0.0.0" Jul 15 12:20:26.100834 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017366 2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 15 12:20:26.100834 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017373 2560 flags.go:64] FLAG: --anonymous-auth="true" Jul 15 12:20:26.100834 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017380 2560 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 15 12:20:26.100834 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017384 2560 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 15 12:20:26.102157 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017387 2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 15 12:20:26.102157 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017391 2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 15 12:20:26.102157 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017395 2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 15 12:20:26.102157 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017398 2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 15 12:20:26.102157 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017402 2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 15 12:20:26.102157 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017405 2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 15 12:20:26.102157 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017408 2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 15 12:20:26.102157 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017412 2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 15 12:20:26.102157 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017415 2560 flags.go:64] FLAG: --cgroup-root="" Jul 15 12:20:26.102157 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017418 2560 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 15 12:20:26.102157 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017421 2560 flags.go:64] FLAG: --client-ca-file="" Jul 15 12:20:26.102157 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017423 2560 flags.go:64] FLAG: --cloud-provider="external" Jul 15 12:20:26.102157 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017427 2560 flags.go:64] FLAG: --cluster-dns="[]" Jul 15 12:20:26.102157 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017431 2560 flags.go:64] FLAG: --cluster-domain="" Jul 15 12:20:26.102157 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017434 2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 15 12:20:26.102157 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017437 2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 15 12:20:26.102157 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017440 2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 15 12:20:26.102157 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017443 2560 flags.go:64] FLAG: --container-log-max-files="5" Jul 15 12:20:26.102157 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017447 2560 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 15 12:20:26.102157 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017451 2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 15 12:20:26.102157 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017454 2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 15 12:20:26.102157 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017457 2560 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 15 12:20:26.102157 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017460 2560 flags.go:64] FLAG: --contention-profiling="false" Jul 15 12:20:26.102157 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017463 2560 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 15 12:20:26.103502 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017466 2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 15 12:20:26.103502 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017469 2560 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 15 12:20:26.103502 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017472 2560 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 15 12:20:26.103502 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017476 2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 15 12:20:26.103502 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017479 2560 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 15 12:20:26.103502 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017481 2560 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 15 12:20:26.103502 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017486 2560 flags.go:64] FLAG: --enable-load-reader="false" Jul 15 12:20:26.103502 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017489 2560 flags.go:64] FLAG: --enable-server="true" Jul 15 12:20:26.103502 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017492 2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 15 12:20:26.103502 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017495 2560 flags.go:64] FLAG: --event-burst="100" Jul 15 12:20:26.103502 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017498 2560 flags.go:64] FLAG: --event-qps="50" Jul 15 12:20:26.103502 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017501 2560 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 15 12:20:26.103502 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017504 2560 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 15 12:20:26.103502 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017507 2560 flags.go:64] FLAG: --eviction-hard="" Jul 15 12:20:26.103502 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017511 2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 15 12:20:26.103502 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017513 2560 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 15 12:20:26.103502 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017516 2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 15 12:20:26.103502 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017519 2560 flags.go:64] FLAG: --eviction-soft="" Jul 15 12:20:26.103502 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017522 2560 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 15 12:20:26.103502 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017525 2560 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 15 12:20:26.103502 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017527 2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 15 12:20:26.103502 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017530 2560 flags.go:64] FLAG: --experimental-mounter-path="" Jul 15 12:20:26.103502 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017533 2560 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 15 12:20:26.103502 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017536 2560 flags.go:64] FLAG: --fail-swap-on="true" Jul 15 12:20:26.103502 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017539 2560 flags.go:64] FLAG: --feature-gates="" Jul 15 12:20:26.105263 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017542 2560 flags.go:64] FLAG: --file-check-frequency="20s" Jul 15 12:20:26.105263 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017545 2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 15 12:20:26.105263 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017549 2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 15 12:20:26.105263 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017552 2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 15 12:20:26.105263 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017555 2560 flags.go:64] FLAG: --healthz-port="10248" Jul 15 12:20:26.105263 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017558 2560 flags.go:64] FLAG: --help="false" Jul 15 12:20:26.105263 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017561 2560 flags.go:64] FLAG: --hostname-override="ip-10-0-139-188.ec2.internal" Jul 15 12:20:26.105263 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017564 2560 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 15 12:20:26.105263 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017567 2560 flags.go:64] FLAG: --http-check-frequency="20s" Jul 15 12:20:26.105263 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017569 2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 15 12:20:26.105263 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017573 2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 15 12:20:26.105263 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017576 2560 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 15 12:20:26.105263 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017579 2560 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 15 12:20:26.105263 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017584 2560 flags.go:64] FLAG: --image-service-endpoint="" Jul 15 12:20:26.105263 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017587 2560 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 15 12:20:26.105263 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017590 2560 flags.go:64] FLAG: --kube-api-burst="100" Jul 15 12:20:26.105263 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017593 2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 15 12:20:26.105263 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017596 2560 flags.go:64] FLAG: --kube-api-qps="50" Jul 15 12:20:26.105263 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017599 2560 flags.go:64] FLAG: --kube-reserved="" Jul 15 12:20:26.105263 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017602 2560 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 15 12:20:26.105263 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017605 2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 15 12:20:26.105263 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017608 2560 flags.go:64] FLAG: --kubelet-cgroups="" Jul 15 12:20:26.105263 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017611 2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 15 12:20:26.105263 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017614 2560 flags.go:64] FLAG: --lock-file="" Jul 15 12:20:26.108601 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017616 2560 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 15 12:20:26.108601 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017619 2560 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 15 12:20:26.108601 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017622 2560 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 15 12:20:26.108601 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017627 2560 flags.go:64] FLAG: --log-json-split-stream="false" Jul 15 12:20:26.108601 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017630 2560 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 15 12:20:26.108601 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017633 2560 flags.go:64] FLAG: --log-text-split-stream="false" Jul 15 12:20:26.108601 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017636 2560 flags.go:64] FLAG: --logging-format="text" Jul 15 12:20:26.108601 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017639 2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 15 12:20:26.108601 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017642 2560 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 15 12:20:26.108601 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017645 2560 flags.go:64] FLAG: --manifest-url="" Jul 15 12:20:26.108601 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017649 2560 flags.go:64] FLAG: --manifest-url-header="" Jul 15 12:20:26.108601 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017653 2560 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 15 12:20:26.108601 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017656 2560 flags.go:64] FLAG: --max-open-files="1000000" Jul 15 12:20:26.108601 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017662 2560 flags.go:64] FLAG: --max-pods="110" Jul 15 12:20:26.108601 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017665 2560 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 15 12:20:26.108601 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017668 2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 15 12:20:26.108601 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017671 2560 flags.go:64] FLAG: --memory-manager-policy="None" Jul 15 12:20:26.108601 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017673 2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 15 12:20:26.108601 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017676 2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 15 12:20:26.108601 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017679 2560 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 15 12:20:26.108601 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017682 2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 15 12:20:26.108601 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017690 2560 flags.go:64] FLAG: --node-status-max-images="50" Jul 15 12:20:26.108601 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017693 2560 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 15 12:20:26.108601 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017697 2560 flags.go:64] FLAG: --oom-score-adj="-999" Jul 15 12:20:26.108601 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017700 2560 flags.go:64] FLAG: --pod-cidr="" Jul 15 12:20:26.110107 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017703 2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 15 12:20:26.110107 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017708 2560 flags.go:64] FLAG: --pod-manifest-path="" Jul 15 12:20:26.110107 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017711 2560 flags.go:64] FLAG: --pod-max-pids="-1" Jul 15 12:20:26.110107 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017714 2560 flags.go:64] FLAG: --pods-per-core="0" Jul 15 12:20:26.110107 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017717 2560 flags.go:64] FLAG: --port="10250" Jul 15 12:20:26.110107 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017720 2560 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 15 12:20:26.110107 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017723 2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00100b3041e8806b4" Jul 15 12:20:26.110107 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017726 2560 flags.go:64] FLAG: --qos-reserved="" Jul 15 12:20:26.110107 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017729 2560 flags.go:64] FLAG: --read-only-port="10255" Jul 15 12:20:26.110107 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017732 2560 flags.go:64] FLAG: --register-node="true" Jul 15 12:20:26.110107 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017735 2560 flags.go:64] FLAG: --register-with-taints="" Jul 15 12:20:26.110107 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017738 2560 flags.go:64] FLAG: --registry-burst="10" Jul 15 12:20:26.110107 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017741 2560 flags.go:64] FLAG: --registry-qps="5" Jul 15 12:20:26.110107 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017744 2560 flags.go:64] FLAG: --reserved-cpus="" Jul 15 12:20:26.110107 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017746 2560 flags.go:64] FLAG: --reserved-memory="" Jul 15 12:20:26.110107 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017750 2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 15 12:20:26.110107 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017753 2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 15 12:20:26.110107 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017757 2560 flags.go:64] FLAG: --rotate-certificates="false" Jul 15 12:20:26.110107 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017760 2560 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 15 12:20:26.110107 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017763 2560 flags.go:64] FLAG: --runonce="false" Jul 15 12:20:26.110107 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017766 2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 15 12:20:26.110107 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017769 2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 15 12:20:26.110107 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017771 2560 flags.go:64] FLAG: --seccomp-default="false" Jul 15 12:20:26.110107 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017775 2560 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 15 12:20:26.110107 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017777 2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 15 12:20:26.112182 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017780 2560 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 15 12:20:26.112182 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017783 2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 15 12:20:26.112182 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017786 2560 flags.go:64] FLAG: --storage-driver-password="root" Jul 15 12:20:26.112182 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017789 2560 flags.go:64] FLAG: --storage-driver-secure="false" Jul 15 12:20:26.112182 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017793 2560 flags.go:64] FLAG: --storage-driver-table="stats" Jul 15 12:20:26.112182 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017796 2560 flags.go:64] FLAG: --storage-driver-user="root" Jul 15 12:20:26.112182 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017799 2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 15 12:20:26.112182 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017802 2560 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 15 12:20:26.112182 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017805 2560 flags.go:64] FLAG: --system-cgroups="" Jul 15 12:20:26.112182 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017808 2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 15 12:20:26.112182 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017813 2560 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 15 12:20:26.112182 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017816 2560 flags.go:64] FLAG: --tls-cert-file="" Jul 15 12:20:26.112182 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017819 2560 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 15 12:20:26.112182 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017823 2560 flags.go:64] FLAG: --tls-min-version="" Jul 15 12:20:26.112182 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017826 2560 flags.go:64] FLAG: --tls-private-key-file="" Jul 15 12:20:26.112182 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017828 2560 flags.go:64] FLAG: --topology-manager-policy="none" Jul 15 12:20:26.112182 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017831 2560 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 15 12:20:26.112182 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017834 2560 flags.go:64] FLAG: --topology-manager-scope="container" Jul 15 12:20:26.112182 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017837 2560 flags.go:64] FLAG: --v="2" Jul 15 12:20:26.112182 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017841 2560 flags.go:64] FLAG: --version="false" Jul 15 12:20:26.112182 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017845 2560 flags.go:64] FLAG: --vmodule="" Jul 15 12:20:26.112182 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017861 2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 15 12:20:26.112182 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.017865 2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 15 12:20:26.112182 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017977 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 12:20:26.112182 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017982 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 12:20:26.113466 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017985 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 12:20:26.113466 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017988 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 15 12:20:26.113466 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017991 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 12:20:26.113466 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017994 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 12:20:26.113466 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.017997 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 12:20:26.113466 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018000 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 12:20:26.113466 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018003 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 12:20:26.113466 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018006 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 12:20:26.113466 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018008 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 12:20:26.113466 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018011 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 12:20:26.113466 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018014 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 12:20:26.113466 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018018 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 12:20:26.113466 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018021 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 12:20:26.113466 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018023 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 12:20:26.113466 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018026 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 12:20:26.113466 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018029 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 12:20:26.113466 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018031 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 12:20:26.113466 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018034 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 12:20:26.113466 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018037 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 12:20:26.113466 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018039 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 12:20:26.114471 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018042 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 12:20:26.114471 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018044 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 12:20:26.114471 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018047 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 12:20:26.114471 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018050 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 12:20:26.114471 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018053 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 12:20:26.114471 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018055 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 12:20:26.114471 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018058 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 12:20:26.114471 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018060 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 12:20:26.114471 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018063 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 12:20:26.114471 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018065 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 12:20:26.114471 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018068 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 12:20:26.114471 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018071 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 12:20:26.114471 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018074 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 12:20:26.114471 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018077 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 12:20:26.114471 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018079 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 12:20:26.114471 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018081 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 12:20:26.114471 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018084 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 12:20:26.114471 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018086 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 12:20:26.114471 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018089 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 12:20:26.114471 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018091 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 12:20:26.115230 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018094 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 12:20:26.115230 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018097 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 12:20:26.115230 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018099 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 12:20:26.115230 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018103 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 12:20:26.115230 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018106 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 12:20:26.115230 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018108 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 12:20:26.115230 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018111 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 12:20:26.115230 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018116 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 12:20:26.115230 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018119 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 12:20:26.115230 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018122 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 12:20:26.115230 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018125 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 12:20:26.115230 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018128 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 12:20:26.115230 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018131 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 12:20:26.115230 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018134 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 12:20:26.115230 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018137 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 12:20:26.115230 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018140 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 12:20:26.115230 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018143 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 12:20:26.115230 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018146 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 12:20:26.115230 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018148 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 12:20:26.239890 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018151 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 12:20:26.239890 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018155 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 12:20:26.239890 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018158 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 12:20:26.239890 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018161 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 12:20:26.239890 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018164 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 12:20:26.239890 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018167 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 12:20:26.239890 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018171 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 12:20:26.239890 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018174 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 12:20:26.239890 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018177 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 12:20:26.239890 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018179 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 12:20:26.239890 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018182 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 12:20:26.239890 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018184 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 12:20:26.239890 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018187 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 12:20:26.239890 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018189 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 12:20:26.239890 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018192 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 12:20:26.239890 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018194 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 12:20:26.239890 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018198 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 12:20:26.239890 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018201 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 12:20:26.239890 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018203 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 12:20:26.254812 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018206 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 12:20:26.254812 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018209 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 12:20:26.254812 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018212 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 12:20:26.254812 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018214 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 12:20:26.254812 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018217 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 12:20:26.254812 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018220 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 12:20:26.254812 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018222 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 12:20:26.254812 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018225 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 12:20:26.254812 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018227 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 12:20:26.254812 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018230 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 12:20:26.254812 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018232 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 12:20:26.254812 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018235 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 12:20:26.254812 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018237 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 12:20:26.254812 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018240 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 12:20:26.254812 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018242 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 12:20:26.254812 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018245 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 12:20:26.254812 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018247 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 12:20:26.254812 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018251 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 12:20:26.254812 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.018254 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 12:20:26.256309 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.018883 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 12:20:26.256309 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.025394 2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 15 12:20:26.256309 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.025409 2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 15 12:20:26.256309 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.026111 2560 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 15 12:20:26.256309 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.026118 2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 15 12:20:26.256309 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026178 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 12:20:26.256309 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026184 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 12:20:26.256309 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026187 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 12:20:26.256309 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026190 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 12:20:26.256309 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026193 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 12:20:26.256309 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026196 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 12:20:26.256309 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026199 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 12:20:26.256309 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026202 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 12:20:26.256309 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026206 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 12:20:26.257083 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026210 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 12:20:26.257083 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026213 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 12:20:26.257083 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026216 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 12:20:26.257083 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026218 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 12:20:26.257083 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026221 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 12:20:26.257083 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026224 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 12:20:26.257083 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026227 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 12:20:26.257083 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026229 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 12:20:26.257083 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026232 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 12:20:26.257083 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026235 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 12:20:26.257083 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026238 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 12:20:26.257083 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026240 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 12:20:26.257083 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026243 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 12:20:26.257083 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026246 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 12:20:26.257083 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026248 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 12:20:26.257083 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026251 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 12:20:26.257083 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026253 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 12:20:26.257083 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026256 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 12:20:26.257083 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026258 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 15 12:20:26.257747 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026261 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 12:20:26.257747 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026265 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 12:20:26.257747 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026267 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 12:20:26.257747 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026270 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 12:20:26.257747 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026274 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 12:20:26.257747 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026278 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 12:20:26.257747 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026281 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 12:20:26.257747 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026284 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 12:20:26.257747 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026286 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 12:20:26.257747 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026289 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 12:20:26.257747 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026293 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 12:20:26.257747 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026296 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 12:20:26.257747 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026299 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 12:20:26.257747 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026302 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 12:20:26.257747 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026304 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 12:20:26.257747 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026307 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 12:20:26.257747 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026309 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 12:20:26.257747 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026312 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 12:20:26.257747 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026314 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 12:20:26.258891 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026317 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 12:20:26.258891 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026319 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 12:20:26.258891 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026322 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 12:20:26.258891 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026325 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 12:20:26.258891 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026328 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 12:20:26.258891 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026331 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 12:20:26.258891 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026333 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 12:20:26.258891 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026336 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 12:20:26.258891 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026338 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 12:20:26.258891 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026341 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 12:20:26.258891 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026343 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 12:20:26.258891 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026346 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 12:20:26.258891 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026348 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 12:20:26.258891 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026350 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 12:20:26.258891 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026353 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 12:20:26.258891 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026356 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 12:20:26.258891 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026359 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 12:20:26.258891 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026362 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 12:20:26.258891 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026364 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 12:20:26.258891 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026367 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 12:20:26.259535 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026370 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 12:20:26.259535 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026372 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 12:20:26.259535 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026375 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 12:20:26.259535 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026377 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 12:20:26.259535 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026379 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 12:20:26.259535 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026382 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 12:20:26.259535 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026385 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 12:20:26.259535 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026388 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 12:20:26.259535 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026390 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 12:20:26.259535 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026393 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 12:20:26.259535 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026396 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 12:20:26.259535 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026398 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 12:20:26.259535 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026400 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 12:20:26.259535 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026403 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 12:20:26.259535 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026408 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 12:20:26.259535 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026410 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 12:20:26.259535 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026413 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 12:20:26.259535 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026416 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 12:20:26.259535 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026419 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 12:20:26.259535 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026421 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 12:20:26.260182 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026424 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 12:20:26.260182 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026426 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 12:20:26.260182 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026429 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 12:20:26.260182 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026431 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 12:20:26.260182 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026434 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 12:20:26.260182 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026436 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 12:20:26.260182 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026441 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 12:20:26.260182 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026444 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 12:20:26.260182 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026447 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 12:20:26.260182 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026450 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 12:20:26.260182 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026452 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 12:20:26.260182 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026455 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 12:20:26.260182 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.026460 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 12:20:26.260182 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026568 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 12:20:26.260182 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026574 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 12:20:26.260726 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026577 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 12:20:26.260726 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026579 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 12:20:26.260726 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026582 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 12:20:26.260726 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026585 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 12:20:26.260726 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026588 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 12:20:26.260726 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026591 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 12:20:26.260726 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026593 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 12:20:26.260726 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026596 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 12:20:26.260726 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026598 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 12:20:26.260726 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026601 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 12:20:26.260726 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026604 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 12:20:26.260726 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026608 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 12:20:26.260726 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026610 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 12:20:26.260726 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026613 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 12:20:26.260726 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026615 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 12:20:26.260726 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026618 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 12:20:26.260726 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026620 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 12:20:26.260726 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026623 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 12:20:26.260726 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026625 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 12:20:26.260726 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026628 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 12:20:26.261345 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026630 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 12:20:26.261345 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026633 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 12:20:26.261345 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026636 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 12:20:26.261345 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026638 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 12:20:26.261345 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026642 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 12:20:26.261345 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026644 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 12:20:26.261345 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026647 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 12:20:26.261345 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026650 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 12:20:26.261345 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026654 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 12:20:26.261345 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026658 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 12:20:26.261345 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026660 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 12:20:26.261345 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026663 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 12:20:26.261345 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026666 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 12:20:26.261345 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026669 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 12:20:26.261345 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026672 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 12:20:26.261345 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026674 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 12:20:26.261345 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026677 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 12:20:26.261345 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026679 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 12:20:26.261345 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026682 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 12:20:26.261345 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026685 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 12:20:26.261971 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026687 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 12:20:26.261971 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026690 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 12:20:26.261971 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026692 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 12:20:26.261971 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026696 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 12:20:26.261971 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026699 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 12:20:26.261971 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026701 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 12:20:26.261971 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026704 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 12:20:26.261971 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026706 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 12:20:26.261971 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026709 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 12:20:26.261971 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026711 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 12:20:26.261971 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026714 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 12:20:26.261971 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026716 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 12:20:26.261971 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026719 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 12:20:26.261971 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026721 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 12:20:26.261971 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026724 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 12:20:26.261971 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026726 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 12:20:26.261971 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026730 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 12:20:26.261971 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026734 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 12:20:26.261971 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026736 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 12:20:26.261971 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026739 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 12:20:26.262760 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026742 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 12:20:26.262760 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026744 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 12:20:26.262760 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026747 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 12:20:26.262760 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026750 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 12:20:26.262760 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026753 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 12:20:26.262760 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026755 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 12:20:26.262760 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026758 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 12:20:26.262760 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026760 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 12:20:26.262760 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026763 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 12:20:26.262760 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026765 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 12:20:26.262760 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026768 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 12:20:26.262760 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026770 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 12:20:26.262760 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026773 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 12:20:26.262760 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026775 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 12:20:26.262760 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026778 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 12:20:26.262760 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026782 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 12:20:26.262760 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026784 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 12:20:26.262760 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026787 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 12:20:26.262760 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026789 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 12:20:26.262760 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026792 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 12:20:26.263377 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026795 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 12:20:26.263377 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026797 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 12:20:26.263377 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026800 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 12:20:26.263377 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026802 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 12:20:26.263377 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026805 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 12:20:26.263377 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026807 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 12:20:26.263377 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026810 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 12:20:26.263377 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026812 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 12:20:26.263377 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026816 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 12:20:26.263377 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026818 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 12:20:26.263377 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026821 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 12:20:26.263377 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026824 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 15 12:20:26.263377 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026828 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 12:20:26.263377 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026831 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 12:20:26.263377 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026833 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 12:20:26.263377 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026837 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 12:20:26.263377 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:26.026841 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 12:20:26.264660 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.026846 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 12:20:26.264660 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.026962 2560 server.go:962] "Client rotation is on, will bootstrap in background" Jul 15 12:20:26.264660 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.028938 2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 15 12:20:26.264660 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.030238 2560 server.go:1019] "Starting client certificate rotation" Jul 15 12:20:26.264660 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.030336 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 12:20:26.264660 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.030405 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 12:20:26.264660 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.033000 2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 12:20:26.264660 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.036083 2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 12:20:26.264660 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.047323 2560 log.go:25] "Validated CRI v1 runtime API" Jul 15 12:20:26.264660 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.056351 2560 log.go:25] "Validated CRI v1 image API" Jul 15 12:20:26.264660 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.057845 2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 15 12:20:26.264660 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.060395 2560 fs.go:135] Filesystem UUIDs: map[0363268a-29f9-4f03-b775-62022b28bb8d:/dev/nvme0n1p4 23a172a4-a489-443e-afda-92d3783e3645:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 15 12:20:26.265085 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.060410 2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 15 12:20:26.266127 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.064897 2560 manager.go:216] Machine: {Timestamp:2026-07-15 12:20:26.064414225 +0000 UTC m=+0.335996472 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100936 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2974f07105298ca0f6ad5942d0b96d SystemUUID:ec2974f0-7105-298c-a0f6-ad5942d0b96d BootID:28c3614b-451f-4da7-ba76-c41254b26272 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:77:82:f4:5d:ff Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:77:82:f4:5d:ff Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0e:e5:e8:e8:44:90 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 15 12:20:26.266127 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.064987 2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 15 12:20:26.266127 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.065093 2560 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 15 12:20:26.266127 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.066216 2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 15 12:20:26.266127 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.066246 2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-188.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 15 12:20:26.266127 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.066883 2560 topology_manager.go:138] "Creating topology manager with none policy" Jul 15 12:20:26.266127 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.066893 2560 container_manager_linux.go:309] "Creating device plugin manager" Jul 15 12:20:26.266127 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.066901 2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.067617 2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.068756 2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.069624 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.071814 2560 state_mem.go:36] "Initialized new in-memory state store" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.071942 2560 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.071968 2560 kubelet.go:486] "Attempting to sync node with API server" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.071977 2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.071994 2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.072007 2560 kubelet.go:398] "Adding apiserver pod source" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.072017 2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.072361 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.072377 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.075424 2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.077196 2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.077242 2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.077451 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.077471 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.077484 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.077493 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.077504 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.077514 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.077951 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.077959 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.077966 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.077972 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.077988 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.077996 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.078568 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.078576 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.082954 2560 server.go:1268] "Started kubelet" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.083039 2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.083042 2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.083116 2560 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.084531 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-188.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.090075 2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:26.091732 2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.091767 2560 server.go:310] "Adding debug handlers to kubelet server" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.095714 2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.095751 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:26.096020 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-188.ec2.internal\" not found" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.096508 2560 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 15 12:20:27.535036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.096524 2560 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 15 12:20:27.536524 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.096534 2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 15 12:20:27.536524 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.096623 2560 reconstruct.go:97] "Volume reconstruction finished" Jul 15 12:20:27.536524 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.096629 2560 reconciler.go:29] "Reconciler: start to sync state" Jul 15 12:20:27.536524 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:26.097595 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-188.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 15 12:20:27.536524 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:26.097771 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 15 12:20:27.536524 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.098039 2560 factory.go:55] Registering systemd factory Jul 15 12:20:27.536524 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.098071 2560 factory.go:223] Registration of the systemd container factory successfully Jul 15 12:20:27.536524 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.098281 2560 factory.go:153] Registering CRI-O factory Jul 15 12:20:27.536524 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.098295 2560 factory.go:223] Registration of the crio container factory successfully Jul 15 12:20:27.536524 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.098342 2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 15 12:20:27.536524 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.098369 2560 factory.go:103] Registering Raw factory Jul 15 12:20:27.536524 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.098383 2560 manager.go:1196] Started watching for new ooms in manager Jul 15 12:20:27.536524 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.098746 2560 manager.go:349] Starting recovery of all containers Jul 15 12:20:27.536524 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:26.097519 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-188.ec2.internal.18c275e45e0b3ff6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-188.ec2.internal,UID:ip-10-0-139-188.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-188.ec2.internal,},FirstTimestamp:2026-07-15 12:20:26.082926582 +0000 UTC m=+0.354508813,LastTimestamp:2026-07-15 12:20:26.082926582 +0000 UTC m=+0.354508813,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-188.ec2.internal,}" Jul 15 12:20:27.536524 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:26.101147 2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-188.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 15 12:20:27.536524 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:26.101297 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 15 12:20:27.536524 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.104009 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4mw89" Jul 15 12:20:27.536524 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.108345 2560 manager.go:354] Recovery completed Jul 15 12:20:27.536524 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.113579 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 12:20:27.536524 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.115883 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-188.ec2.internal" event="NodeHasSufficientMemory" Jul 15 12:20:27.536524 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.115915 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 12:20:27.536524 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.115925 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-188.ec2.internal" event="NodeHasSufficientPID" Jul 15 12:20:27.536524 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.116413 2560 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 15 12:20:27.536524 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.116422 2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 15 12:20:27.536524 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.116438 2560 state_mem.go:36] "Initialized new in-memory state store" Jul 15 12:20:27.536524 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.116466 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 12:20:27.536524 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.117223 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-188.ec2.internal" event="NodeHasSufficientMemory" Jul 15 12:20:27.536524 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.117251 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 12:20:27.536524 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.117264 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-188.ec2.internal" event="NodeHasSufficientPID" Jul 15 12:20:27.537435 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.117777 2560 policy_none.go:49] "None policy: Start" Jul 15 12:20:27.537435 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.117792 2560 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 15 12:20:27.537435 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.117805 2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 15 12:20:27.537435 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:26.118224 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-188.ec2.internal.18c275e4600265e1 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-188.ec2.internal,UID:ip-10-0-139-188.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-188.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-188.ec2.internal,},FirstTimestamp:2026-07-15 12:20:26.115900897 +0000 UTC m=+0.387483123,LastTimestamp:2026-07-15 12:20:26.115900897 +0000 UTC m=+0.387483123,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-188.ec2.internal,}" Jul 15 12:20:27.537435 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.120507 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4mw89" Jul 15 12:20:27.537435 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.122561 2560 policy_none.go:47] "Start" Jul 15 12:20:27.537435 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.161968 2560 manager.go:337] "Starting Device Plugin manager" Jul 15 12:20:27.537435 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:26.162167 2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 15 12:20:27.537435 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.162181 2560 server.go:87] "Starting device plugin registration server" Jul 15 12:20:27.537435 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.162457 2560 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 15 12:20:27.537435 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.162469 2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 15 12:20:27.537435 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.162887 2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 15 12:20:27.537435 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.162960 2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 15 12:20:27.537435 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.162969 2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 15 12:20:27.537435 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:26.163596 2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 15 12:20:27.537435 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:26.163641 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-188.ec2.internal\" not found" Jul 15 12:20:27.537435 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.263080 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 12:20:27.537435 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.269230 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-188.ec2.internal" event="NodeHasSufficientMemory" Jul 15 12:20:27.537435 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.269259 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 12:20:27.537435 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.269269 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-188.ec2.internal" event="NodeHasSufficientPID" Jul 15 12:20:27.537435 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.269296 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-188.ec2.internal" Jul 15 12:20:27.537435 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.270243 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 15 12:20:27.537435 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.271448 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 15 12:20:27.537435 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.271480 2560 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 15 12:20:27.537435 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.271500 2560 kubelet.go:2448] "Starting kubelet main sync loop" Jul 15 12:20:27.537435 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:26.271528 2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 15 12:20:27.537435 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.273982 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 12:20:27.537435 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:26.278171 2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-188.ec2.internal" Jul 15 12:20:27.537435 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:26.278190 2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-188.ec2.internal\": node \"ip-10-0-139-188.ec2.internal\" not found" Jul 15 12:20:27.537435 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:27.463774 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-188.ec2.internal\" not found" Jul 15 12:20:27.537435 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:27.503284 2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-188.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-188.ec2.internal"] Jul 15 12:20:27.537435 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:27.503382 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 12:20:27.537435 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:27.505152 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-188.ec2.internal" event="NodeHasSufficientMemory" Jul 15 12:20:27.538388 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:27.505176 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 12:20:27.538388 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:27.505185 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-188.ec2.internal" event="NodeHasSufficientPID" Jul 15 12:20:27.538388 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:27.506517 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 12:20:27.538388 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:27.506661 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-188.ec2.internal" Jul 15 12:20:27.538388 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:27.506694 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 12:20:27.538388 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:27.507232 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-188.ec2.internal" event="NodeHasSufficientMemory" Jul 15 12:20:27.538388 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:27.507255 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 12:20:27.538388 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:27.507269 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-188.ec2.internal" event="NodeHasSufficientPID" Jul 15 12:20:27.538388 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:27.507282 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-188.ec2.internal" event="NodeHasSufficientMemory" Jul 15 12:20:27.538388 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:27.507299 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 12:20:27.538388 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:27.507308 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-188.ec2.internal" event="NodeHasSufficientPID" Jul 15 12:20:27.538388 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:27.508510 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-188.ec2.internal" Jul 15 12:20:27.538388 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:27.508529 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 12:20:27.538388 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:27.509138 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-188.ec2.internal" event="NodeHasSufficientMemory" Jul 15 12:20:27.538388 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:27.509166 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 12:20:27.538388 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:27.509182 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-188.ec2.internal" event="NodeHasSufficientPID" Jul 15 12:20:27.538388 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:27.535410 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2a2da66c918180b41f7f0e19a0be1a68-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-188.ec2.internal\" (UID: \"2a2da66c918180b41f7f0e19a0be1a68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-188.ec2.internal" Jul 15 12:20:27.538388 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:27.535430 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2a2da66c918180b41f7f0e19a0be1a68-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-188.ec2.internal\" (UID: \"2a2da66c918180b41f7f0e19a0be1a68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-188.ec2.internal" Jul 15 12:20:27.538388 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:27.535447 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a30df3ddb10f29f1462b23c270b4893e-config\") pod \"kube-apiserver-proxy-ip-10-0-139-188.ec2.internal\" (UID: \"a30df3ddb10f29f1462b23c270b4893e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-188.ec2.internal" Jul 15 12:20:27.640586 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:27.563263 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-188.ec2.internal\" not found" node="ip-10-0-139-188.ec2.internal" Jul 15 12:20:27.640586 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:27.563886 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-188.ec2.internal\" not found" Jul 15 12:20:27.640586 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:27.567745 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-188.ec2.internal\" not found" node="ip-10-0-139-188.ec2.internal" Jul 15 12:20:27.640586 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:27.636288 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2a2da66c918180b41f7f0e19a0be1a68-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-188.ec2.internal\" (UID: \"2a2da66c918180b41f7f0e19a0be1a68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-188.ec2.internal" Jul 15 12:20:27.640586 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:27.636313 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2a2da66c918180b41f7f0e19a0be1a68-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-188.ec2.internal\" (UID: \"2a2da66c918180b41f7f0e19a0be1a68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-188.ec2.internal" Jul 15 12:20:27.640586 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:27.636331 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a30df3ddb10f29f1462b23c270b4893e-config\") pod \"kube-apiserver-proxy-ip-10-0-139-188.ec2.internal\" (UID: \"a30df3ddb10f29f1462b23c270b4893e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-188.ec2.internal" Jul 15 12:20:27.640586 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:27.636382 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a30df3ddb10f29f1462b23c270b4893e-config\") pod \"kube-apiserver-proxy-ip-10-0-139-188.ec2.internal\" (UID: \"a30df3ddb10f29f1462b23c270b4893e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-188.ec2.internal" Jul 15 12:20:27.640586 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:27.636404 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2a2da66c918180b41f7f0e19a0be1a68-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-188.ec2.internal\" (UID: \"2a2da66c918180b41f7f0e19a0be1a68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-188.ec2.internal" Jul 15 12:20:27.640586 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:27.636382 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2a2da66c918180b41f7f0e19a0be1a68-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-188.ec2.internal\" (UID: \"2a2da66c918180b41f7f0e19a0be1a68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-188.ec2.internal" Jul 15 12:20:27.748686 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:27.664461 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-188.ec2.internal\" not found" Jul 15 12:20:27.765970 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:27.765131 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-188.ec2.internal\" not found" Jul 15 12:20:27.876283 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:27.865668 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-188.ec2.internal" Jul 15 12:20:27.876283 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:27.865669 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-188.ec2.internal\" not found" Jul 15 12:20:27.876283 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:27.870383 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-188.ec2.internal" Jul 15 12:20:27.974183 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:27.966820 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-188.ec2.internal\" not found" Jul 15 12:20:28.088537 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:28.067269 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-188.ec2.internal\" not found" Jul 15 12:20:28.200478 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:28.160693 2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 15 12:20:28.200478 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:28.160847 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 15 12:20:28.200478 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:28.167884 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-188.ec2.internal\" not found" Jul 15 12:20:28.299463 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:28.226781 2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 12:20:28.299463 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:28.254975 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 12:15:27 +0000 UTC" deadline="2028-02-24 21:10:48.955196169 +0000 UTC" Jul 15 12:20:28.299463 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:28.255004 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14144h50m20.700195043s" Jul 15 12:20:28.299463 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:28.268145 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-188.ec2.internal\" not found" Jul 15 12:20:28.299463 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:28.274935 2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 12:20:28.368720 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:28.368687 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-188.ec2.internal\" not found" Jul 15 12:20:28.469633 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:28.469604 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-188.ec2.internal\" not found" Jul 15 12:20:28.570100 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:28.570056 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-188.ec2.internal\" not found" Jul 15 12:20:28.588881 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:28.588853 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda30df3ddb10f29f1462b23c270b4893e.slice/crio-3360628a0c9c1a3ac5fc9e52b882d4b1327faf289f8dea57697673ae38f44c07 WatchSource:0}: Error finding container 3360628a0c9c1a3ac5fc9e52b882d4b1327faf289f8dea57697673ae38f44c07: Status 404 returned error can't find the container with id 3360628a0c9c1a3ac5fc9e52b882d4b1327faf289f8dea57697673ae38f44c07 Jul 15 12:20:28.590473 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:28.590380 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a2da66c918180b41f7f0e19a0be1a68.slice/crio-5d6912fc13432b50daa9ac4c1dd165ceb337a8d59f54c178cf2d173991665c6b WatchSource:0}: Error finding container 5d6912fc13432b50daa9ac4c1dd165ceb337a8d59f54c178cf2d173991665c6b: Status 404 returned error can't find the container with id 5d6912fc13432b50daa9ac4c1dd165ceb337a8d59f54c178cf2d173991665c6b Jul 15 12:20:28.594917 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:28.594901 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 12:20:28.608993 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:28.608975 2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 12:20:28.627897 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:28.627876 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-188.ec2.internal" Jul 15 12:20:28.640147 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:28.640127 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 12:20:28.641194 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:28.641181 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-188.ec2.internal" Jul 15 12:20:28.648971 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:28.648958 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 12:20:29.203791 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.203758 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 12:20:29.203959 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.203935 2560 apiserver.go:52] "Watching apiserver" Jul 15 12:20:29.216235 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.216205 2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 15 12:20:29.217327 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.217298 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-h7bxb","openshift-network-operator/iptables-alerter-9c5zn","kube-system/global-pull-secret-syncer-79bqk","kube-system/kube-apiserver-proxy-ip-10-0-139-188.ec2.internal","openshift-dns/node-resolver-6xkhw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-188.ec2.internal","openshift-network-diagnostics/network-check-target-c94x9","openshift-ovn-kubernetes/ovnkube-node-bg9fm","kube-system/konnectivity-agent-rw5hk","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zvlp","openshift-cluster-node-tuning-operator/tuned-whxjk","openshift-image-registry/node-ca-m7tbh","openshift-multus/multus-additional-cni-plugins-694pn","openshift-multus/multus-lcf4n"] Jul 15 12:20:29.220219 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.220199 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zvlp" Jul 15 12:20:29.223677 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.223089 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-694pn" Jul 15 12:20:29.224433 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.224014 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 15 12:20:29.224433 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.224036 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 15 12:20:29.224433 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.224276 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 15 12:20:29.227863 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.226807 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7sqrn\"" Jul 15 12:20:29.227863 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.226998 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 15 12:20:29.227863 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.227100 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 15 12:20:29.227863 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.227240 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 15 12:20:29.227863 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.227284 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 15 12:20:29.227863 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.227448 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 15 12:20:29.227863 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.227507 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-6llfh\"" Jul 15 12:20:29.227863 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.227624 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:20:29.227863 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.227701 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.227863 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:29.227739 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79bqk" podUID="a919b128-8dca-4cee-81a2-51a65300820c" Jul 15 12:20:29.227863 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.227749 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 15 12:20:29.230291 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.229505 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6xkhw" Jul 15 12:20:29.230291 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.229831 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-856tb\"" Jul 15 12:20:29.230291 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.230076 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 15 12:20:29.232613 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.232155 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.232613 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.232219 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wrdf9\"" Jul 15 12:20:29.232613 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.232231 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 15 12:20:29.232613 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.232159 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 15 12:20:29.234942 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.234440 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-b2zh5\"" Jul 15 12:20:29.234942 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.234599 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rw5hk" Jul 15 12:20:29.234942 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.234633 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 15 12:20:29.234942 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.234632 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 15 12:20:29.234942 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.234848 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 15 12:20:29.235611 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.235596 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 15 12:20:29.235611 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.235601 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 15 12:20:29.235764 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.235626 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 15 12:20:29.237290 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.237217 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:20:29.237376 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:29.237305 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c94x9" podUID="b8db43c0-4eee-4627-b30d-53206ad7666a" Jul 15 12:20:29.238361 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.238343 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 15 12:20:29.238520 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.238346 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 15 12:20:29.238520 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.238512 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4cm72\"" Jul 15 12:20:29.239353 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.239248 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9c5zn" Jul 15 12:20:29.241993 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.241977 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:20:29.242082 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:29.242038 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h7bxb" podUID="332b1403-fae5-4270-b17f-5b7597d59006" Jul 15 12:20:29.243318 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.243274 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvzf4\" (UniqueName: \"kubernetes.io/projected/de344d40-415e-49c1-b0eb-67c0292a59fa-kube-api-access-rvzf4\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.243318 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.243307 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/610463f4-1622-4297-9bd5-cf12fc02ed55-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5zvlp\" (UID: \"610463f4-1622-4297-9bd5-cf12fc02ed55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zvlp" Jul 15 12:20:29.243471 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.243333 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrkqn\" (UniqueName: \"kubernetes.io/projected/610463f4-1622-4297-9bd5-cf12fc02ed55-kube-api-access-lrkqn\") pod \"aws-ebs-csi-driver-node-5zvlp\" (UID: \"610463f4-1622-4297-9bd5-cf12fc02ed55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zvlp" Jul 15 12:20:29.243471 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.243358 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7028a878-0c31-4518-ac5b-b63dc858a010-cni-binary-copy\") pod \"multus-additional-cni-plugins-694pn\" (UID: \"7028a878-0c31-4518-ac5b-b63dc858a010\") " pod="openshift-multus/multus-additional-cni-plugins-694pn" Jul 15 12:20:29.243570 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.243479 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/610463f4-1622-4297-9bd5-cf12fc02ed55-sys-fs\") pod \"aws-ebs-csi-driver-node-5zvlp\" (UID: \"610463f4-1622-4297-9bd5-cf12fc02ed55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zvlp" Jul 15 12:20:29.243570 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.243502 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7028a878-0c31-4518-ac5b-b63dc858a010-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-694pn\" (UID: \"7028a878-0c31-4518-ac5b-b63dc858a010\") " pod="openshift-multus/multus-additional-cni-plugins-694pn" Jul 15 12:20:29.243570 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.243531 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a919b128-8dca-4cee-81a2-51a65300820c-original-pull-secret\") pod \"global-pull-secret-syncer-79bqk\" (UID: \"a919b128-8dca-4cee-81a2-51a65300820c\") " pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:20:29.243744 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.243573 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-system-cni-dir\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.243744 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.243611 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-host-run-k8s-cni-cncf-io\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.243744 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.243658 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-etc-kubernetes\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.243744 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.243686 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/74f12cb2-ed42-4446-bb5d-25cb7a992fa3-hosts-file\") pod \"node-resolver-6xkhw\" (UID: \"74f12cb2-ed42-4446-bb5d-25cb7a992fa3\") " pod="openshift-dns/node-resolver-6xkhw" Jul 15 12:20:29.243744 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.243713 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 15 12:20:29.243974 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.243712 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7028a878-0c31-4518-ac5b-b63dc858a010-tuning-conf-dir\") pod \"multus-additional-cni-plugins-694pn\" (UID: \"7028a878-0c31-4518-ac5b-b63dc858a010\") " pod="openshift-multus/multus-additional-cni-plugins-694pn" Jul 15 12:20:29.243974 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.243819 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 15 12:20:29.243974 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.243849 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7028a878-0c31-4518-ac5b-b63dc858a010-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-694pn\" (UID: \"7028a878-0c31-4518-ac5b-b63dc858a010\") " pod="openshift-multus/multus-additional-cni-plugins-694pn" Jul 15 12:20:29.243974 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.243911 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-hostroot\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.243974 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.243938 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltbql\" (UniqueName: \"kubernetes.io/projected/74f12cb2-ed42-4446-bb5d-25cb7a992fa3-kube-api-access-ltbql\") pod \"node-resolver-6xkhw\" (UID: \"74f12cb2-ed42-4446-bb5d-25cb7a992fa3\") " pod="openshift-dns/node-resolver-6xkhw" Jul 15 12:20:29.243974 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.243966 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/610463f4-1622-4297-9bd5-cf12fc02ed55-etc-selinux\") pod \"aws-ebs-csi-driver-node-5zvlp\" (UID: \"610463f4-1622-4297-9bd5-cf12fc02ed55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zvlp" Jul 15 12:20:29.244258 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.243989 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7028a878-0c31-4518-ac5b-b63dc858a010-system-cni-dir\") pod \"multus-additional-cni-plugins-694pn\" (UID: \"7028a878-0c31-4518-ac5b-b63dc858a010\") " pod="openshift-multus/multus-additional-cni-plugins-694pn" Jul 15 12:20:29.244258 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.244014 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7028a878-0c31-4518-ac5b-b63dc858a010-os-release\") pod \"multus-additional-cni-plugins-694pn\" (UID: \"7028a878-0c31-4518-ac5b-b63dc858a010\") " pod="openshift-multus/multus-additional-cni-plugins-694pn" Jul 15 12:20:29.244258 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.244021 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-smkjm\"" Jul 15 12:20:29.244258 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.244036 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/610463f4-1622-4297-9bd5-cf12fc02ed55-socket-dir\") pod \"aws-ebs-csi-driver-node-5zvlp\" (UID: \"610463f4-1622-4297-9bd5-cf12fc02ed55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zvlp" Jul 15 12:20:29.244258 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.244060 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/610463f4-1622-4297-9bd5-cf12fc02ed55-device-dir\") pod \"aws-ebs-csi-driver-node-5zvlp\" (UID: \"610463f4-1622-4297-9bd5-cf12fc02ed55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zvlp" Jul 15 12:20:29.244258 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.244086 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-os-release\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.244258 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.244128 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-host-var-lib-kubelet\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.244258 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.244155 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzqgh\" (UniqueName: \"kubernetes.io/projected/7028a878-0c31-4518-ac5b-b63dc858a010-kube-api-access-fzqgh\") pod \"multus-additional-cni-plugins-694pn\" (UID: \"7028a878-0c31-4518-ac5b-b63dc858a010\") " pod="openshift-multus/multus-additional-cni-plugins-694pn" Jul 15 12:20:29.244258 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.244169 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 15 12:20:29.244258 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.244186 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-host-run-netns\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.244258 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.244245 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74f12cb2-ed42-4446-bb5d-25cb7a992fa3-tmp-dir\") pod \"node-resolver-6xkhw\" (UID: \"74f12cb2-ed42-4446-bb5d-25cb7a992fa3\") " pod="openshift-dns/node-resolver-6xkhw" Jul 15 12:20:29.244809 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.244282 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a919b128-8dca-4cee-81a2-51a65300820c-dbus\") pod \"global-pull-secret-syncer-79bqk\" (UID: \"a919b128-8dca-4cee-81a2-51a65300820c\") " pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:20:29.244809 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.244352 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-host-var-lib-cni-bin\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.244809 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.244389 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-multus-conf-dir\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.244809 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.244426 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/610463f4-1622-4297-9bd5-cf12fc02ed55-registration-dir\") pod \"aws-ebs-csi-driver-node-5zvlp\" (UID: \"610463f4-1622-4297-9bd5-cf12fc02ed55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zvlp" Jul 15 12:20:29.244809 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.244470 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7028a878-0c31-4518-ac5b-b63dc858a010-cnibin\") pod \"multus-additional-cni-plugins-694pn\" (UID: \"7028a878-0c31-4518-ac5b-b63dc858a010\") " pod="openshift-multus/multus-additional-cni-plugins-694pn" Jul 15 12:20:29.244809 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.244521 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a919b128-8dca-4cee-81a2-51a65300820c-kubelet-config\") pod \"global-pull-secret-syncer-79bqk\" (UID: \"a919b128-8dca-4cee-81a2-51a65300820c\") " pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:20:29.244809 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.244557 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/de344d40-415e-49c1-b0eb-67c0292a59fa-cni-binary-copy\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.244809 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.244580 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-host-var-lib-cni-multus\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.244809 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.244602 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/de344d40-415e-49c1-b0eb-67c0292a59fa-multus-daemon-config\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.244809 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.244631 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-multus-cni-dir\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.244809 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.244697 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-cnibin\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.244809 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.244719 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-multus-socket-dir-parent\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.244809 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.244742 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-host-run-multus-certs\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.246226 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.246164 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.251657 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.249136 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hldt9\"" Jul 15 12:20:29.251657 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.249451 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 15 12:20:29.251657 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.249502 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-m7tbh" Jul 15 12:20:29.251657 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.249507 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 15 12:20:29.251657 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.250569 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 12:20:29.255034 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.255013 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 15 12:20:29.255443 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.255310 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 15 12:20:29.255528 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.255474 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 15 12:20:29.255592 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.255565 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-82g49\"" Jul 15 12:20:29.271881 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.271861 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-pvffz" Jul 15 12:20:29.279653 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.279622 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pvffz" Jul 15 12:20:29.328653 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.328611 2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 15 12:20:29.345554 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.345497 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvzf4\" (UniqueName: \"kubernetes.io/projected/de344d40-415e-49c1-b0eb-67c0292a59fa-kube-api-access-rvzf4\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.345554 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.345539 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/610463f4-1622-4297-9bd5-cf12fc02ed55-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5zvlp\" (UID: \"610463f4-1622-4297-9bd5-cf12fc02ed55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zvlp" Jul 15 12:20:29.345837 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.345570 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-etc-openvswitch\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.345837 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.345593 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-log-socket\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.345837 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.345616 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/610463f4-1622-4297-9bd5-cf12fc02ed55-sys-fs\") pod \"aws-ebs-csi-driver-node-5zvlp\" (UID: \"610463f4-1622-4297-9bd5-cf12fc02ed55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zvlp" Jul 15 12:20:29.345837 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.345654 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-host-run-ovn-kubernetes\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.345837 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.345679 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-host-cni-netd\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.345837 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.345713 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/62753f0f-8783-4ec9-b5f2-1e7663da727e-var-lib-kubelet\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.345837 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.345739 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-system-cni-dir\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.345837 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.345761 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-etc-kubernetes\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.345837 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.345789 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7028a878-0c31-4518-ac5b-b63dc858a010-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-694pn\" (UID: \"7028a878-0c31-4518-ac5b-b63dc858a010\") " pod="openshift-multus/multus-additional-cni-plugins-694pn" Jul 15 12:20:29.345837 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.345814 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/37807770-29e4-460a-bea4-efc59d333d9a-iptables-alerter-script\") pod \"iptables-alerter-9c5zn\" (UID: \"37807770-29e4-460a-bea4-efc59d333d9a\") " pod="openshift-network-operator/iptables-alerter-9c5zn" Jul 15 12:20:29.345837 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.345835 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/62753f0f-8783-4ec9-b5f2-1e7663da727e-sys\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.345837 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.345831 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/610463f4-1622-4297-9bd5-cf12fc02ed55-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5zvlp\" (UID: \"610463f4-1622-4297-9bd5-cf12fc02ed55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zvlp" Jul 15 12:20:29.346506 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.345872 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-hostroot\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.346506 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.345903 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-etc-kubernetes\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.346506 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.345951 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/610463f4-1622-4297-9bd5-cf12fc02ed55-sys-fs\") pod \"aws-ebs-csi-driver-node-5zvlp\" (UID: \"610463f4-1622-4297-9bd5-cf12fc02ed55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zvlp" Jul 15 12:20:29.346506 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.345896 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/610463f4-1622-4297-9bd5-cf12fc02ed55-etc-selinux\") pod \"aws-ebs-csi-driver-node-5zvlp\" (UID: \"610463f4-1622-4297-9bd5-cf12fc02ed55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zvlp" Jul 15 12:20:29.346506 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.346004 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/610463f4-1622-4297-9bd5-cf12fc02ed55-etc-selinux\") pod \"aws-ebs-csi-driver-node-5zvlp\" (UID: \"610463f4-1622-4297-9bd5-cf12fc02ed55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zvlp" Jul 15 12:20:29.346506 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.346054 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-hostroot\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.346506 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.346037 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7028a878-0c31-4518-ac5b-b63dc858a010-os-release\") pod \"multus-additional-cni-plugins-694pn\" (UID: \"7028a878-0c31-4518-ac5b-b63dc858a010\") " pod="openshift-multus/multus-additional-cni-plugins-694pn" Jul 15 12:20:29.346506 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.346125 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/62753f0f-8783-4ec9-b5f2-1e7663da727e-etc-systemd\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.346506 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.346138 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-system-cni-dir\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.346506 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.346152 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-var-lib-openvswitch\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.346506 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.346164 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7028a878-0c31-4518-ac5b-b63dc858a010-os-release\") pod \"multus-additional-cni-plugins-694pn\" (UID: \"7028a878-0c31-4518-ac5b-b63dc858a010\") " pod="openshift-multus/multus-additional-cni-plugins-694pn" Jul 15 12:20:29.346506 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.346178 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.346506 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.346204 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/62753f0f-8783-4ec9-b5f2-1e7663da727e-lib-modules\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.346506 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.346229 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/610463f4-1622-4297-9bd5-cf12fc02ed55-device-dir\") pod \"aws-ebs-csi-driver-node-5zvlp\" (UID: \"610463f4-1622-4297-9bd5-cf12fc02ed55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zvlp" Jul 15 12:20:29.347368 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.346254 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-os-release\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.347368 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.346302 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/610463f4-1622-4297-9bd5-cf12fc02ed55-device-dir\") pod \"aws-ebs-csi-driver-node-5zvlp\" (UID: \"610463f4-1622-4297-9bd5-cf12fc02ed55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zvlp" Jul 15 12:20:29.347368 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.346327 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzqgh\" (UniqueName: \"kubernetes.io/projected/7028a878-0c31-4518-ac5b-b63dc858a010-kube-api-access-fzqgh\") pod \"multus-additional-cni-plugins-694pn\" (UID: \"7028a878-0c31-4518-ac5b-b63dc858a010\") " pod="openshift-multus/multus-additional-cni-plugins-694pn" Jul 15 12:20:29.347368 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.346391 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-os-release\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.347368 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.346433 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/37807770-29e4-460a-bea4-efc59d333d9a-host-slash\") pod \"iptables-alerter-9c5zn\" (UID: \"37807770-29e4-460a-bea4-efc59d333d9a\") " pod="openshift-network-operator/iptables-alerter-9c5zn" Jul 15 12:20:29.347368 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.346472 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hz8n2\" (UniqueName: \"kubernetes.io/projected/37807770-29e4-460a-bea4-efc59d333d9a-kube-api-access-hz8n2\") pod \"iptables-alerter-9c5zn\" (UID: \"37807770-29e4-460a-bea4-efc59d333d9a\") " pod="openshift-network-operator/iptables-alerter-9c5zn" Jul 15 12:20:29.347368 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.346500 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7f4w\" (UniqueName: \"kubernetes.io/projected/808669ad-89bb-473a-86e9-7562fcd99736-kube-api-access-p7f4w\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.347368 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.346572 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-host-run-netns\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.347368 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.346569 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7028a878-0c31-4518-ac5b-b63dc858a010-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-694pn\" (UID: \"7028a878-0c31-4518-ac5b-b63dc858a010\") " pod="openshift-multus/multus-additional-cni-plugins-694pn" Jul 15 12:20:29.347368 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.346656 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-host-run-netns\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.347368 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.346610 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74f12cb2-ed42-4446-bb5d-25cb7a992fa3-tmp-dir\") pod \"node-resolver-6xkhw\" (UID: \"74f12cb2-ed42-4446-bb5d-25cb7a992fa3\") " pod="openshift-dns/node-resolver-6xkhw" Jul 15 12:20:29.347368 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.346702 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a919b128-8dca-4cee-81a2-51a65300820c-dbus\") pod \"global-pull-secret-syncer-79bqk\" (UID: \"a919b128-8dca-4cee-81a2-51a65300820c\") " pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:20:29.347368 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.346727 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-host-run-netns\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.347368 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.346757 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-host-cni-bin\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.348030 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.346781 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7c84d9e6-e6f4-4bc0-a95e-707e501a2d80-agent-certs\") pod \"konnectivity-agent-rw5hk\" (UID: \"7c84d9e6-e6f4-4bc0-a95e-707e501a2d80\") " pod="kube-system/konnectivity-agent-rw5hk" Jul 15 12:20:29.348030 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.346804 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vprwx\" (UniqueName: \"kubernetes.io/projected/b8db43c0-4eee-4627-b30d-53206ad7666a-kube-api-access-vprwx\") pod \"network-check-target-c94x9\" (UID: \"b8db43c0-4eee-4627-b30d-53206ad7666a\") " pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:20:29.348030 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.346847 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/de344d40-415e-49c1-b0eb-67c0292a59fa-cni-binary-copy\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.348030 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.346873 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/de344d40-415e-49c1-b0eb-67c0292a59fa-multus-daemon-config\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.348030 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.346923 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-host-slash\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.348030 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.346948 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-multus-cni-dir\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.348030 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.346968 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74f12cb2-ed42-4446-bb5d-25cb7a992fa3-tmp-dir\") pod \"node-resolver-6xkhw\" (UID: \"74f12cb2-ed42-4446-bb5d-25cb7a992fa3\") " pod="openshift-dns/node-resolver-6xkhw" Jul 15 12:20:29.348030 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.346972 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-multus-socket-dir-parent\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.348030 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.347037 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-host-run-multus-certs\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.348030 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.347066 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrkqn\" (UniqueName: \"kubernetes.io/projected/610463f4-1622-4297-9bd5-cf12fc02ed55-kube-api-access-lrkqn\") pod \"aws-ebs-csi-driver-node-5zvlp\" (UID: \"610463f4-1622-4297-9bd5-cf12fc02ed55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zvlp" Jul 15 12:20:29.348030 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.347091 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7028a878-0c31-4518-ac5b-b63dc858a010-cni-binary-copy\") pod \"multus-additional-cni-plugins-694pn\" (UID: \"7028a878-0c31-4518-ac5b-b63dc858a010\") " pod="openshift-multus/multus-additional-cni-plugins-694pn" Jul 15 12:20:29.348030 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.347094 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-multus-socket-dir-parent\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.348030 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.347145 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-host-run-multus-certs\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.348030 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.347191 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7028a878-0c31-4518-ac5b-b63dc858a010-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-694pn\" (UID: \"7028a878-0c31-4518-ac5b-b63dc858a010\") " pod="openshift-multus/multus-additional-cni-plugins-694pn" Jul 15 12:20:29.348723 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.347233 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/62753f0f-8783-4ec9-b5f2-1e7663da727e-etc-sysctl-d\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.348723 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.347263 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/62753f0f-8783-4ec9-b5f2-1e7663da727e-host\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.348723 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.347341 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a919b128-8dca-4cee-81a2-51a65300820c-original-pull-secret\") pod \"global-pull-secret-syncer-79bqk\" (UID: \"a919b128-8dca-4cee-81a2-51a65300820c\") " pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:20:29.348723 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.347370 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bvdz\" (UniqueName: \"kubernetes.io/projected/62753f0f-8783-4ec9-b5f2-1e7663da727e-kube-api-access-6bvdz\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.348723 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.347393 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-run-systemd\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.348723 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.347415 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/808669ad-89bb-473a-86e9-7562fcd99736-ovn-node-metrics-cert\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.348723 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.347455 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/05b26d09-b957-4b76-80fd-b30046fbbfc5-host\") pod \"node-ca-m7tbh\" (UID: \"05b26d09-b957-4b76-80fd-b30046fbbfc5\") " pod="openshift-image-registry/node-ca-m7tbh" Jul 15 12:20:29.348723 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.347489 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-host-run-k8s-cni-cncf-io\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.348723 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.347515 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/74f12cb2-ed42-4446-bb5d-25cb7a992fa3-hosts-file\") pod \"node-resolver-6xkhw\" (UID: \"74f12cb2-ed42-4446-bb5d-25cb7a992fa3\") " pod="openshift-dns/node-resolver-6xkhw" Jul 15 12:20:29.348723 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.347538 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7028a878-0c31-4518-ac5b-b63dc858a010-tuning-conf-dir\") pod \"multus-additional-cni-plugins-694pn\" (UID: \"7028a878-0c31-4518-ac5b-b63dc858a010\") " pod="openshift-multus/multus-additional-cni-plugins-694pn" Jul 15 12:20:29.348723 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.347569 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/62753f0f-8783-4ec9-b5f2-1e7663da727e-etc-modprobe-d\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.348723 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.347592 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62753f0f-8783-4ec9-b5f2-1e7663da727e-etc-kubernetes\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.348723 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.347604 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7028a878-0c31-4518-ac5b-b63dc858a010-cni-binary-copy\") pod \"multus-additional-cni-plugins-694pn\" (UID: \"7028a878-0c31-4518-ac5b-b63dc858a010\") " pod="openshift-multus/multus-additional-cni-plugins-694pn" Jul 15 12:20:29.348723 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.347614 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/808669ad-89bb-473a-86e9-7562fcd99736-env-overrides\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.349325 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.347678 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltbql\" (UniqueName: \"kubernetes.io/projected/74f12cb2-ed42-4446-bb5d-25cb7a992fa3-kube-api-access-ltbql\") pod \"node-resolver-6xkhw\" (UID: \"74f12cb2-ed42-4446-bb5d-25cb7a992fa3\") " pod="openshift-dns/node-resolver-6xkhw" Jul 15 12:20:29.349325 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.347711 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7028a878-0c31-4518-ac5b-b63dc858a010-system-cni-dir\") pod \"multus-additional-cni-plugins-694pn\" (UID: \"7028a878-0c31-4518-ac5b-b63dc858a010\") " pod="openshift-multus/multus-additional-cni-plugins-694pn" Jul 15 12:20:29.349325 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.347740 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/62753f0f-8783-4ec9-b5f2-1e7663da727e-etc-sysctl-conf\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.349325 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.347745 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/de344d40-415e-49c1-b0eb-67c0292a59fa-cni-binary-copy\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.349325 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.347768 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/610463f4-1622-4297-9bd5-cf12fc02ed55-socket-dir\") pod \"aws-ebs-csi-driver-node-5zvlp\" (UID: \"610463f4-1622-4297-9bd5-cf12fc02ed55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zvlp" Jul 15 12:20:29.349325 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.347872 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/610463f4-1622-4297-9bd5-cf12fc02ed55-socket-dir\") pod \"aws-ebs-csi-driver-node-5zvlp\" (UID: \"610463f4-1622-4297-9bd5-cf12fc02ed55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zvlp" Jul 15 12:20:29.349325 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.347919 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-multus-cni-dir\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.349325 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:29.348036 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 12:20:29.349325 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348065 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7028a878-0c31-4518-ac5b-b63dc858a010-system-cni-dir\") pod \"multus-additional-cni-plugins-694pn\" (UID: \"7028a878-0c31-4518-ac5b-b63dc858a010\") " pod="openshift-multus/multus-additional-cni-plugins-694pn" Jul 15 12:20:29.349325 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348098 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7028a878-0c31-4518-ac5b-b63dc858a010-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-694pn\" (UID: \"7028a878-0c31-4518-ac5b-b63dc858a010\") " pod="openshift-multus/multus-additional-cni-plugins-694pn" Jul 15 12:20:29.349325 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348104 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-host-var-lib-kubelet\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.349325 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348140 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-host-kubelet\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.349325 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348172 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/808669ad-89bb-473a-86e9-7562fcd99736-ovnkube-config\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.349325 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348178 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-host-run-k8s-cni-cncf-io\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.349325 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348205 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/808669ad-89bb-473a-86e9-7562fcd99736-ovnkube-script-lib\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.350052 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348231 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/332b1403-fae5-4270-b17f-5b7597d59006-metrics-certs\") pod \"network-metrics-daemon-h7bxb\" (UID: \"332b1403-fae5-4270-b17f-5b7597d59006\") " pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:20:29.350052 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348234 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/74f12cb2-ed42-4446-bb5d-25cb7a992fa3-hosts-file\") pod \"node-resolver-6xkhw\" (UID: \"74f12cb2-ed42-4446-bb5d-25cb7a992fa3\") " pod="openshift-dns/node-resolver-6xkhw" Jul 15 12:20:29.350052 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348253 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a919b128-8dca-4cee-81a2-51a65300820c-dbus\") pod \"global-pull-secret-syncer-79bqk\" (UID: \"a919b128-8dca-4cee-81a2-51a65300820c\") " pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:20:29.350052 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:29.348261 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a919b128-8dca-4cee-81a2-51a65300820c-original-pull-secret podName:a919b128-8dca-4cee-81a2-51a65300820c nodeName:}" failed. No retries permitted until 2026-07-15 12:20:29.848229022 +0000 UTC m=+2.989025130 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a919b128-8dca-4cee-81a2-51a65300820c-original-pull-secret") pod "global-pull-secret-syncer-79bqk" (UID: "a919b128-8dca-4cee-81a2-51a65300820c") : object "kube-system"/"original-pull-secret" not registered Jul 15 12:20:29.350052 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348290 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-host-var-lib-kubelet\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.350052 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348340 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/62753f0f-8783-4ec9-b5f2-1e7663da727e-tmp\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.350052 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348364 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/05b26d09-b957-4b76-80fd-b30046fbbfc5-serviceca\") pod \"node-ca-m7tbh\" (UID: \"05b26d09-b957-4b76-80fd-b30046fbbfc5\") " pod="openshift-image-registry/node-ca-m7tbh" Jul 15 12:20:29.350052 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348376 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/de344d40-415e-49c1-b0eb-67c0292a59fa-multus-daemon-config\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.350052 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348387 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58vbb\" (UniqueName: \"kubernetes.io/projected/05b26d09-b957-4b76-80fd-b30046fbbfc5-kube-api-access-58vbb\") pod \"node-ca-m7tbh\" (UID: \"05b26d09-b957-4b76-80fd-b30046fbbfc5\") " pod="openshift-image-registry/node-ca-m7tbh" Jul 15 12:20:29.350052 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348414 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7028a878-0c31-4518-ac5b-b63dc858a010-tuning-conf-dir\") pod \"multus-additional-cni-plugins-694pn\" (UID: \"7028a878-0c31-4518-ac5b-b63dc858a010\") " pod="openshift-multus/multus-additional-cni-plugins-694pn" Jul 15 12:20:29.350052 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348425 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-host-var-lib-cni-bin\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.350052 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348449 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-multus-conf-dir\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.350052 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348474 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/610463f4-1622-4297-9bd5-cf12fc02ed55-registration-dir\") pod \"aws-ebs-csi-driver-node-5zvlp\" (UID: \"610463f4-1622-4297-9bd5-cf12fc02ed55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zvlp" Jul 15 12:20:29.350052 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348493 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-host-var-lib-cni-bin\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.350676 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348504 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7028a878-0c31-4518-ac5b-b63dc858a010-cnibin\") pod \"multus-additional-cni-plugins-694pn\" (UID: \"7028a878-0c31-4518-ac5b-b63dc858a010\") " pod="openshift-multus/multus-additional-cni-plugins-694pn" Jul 15 12:20:29.350676 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348512 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-multus-conf-dir\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.350676 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348527 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a919b128-8dca-4cee-81a2-51a65300820c-kubelet-config\") pod \"global-pull-secret-syncer-79bqk\" (UID: \"a919b128-8dca-4cee-81a2-51a65300820c\") " pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:20:29.350676 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348553 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7028a878-0c31-4518-ac5b-b63dc858a010-cnibin\") pod \"multus-additional-cni-plugins-694pn\" (UID: \"7028a878-0c31-4518-ac5b-b63dc858a010\") " pod="openshift-multus/multus-additional-cni-plugins-694pn" Jul 15 12:20:29.350676 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348560 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/62753f0f-8783-4ec9-b5f2-1e7663da727e-etc-sysconfig\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.350676 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348567 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/610463f4-1622-4297-9bd5-cf12fc02ed55-registration-dir\") pod \"aws-ebs-csi-driver-node-5zvlp\" (UID: \"610463f4-1622-4297-9bd5-cf12fc02ed55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zvlp" Jul 15 12:20:29.350676 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348584 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/62753f0f-8783-4ec9-b5f2-1e7663da727e-run\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.350676 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348601 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a919b128-8dca-4cee-81a2-51a65300820c-kubelet-config\") pod \"global-pull-secret-syncer-79bqk\" (UID: \"a919b128-8dca-4cee-81a2-51a65300820c\") " pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:20:29.350676 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348606 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/62753f0f-8783-4ec9-b5f2-1e7663da727e-etc-tuned\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.350676 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348661 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-host-var-lib-cni-multus\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.350676 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348684 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-systemd-units\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.350676 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348715 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-host-var-lib-cni-multus\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.350676 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348708 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-run-openvswitch\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.350676 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348755 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-run-ovn\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.351426 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348782 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-node-log\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.351426 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348822 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggrvb\" (UniqueName: \"kubernetes.io/projected/332b1403-fae5-4270-b17f-5b7597d59006-kube-api-access-ggrvb\") pod \"network-metrics-daemon-h7bxb\" (UID: \"332b1403-fae5-4270-b17f-5b7597d59006\") " pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:20:29.351426 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348844 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7c84d9e6-e6f4-4bc0-a95e-707e501a2d80-konnectivity-ca\") pod \"konnectivity-agent-rw5hk\" (UID: \"7c84d9e6-e6f4-4bc0-a95e-707e501a2d80\") " pod="kube-system/konnectivity-agent-rw5hk" Jul 15 12:20:29.351426 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348875 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-cnibin\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.351426 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.348968 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/de344d40-415e-49c1-b0eb-67c0292a59fa-cnibin\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.359076 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.359053 2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 15 12:20:29.363195 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.363122 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzqgh\" (UniqueName: \"kubernetes.io/projected/7028a878-0c31-4518-ac5b-b63dc858a010-kube-api-access-fzqgh\") pod \"multus-additional-cni-plugins-694pn\" (UID: \"7028a878-0c31-4518-ac5b-b63dc858a010\") " pod="openshift-multus/multus-additional-cni-plugins-694pn" Jul 15 12:20:29.363498 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.363478 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvzf4\" (UniqueName: \"kubernetes.io/projected/de344d40-415e-49c1-b0eb-67c0292a59fa-kube-api-access-rvzf4\") pod \"multus-lcf4n\" (UID: \"de344d40-415e-49c1-b0eb-67c0292a59fa\") " pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.364946 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.364923 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrkqn\" (UniqueName: \"kubernetes.io/projected/610463f4-1622-4297-9bd5-cf12fc02ed55-kube-api-access-lrkqn\") pod \"aws-ebs-csi-driver-node-5zvlp\" (UID: \"610463f4-1622-4297-9bd5-cf12fc02ed55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zvlp" Jul 15 12:20:29.365468 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.365448 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltbql\" (UniqueName: \"kubernetes.io/projected/74f12cb2-ed42-4446-bb5d-25cb7a992fa3-kube-api-access-ltbql\") pod \"node-resolver-6xkhw\" (UID: \"74f12cb2-ed42-4446-bb5d-25cb7a992fa3\") " pod="openshift-dns/node-resolver-6xkhw" Jul 15 12:20:29.408029 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.407963 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-188.ec2.internal" event={"ID":"2a2da66c918180b41f7f0e19a0be1a68","Type":"ContainerStarted","Data":"5d6912fc13432b50daa9ac4c1dd165ceb337a8d59f54c178cf2d173991665c6b"} Jul 15 12:20:29.409699 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.409668 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-188.ec2.internal" event={"ID":"a30df3ddb10f29f1462b23c270b4893e","Type":"ContainerStarted","Data":"3360628a0c9c1a3ac5fc9e52b882d4b1327faf289f8dea57697673ae38f44c07"} Jul 15 12:20:29.449880 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.449843 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-host-run-netns\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.449880 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.449886 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-host-cni-bin\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.450103 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.449941 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-host-cni-bin\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.450103 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.449952 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-host-run-netns\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.450103 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.450003 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7c84d9e6-e6f4-4bc0-a95e-707e501a2d80-agent-certs\") pod \"konnectivity-agent-rw5hk\" (UID: \"7c84d9e6-e6f4-4bc0-a95e-707e501a2d80\") " pod="kube-system/konnectivity-agent-rw5hk" Jul 15 12:20:29.450103 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.450034 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vprwx\" (UniqueName: \"kubernetes.io/projected/b8db43c0-4eee-4627-b30d-53206ad7666a-kube-api-access-vprwx\") pod \"network-check-target-c94x9\" (UID: \"b8db43c0-4eee-4627-b30d-53206ad7666a\") " pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:20:29.450103 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.450066 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-host-slash\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.450103 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.450102 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/62753f0f-8783-4ec9-b5f2-1e7663da727e-etc-sysctl-d\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.450397 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.450125 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/62753f0f-8783-4ec9-b5f2-1e7663da727e-host\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.450397 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.450163 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bvdz\" (UniqueName: \"kubernetes.io/projected/62753f0f-8783-4ec9-b5f2-1e7663da727e-kube-api-access-6bvdz\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.450397 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.450174 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-host-slash\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.450397 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.450186 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-run-systemd\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.450397 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.450220 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-run-systemd\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.450397 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.450239 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/808669ad-89bb-473a-86e9-7562fcd99736-ovn-node-metrics-cert\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.450397 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.450265 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/05b26d09-b957-4b76-80fd-b30046fbbfc5-host\") pod \"node-ca-m7tbh\" (UID: \"05b26d09-b957-4b76-80fd-b30046fbbfc5\") " pod="openshift-image-registry/node-ca-m7tbh" Jul 15 12:20:29.450397 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.450293 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/62753f0f-8783-4ec9-b5f2-1e7663da727e-etc-modprobe-d\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.450397 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.450317 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62753f0f-8783-4ec9-b5f2-1e7663da727e-etc-kubernetes\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.450397 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.450339 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/808669ad-89bb-473a-86e9-7562fcd99736-env-overrides\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.450397 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.450353 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/62753f0f-8783-4ec9-b5f2-1e7663da727e-host\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.450397 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.450359 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/62753f0f-8783-4ec9-b5f2-1e7663da727e-etc-sysctl-d\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.450397 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.450362 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/62753f0f-8783-4ec9-b5f2-1e7663da727e-etc-sysctl-conf\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.451031 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.450404 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-host-kubelet\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.451031 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.450436 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/808669ad-89bb-473a-86e9-7562fcd99736-ovnkube-config\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.451031 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.450460 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/808669ad-89bb-473a-86e9-7562fcd99736-ovnkube-script-lib\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.451031 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.450484 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/332b1403-fae5-4270-b17f-5b7597d59006-metrics-certs\") pod \"network-metrics-daemon-h7bxb\" (UID: \"332b1403-fae5-4270-b17f-5b7597d59006\") " pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:20:29.451031 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.450491 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/62753f0f-8783-4ec9-b5f2-1e7663da727e-etc-sysctl-conf\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.451031 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.450507 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/62753f0f-8783-4ec9-b5f2-1e7663da727e-tmp\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.451031 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.450535 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/05b26d09-b957-4b76-80fd-b30046fbbfc5-serviceca\") pod \"node-ca-m7tbh\" (UID: \"05b26d09-b957-4b76-80fd-b30046fbbfc5\") " pod="openshift-image-registry/node-ca-m7tbh" Jul 15 12:20:29.451031 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.450559 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58vbb\" (UniqueName: \"kubernetes.io/projected/05b26d09-b957-4b76-80fd-b30046fbbfc5-kube-api-access-58vbb\") pod \"node-ca-m7tbh\" (UID: \"05b26d09-b957-4b76-80fd-b30046fbbfc5\") " pod="openshift-image-registry/node-ca-m7tbh" Jul 15 12:20:29.451031 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.450587 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/62753f0f-8783-4ec9-b5f2-1e7663da727e-etc-sysconfig\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.451031 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.450628 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/62753f0f-8783-4ec9-b5f2-1e7663da727e-run\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.451031 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.450662 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/62753f0f-8783-4ec9-b5f2-1e7663da727e-etc-sysconfig\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.451031 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.450673 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/62753f0f-8783-4ec9-b5f2-1e7663da727e-etc-tuned\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.451031 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:29.450720 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 12:20:29.451031 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:29.450774 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/332b1403-fae5-4270-b17f-5b7597d59006-metrics-certs podName:332b1403-fae5-4270-b17f-5b7597d59006 nodeName:}" failed. No retries permitted until 2026-07-15 12:20:29.950755391 +0000 UTC m=+3.091551480 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/332b1403-fae5-4270-b17f-5b7597d59006-metrics-certs") pod "network-metrics-daemon-h7bxb" (UID: "332b1403-fae5-4270-b17f-5b7597d59006") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 12:20:29.451765 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.450816 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-host-kubelet\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.451765 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.450957 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/62753f0f-8783-4ec9-b5f2-1e7663da727e-etc-modprobe-d\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.451765 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.451001 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/05b26d09-b957-4b76-80fd-b30046fbbfc5-host\") pod \"node-ca-m7tbh\" (UID: \"05b26d09-b957-4b76-80fd-b30046fbbfc5\") " pod="openshift-image-registry/node-ca-m7tbh" Jul 15 12:20:29.451765 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.451030 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-systemd-units\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.451765 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.451073 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-run-openvswitch\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.451765 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.451098 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-run-ovn\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.451765 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.451124 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-node-log\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.451765 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.451150 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggrvb\" (UniqueName: \"kubernetes.io/projected/332b1403-fae5-4270-b17f-5b7597d59006-kube-api-access-ggrvb\") pod \"network-metrics-daemon-h7bxb\" (UID: \"332b1403-fae5-4270-b17f-5b7597d59006\") " pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:20:29.451765 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.451175 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7c84d9e6-e6f4-4bc0-a95e-707e501a2d80-konnectivity-ca\") pod \"konnectivity-agent-rw5hk\" (UID: \"7c84d9e6-e6f4-4bc0-a95e-707e501a2d80\") " pod="kube-system/konnectivity-agent-rw5hk" Jul 15 12:20:29.451765 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.451200 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-etc-openvswitch\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.451765 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.451223 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-log-socket\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.451765 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.451237 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-run-ovn\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.451765 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.451248 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-host-run-ovn-kubernetes\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.451765 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.451282 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-host-cni-netd\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.451765 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.451294 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/05b26d09-b957-4b76-80fd-b30046fbbfc5-serviceca\") pod \"node-ca-m7tbh\" (UID: \"05b26d09-b957-4b76-80fd-b30046fbbfc5\") " pod="openshift-image-registry/node-ca-m7tbh" Jul 15 12:20:29.452436 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.451311 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/62753f0f-8783-4ec9-b5f2-1e7663da727e-var-lib-kubelet\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.452436 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.451350 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/37807770-29e4-460a-bea4-efc59d333d9a-iptables-alerter-script\") pod \"iptables-alerter-9c5zn\" (UID: \"37807770-29e4-460a-bea4-efc59d333d9a\") " pod="openshift-network-operator/iptables-alerter-9c5zn" Jul 15 12:20:29.452436 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.451357 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/62753f0f-8783-4ec9-b5f2-1e7663da727e-var-lib-kubelet\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.452436 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.451398 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-systemd-units\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.452436 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.451431 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-run-openvswitch\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.452436 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.451466 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-host-cni-netd\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.452436 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.451511 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62753f0f-8783-4ec9-b5f2-1e7663da727e-etc-kubernetes\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.452436 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.451287 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-host-run-ovn-kubernetes\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.452436 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.451764 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-node-log\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.452436 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.451841 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7c84d9e6-e6f4-4bc0-a95e-707e501a2d80-konnectivity-ca\") pod \"konnectivity-agent-rw5hk\" (UID: \"7c84d9e6-e6f4-4bc0-a95e-707e501a2d80\") " pod="kube-system/konnectivity-agent-rw5hk" Jul 15 12:20:29.452436 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.451861 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/37807770-29e4-460a-bea4-efc59d333d9a-iptables-alerter-script\") pod \"iptables-alerter-9c5zn\" (UID: \"37807770-29e4-460a-bea4-efc59d333d9a\") " pod="openshift-network-operator/iptables-alerter-9c5zn" Jul 15 12:20:29.452436 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.451901 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/62753f0f-8783-4ec9-b5f2-1e7663da727e-sys\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.452436 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.451896 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-etc-openvswitch\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.452436 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.451934 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/62753f0f-8783-4ec9-b5f2-1e7663da727e-etc-systemd\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.452436 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.451956 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-log-socket\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.453153 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.451961 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-var-lib-openvswitch\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.453153 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.452007 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/62753f0f-8783-4ec9-b5f2-1e7663da727e-sys\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.453153 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.452015 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-var-lib-openvswitch\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.453153 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.452024 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/62753f0f-8783-4ec9-b5f2-1e7663da727e-etc-systemd\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.453153 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.452037 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.453153 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.452063 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/62753f0f-8783-4ec9-b5f2-1e7663da727e-run\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.453153 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.452066 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/62753f0f-8783-4ec9-b5f2-1e7663da727e-lib-modules\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.453153 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.452097 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/37807770-29e4-460a-bea4-efc59d333d9a-host-slash\") pod \"iptables-alerter-9c5zn\" (UID: \"37807770-29e4-460a-bea4-efc59d333d9a\") " pod="openshift-network-operator/iptables-alerter-9c5zn" Jul 15 12:20:29.453153 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.452122 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hz8n2\" (UniqueName: \"kubernetes.io/projected/37807770-29e4-460a-bea4-efc59d333d9a-kube-api-access-hz8n2\") pod \"iptables-alerter-9c5zn\" (UID: \"37807770-29e4-460a-bea4-efc59d333d9a\") " pod="openshift-network-operator/iptables-alerter-9c5zn" Jul 15 12:20:29.453153 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.452123 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/808669ad-89bb-473a-86e9-7562fcd99736-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.453153 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.452145 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7f4w\" (UniqueName: \"kubernetes.io/projected/808669ad-89bb-473a-86e9-7562fcd99736-kube-api-access-p7f4w\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.453153 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.452172 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/37807770-29e4-460a-bea4-efc59d333d9a-host-slash\") pod \"iptables-alerter-9c5zn\" (UID: \"37807770-29e4-460a-bea4-efc59d333d9a\") " pod="openshift-network-operator/iptables-alerter-9c5zn" Jul 15 12:20:29.453153 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.452181 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/62753f0f-8783-4ec9-b5f2-1e7663da727e-lib-modules\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.453153 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.453094 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/808669ad-89bb-473a-86e9-7562fcd99736-env-overrides\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.453894 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.453584 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/808669ad-89bb-473a-86e9-7562fcd99736-ovnkube-script-lib\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.454037 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.454011 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/808669ad-89bb-473a-86e9-7562fcd99736-ovnkube-config\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.454430 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.454410 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7c84d9e6-e6f4-4bc0-a95e-707e501a2d80-agent-certs\") pod \"konnectivity-agent-rw5hk\" (UID: \"7c84d9e6-e6f4-4bc0-a95e-707e501a2d80\") " pod="kube-system/konnectivity-agent-rw5hk" Jul 15 12:20:29.454945 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.454913 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/62753f0f-8783-4ec9-b5f2-1e7663da727e-etc-tuned\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.457352 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.457026 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/808669ad-89bb-473a-86e9-7562fcd99736-ovn-node-metrics-cert\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.457620 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:29.457599 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 12:20:29.457745 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:29.457629 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 12:20:29.457745 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:29.457661 2560 projected.go:196] Error preparing data for projected volume kube-api-access-vprwx for pod openshift-network-diagnostics/network-check-target-c94x9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 12:20:29.457745 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:29.457729 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8db43c0-4eee-4627-b30d-53206ad7666a-kube-api-access-vprwx podName:b8db43c0-4eee-4627-b30d-53206ad7666a nodeName:}" failed. No retries permitted until 2026-07-15 12:20:29.9577109 +0000 UTC m=+3.098507007 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vprwx" (UniqueName: "kubernetes.io/projected/b8db43c0-4eee-4627-b30d-53206ad7666a-kube-api-access-vprwx") pod "network-check-target-c94x9" (UID: "b8db43c0-4eee-4627-b30d-53206ad7666a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 12:20:29.457952 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.457744 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/62753f0f-8783-4ec9-b5f2-1e7663da727e-tmp\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.460740 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.460554 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58vbb\" (UniqueName: \"kubernetes.io/projected/05b26d09-b957-4b76-80fd-b30046fbbfc5-kube-api-access-58vbb\") pod \"node-ca-m7tbh\" (UID: \"05b26d09-b957-4b76-80fd-b30046fbbfc5\") " pod="openshift-image-registry/node-ca-m7tbh" Jul 15 12:20:29.460916 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.460897 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggrvb\" (UniqueName: \"kubernetes.io/projected/332b1403-fae5-4270-b17f-5b7597d59006-kube-api-access-ggrvb\") pod \"network-metrics-daemon-h7bxb\" (UID: \"332b1403-fae5-4270-b17f-5b7597d59006\") " pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:20:29.462496 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.462474 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7f4w\" (UniqueName: \"kubernetes.io/projected/808669ad-89bb-473a-86e9-7562fcd99736-kube-api-access-p7f4w\") pod \"ovnkube-node-bg9fm\" (UID: \"808669ad-89bb-473a-86e9-7562fcd99736\") " pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.462896 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.462873 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hz8n2\" (UniqueName: \"kubernetes.io/projected/37807770-29e4-460a-bea4-efc59d333d9a-kube-api-access-hz8n2\") pod \"iptables-alerter-9c5zn\" (UID: \"37807770-29e4-460a-bea4-efc59d333d9a\") " pod="openshift-network-operator/iptables-alerter-9c5zn" Jul 15 12:20:29.464682 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.464662 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bvdz\" (UniqueName: \"kubernetes.io/projected/62753f0f-8783-4ec9-b5f2-1e7663da727e-kube-api-access-6bvdz\") pod \"tuned-whxjk\" (UID: \"62753f0f-8783-4ec9-b5f2-1e7663da727e\") " pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.536105 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.536067 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zvlp" Jul 15 12:20:29.547749 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.547729 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-694pn" Jul 15 12:20:29.557715 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.557693 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lcf4n" Jul 15 12:20:29.561339 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.561313 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6xkhw" Jul 15 12:20:29.569115 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.569095 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:29.576910 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.576891 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rw5hk" Jul 15 12:20:29.586168 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.586149 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9c5zn" Jul 15 12:20:29.593678 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.593662 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-whxjk" Jul 15 12:20:29.601273 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.601254 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-m7tbh" Jul 15 12:20:29.854272 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.854202 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a919b128-8dca-4cee-81a2-51a65300820c-original-pull-secret\") pod \"global-pull-secret-syncer-79bqk\" (UID: \"a919b128-8dca-4cee-81a2-51a65300820c\") " pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:20:29.854403 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:29.854312 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 12:20:29.854403 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:29.854362 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a919b128-8dca-4cee-81a2-51a65300820c-original-pull-secret podName:a919b128-8dca-4cee-81a2-51a65300820c nodeName:}" failed. No retries permitted until 2026-07-15 12:20:30.854349669 +0000 UTC m=+3.995145757 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a919b128-8dca-4cee-81a2-51a65300820c-original-pull-secret") pod "global-pull-secret-syncer-79bqk" (UID: "a919b128-8dca-4cee-81a2-51a65300820c") : object "kube-system"/"original-pull-secret" not registered Jul 15 12:20:29.955447 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:29.955416 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/332b1403-fae5-4270-b17f-5b7597d59006-metrics-certs\") pod \"network-metrics-daemon-h7bxb\" (UID: \"332b1403-fae5-4270-b17f-5b7597d59006\") " pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:20:29.955623 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:29.955540 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 12:20:29.955623 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:29.955603 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/332b1403-fae5-4270-b17f-5b7597d59006-metrics-certs podName:332b1403-fae5-4270-b17f-5b7597d59006 nodeName:}" failed. No retries permitted until 2026-07-15 12:20:30.955585714 +0000 UTC m=+4.096381820 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/332b1403-fae5-4270-b17f-5b7597d59006-metrics-certs") pod "network-metrics-daemon-h7bxb" (UID: "332b1403-fae5-4270-b17f-5b7597d59006") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 12:20:30.055904 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:30.055869 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vprwx\" (UniqueName: \"kubernetes.io/projected/b8db43c0-4eee-4627-b30d-53206ad7666a-kube-api-access-vprwx\") pod \"network-check-target-c94x9\" (UID: \"b8db43c0-4eee-4627-b30d-53206ad7666a\") " pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:20:30.056075 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:30.056056 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 12:20:30.056121 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:30.056081 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 12:20:30.056121 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:30.056095 2560 projected.go:196] Error preparing data for projected volume kube-api-access-vprwx for pod openshift-network-diagnostics/network-check-target-c94x9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 12:20:30.056183 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:30.056153 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8db43c0-4eee-4627-b30d-53206ad7666a-kube-api-access-vprwx podName:b8db43c0-4eee-4627-b30d-53206ad7666a nodeName:}" failed. No retries permitted until 2026-07-15 12:20:31.056139954 +0000 UTC m=+4.196936043 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vprwx" (UniqueName: "kubernetes.io/projected/b8db43c0-4eee-4627-b30d-53206ad7666a-kube-api-access-vprwx") pod "network-check-target-c94x9" (UID: "b8db43c0-4eee-4627-b30d-53206ad7666a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 12:20:30.282793 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:30.282713 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 12:15:29 +0000 UTC" deadline="2028-04-11 19:50:58.232242896 +0000 UTC" Jul 15 12:20:30.282793 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:30.282745 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15271h30m27.949500668s" Jul 15 12:20:30.520822 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:30.520762 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74f12cb2_ed42_4446_bb5d_25cb7a992fa3.slice/crio-9d9024585f8ae5d31f9152d322fbd6d1793d9227bfcac4a3509d93ee1b0c1fcd WatchSource:0}: Error finding container 9d9024585f8ae5d31f9152d322fbd6d1793d9227bfcac4a3509d93ee1b0c1fcd: Status 404 returned error can't find the container with id 9d9024585f8ae5d31f9152d322fbd6d1793d9227bfcac4a3509d93ee1b0c1fcd Jul 15 12:20:30.522126 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:30.522102 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde344d40_415e_49c1_b0eb_67c0292a59fa.slice/crio-f19444c7ce23f27ef7d629ebe815651c4b59273450ea3ae5f6610668571c9778 WatchSource:0}: Error finding container f19444c7ce23f27ef7d629ebe815651c4b59273450ea3ae5f6610668571c9778: Status 404 returned error can't find the container with id f19444c7ce23f27ef7d629ebe815651c4b59273450ea3ae5f6610668571c9778 Jul 15 12:20:30.527156 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:30.527063 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62753f0f_8783_4ec9_b5f2_1e7663da727e.slice/crio-791a7313a8531d3413c4f50c52bb0739cda89884a72c67686f2e531e78fe9422 WatchSource:0}: Error finding container 791a7313a8531d3413c4f50c52bb0739cda89884a72c67686f2e531e78fe9422: Status 404 returned error can't find the container with id 791a7313a8531d3413c4f50c52bb0739cda89884a72c67686f2e531e78fe9422 Jul 15 12:20:30.528372 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:30.528349 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7028a878_0c31_4518_ac5b_b63dc858a010.slice/crio-656f5ea07aa3446b3a7e3e8927811e4902ee939464202fca4f36bd8d904fe53c WatchSource:0}: Error finding container 656f5ea07aa3446b3a7e3e8927811e4902ee939464202fca4f36bd8d904fe53c: Status 404 returned error can't find the container with id 656f5ea07aa3446b3a7e3e8927811e4902ee939464202fca4f36bd8d904fe53c Jul 15 12:20:30.529215 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:30.529192 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod808669ad_89bb_473a_86e9_7562fcd99736.slice/crio-915ae02fa8b6d0824972aa1d4cd3debd0a1f37d61b41a9417acae1265a2cfaca WatchSource:0}: Error finding container 915ae02fa8b6d0824972aa1d4cd3debd0a1f37d61b41a9417acae1265a2cfaca: Status 404 returned error can't find the container with id 915ae02fa8b6d0824972aa1d4cd3debd0a1f37d61b41a9417acae1265a2cfaca Jul 15 12:20:30.530298 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:30.530236 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37807770_29e4_460a_bea4_efc59d333d9a.slice/crio-37b06f4e4716472a08b16af411ddd648a1f1021c0c3d012652c518989cd4cd55 WatchSource:0}: Error finding container 37b06f4e4716472a08b16af411ddd648a1f1021c0c3d012652c518989cd4cd55: Status 404 returned error can't find the container with id 37b06f4e4716472a08b16af411ddd648a1f1021c0c3d012652c518989cd4cd55 Jul 15 12:20:30.531962 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:30.531938 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod610463f4_1622_4297_9bd5_cf12fc02ed55.slice/crio-9735891abb0f04a5573b3eb941deb93ff7df46df898cbb999a14b4a4e95f5291 WatchSource:0}: Error finding container 9735891abb0f04a5573b3eb941deb93ff7df46df898cbb999a14b4a4e95f5291: Status 404 returned error can't find the container with id 9735891abb0f04a5573b3eb941deb93ff7df46df898cbb999a14b4a4e95f5291 Jul 15 12:20:30.533002 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:30.532947 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c84d9e6_e6f4_4bc0_a95e_707e501a2d80.slice/crio-ab2958250ad2f23cc59dd7de544b80234cdf60ca98c7ce1c2158151cddeaaa22 WatchSource:0}: Error finding container ab2958250ad2f23cc59dd7de544b80234cdf60ca98c7ce1c2158151cddeaaa22: Status 404 returned error can't find the container with id ab2958250ad2f23cc59dd7de544b80234cdf60ca98c7ce1c2158151cddeaaa22 Jul 15 12:20:30.533764 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:20:30.533694 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05b26d09_b957_4b76_80fd_b30046fbbfc5.slice/crio-73353f449762d1fc83dcf8cd12f0b56c013e3a63365181561c8ea6dd058e4897 WatchSource:0}: Error finding container 73353f449762d1fc83dcf8cd12f0b56c013e3a63365181561c8ea6dd058e4897: Status 404 returned error can't find the container with id 73353f449762d1fc83dcf8cd12f0b56c013e3a63365181561c8ea6dd058e4897 Jul 15 12:20:30.861378 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:30.861153 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a919b128-8dca-4cee-81a2-51a65300820c-original-pull-secret\") pod \"global-pull-secret-syncer-79bqk\" (UID: \"a919b128-8dca-4cee-81a2-51a65300820c\") " pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:20:30.861378 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:30.861298 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 12:20:30.861537 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:30.861421 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a919b128-8dca-4cee-81a2-51a65300820c-original-pull-secret podName:a919b128-8dca-4cee-81a2-51a65300820c nodeName:}" failed. No retries permitted until 2026-07-15 12:20:32.8614077 +0000 UTC m=+6.002203790 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a919b128-8dca-4cee-81a2-51a65300820c-original-pull-secret") pod "global-pull-secret-syncer-79bqk" (UID: "a919b128-8dca-4cee-81a2-51a65300820c") : object "kube-system"/"original-pull-secret" not registered Jul 15 12:20:30.962657 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:30.962586 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/332b1403-fae5-4270-b17f-5b7597d59006-metrics-certs\") pod \"network-metrics-daemon-h7bxb\" (UID: \"332b1403-fae5-4270-b17f-5b7597d59006\") " pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:20:30.962817 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:30.962750 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 12:20:30.962885 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:30.962834 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/332b1403-fae5-4270-b17f-5b7597d59006-metrics-certs podName:332b1403-fae5-4270-b17f-5b7597d59006 nodeName:}" failed. No retries permitted until 2026-07-15 12:20:32.962812404 +0000 UTC m=+6.103608493 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/332b1403-fae5-4270-b17f-5b7597d59006-metrics-certs") pod "network-metrics-daemon-h7bxb" (UID: "332b1403-fae5-4270-b17f-5b7597d59006") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 12:20:31.063809 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:31.063766 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vprwx\" (UniqueName: \"kubernetes.io/projected/b8db43c0-4eee-4627-b30d-53206ad7666a-kube-api-access-vprwx\") pod \"network-check-target-c94x9\" (UID: \"b8db43c0-4eee-4627-b30d-53206ad7666a\") " pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:20:31.064006 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:31.063988 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 12:20:31.064066 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:31.064017 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 12:20:31.064066 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:31.064031 2560 projected.go:196] Error preparing data for projected volume kube-api-access-vprwx for pod openshift-network-diagnostics/network-check-target-c94x9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 12:20:31.064158 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:31.064088 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8db43c0-4eee-4627-b30d-53206ad7666a-kube-api-access-vprwx podName:b8db43c0-4eee-4627-b30d-53206ad7666a nodeName:}" failed. No retries permitted until 2026-07-15 12:20:33.064070332 +0000 UTC m=+6.204866424 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vprwx" (UniqueName: "kubernetes.io/projected/b8db43c0-4eee-4627-b30d-53206ad7666a-kube-api-access-vprwx") pod "network-check-target-c94x9" (UID: "b8db43c0-4eee-4627-b30d-53206ad7666a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 12:20:31.283861 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:31.283648 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 12:15:29 +0000 UTC" deadline="2027-12-08 13:26:36.500344064 +0000 UTC" Jul 15 12:20:31.283861 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:31.283704 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12265h6m5.216643825s" Jul 15 12:20:31.406141 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:31.405173 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:20:31.406141 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:31.405315 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h7bxb" podUID="332b1403-fae5-4270-b17f-5b7597d59006" Jul 15 12:20:31.406141 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:31.405810 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:20:31.406141 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:31.405938 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79bqk" podUID="a919b128-8dca-4cee-81a2-51a65300820c" Jul 15 12:20:31.406141 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:31.406004 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:20:31.406141 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:31.406078 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c94x9" podUID="b8db43c0-4eee-4627-b30d-53206ad7666a" Jul 15 12:20:31.416030 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:31.415970 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lcf4n" event={"ID":"de344d40-415e-49c1-b0eb-67c0292a59fa","Type":"ContainerStarted","Data":"f19444c7ce23f27ef7d629ebe815651c4b59273450ea3ae5f6610668571c9778"} Jul 15 12:20:31.426531 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:31.426399 2560 generic.go:356] "Generic (PLEG): container finished" podID="2a2da66c918180b41f7f0e19a0be1a68" containerID="f9487b1555485869610b67941e46f5d69a9b7f9588c91489c2e838ee5edfcce3" exitCode=0 Jul 15 12:20:31.426531 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:31.426485 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-188.ec2.internal" event={"ID":"2a2da66c918180b41f7f0e19a0be1a68","Type":"ContainerDied","Data":"f9487b1555485869610b67941e46f5d69a9b7f9588c91489c2e838ee5edfcce3"} Jul 15 12:20:31.452547 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:31.452402 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" event={"ID":"808669ad-89bb-473a-86e9-7562fcd99736","Type":"ContainerStarted","Data":"915ae02fa8b6d0824972aa1d4cd3debd0a1f37d61b41a9417acae1265a2cfaca"} Jul 15 12:20:31.463458 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:31.463375 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9c5zn" event={"ID":"37807770-29e4-460a-bea4-efc59d333d9a","Type":"ContainerStarted","Data":"37b06f4e4716472a08b16af411ddd648a1f1021c0c3d012652c518989cd4cd55"} Jul 15 12:20:31.469203 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:31.469135 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6xkhw" event={"ID":"74f12cb2-ed42-4446-bb5d-25cb7a992fa3","Type":"ContainerStarted","Data":"9d9024585f8ae5d31f9152d322fbd6d1793d9227bfcac4a3509d93ee1b0c1fcd"} Jul 15 12:20:31.478302 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:31.478276 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-188.ec2.internal" event={"ID":"a30df3ddb10f29f1462b23c270b4893e","Type":"ContainerStarted","Data":"889775082fd3818df7606811b2c5b675719abcebd1eb4c2481123d8b698c20e5"} Jul 15 12:20:31.503544 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:31.503463 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-694pn" event={"ID":"7028a878-0c31-4518-ac5b-b63dc858a010","Type":"ContainerStarted","Data":"656f5ea07aa3446b3a7e3e8927811e4902ee939464202fca4f36bd8d904fe53c"} Jul 15 12:20:31.510452 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:31.510424 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-m7tbh" event={"ID":"05b26d09-b957-4b76-80fd-b30046fbbfc5","Type":"ContainerStarted","Data":"73353f449762d1fc83dcf8cd12f0b56c013e3a63365181561c8ea6dd058e4897"} Jul 15 12:20:31.515005 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:31.514203 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-188.ec2.internal" podStartSLOduration=3.514183914 podStartE2EDuration="3.514183914s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:20:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:20:31.51340298 +0000 UTC m=+4.654199093" watchObservedRunningTime="2026-07-15 12:20:31.514183914 +0000 UTC m=+4.654980027" Jul 15 12:20:31.526754 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:31.526719 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rw5hk" event={"ID":"7c84d9e6-e6f4-4bc0-a95e-707e501a2d80","Type":"ContainerStarted","Data":"ab2958250ad2f23cc59dd7de544b80234cdf60ca98c7ce1c2158151cddeaaa22"} Jul 15 12:20:31.551519 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:31.551453 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zvlp" event={"ID":"610463f4-1622-4297-9bd5-cf12fc02ed55","Type":"ContainerStarted","Data":"9735891abb0f04a5573b3eb941deb93ff7df46df898cbb999a14b4a4e95f5291"} Jul 15 12:20:31.558969 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:31.558905 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-whxjk" event={"ID":"62753f0f-8783-4ec9-b5f2-1e7663da727e","Type":"ContainerStarted","Data":"791a7313a8531d3413c4f50c52bb0739cda89884a72c67686f2e531e78fe9422"} Jul 15 12:20:32.587609 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:32.587521 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-188.ec2.internal" event={"ID":"2a2da66c918180b41f7f0e19a0be1a68","Type":"ContainerStarted","Data":"554c484846d763797909c0161382b92df160ea2c22295c992dc3a06b7b6f74fe"} Jul 15 12:20:32.880813 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:32.880155 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a919b128-8dca-4cee-81a2-51a65300820c-original-pull-secret\") pod \"global-pull-secret-syncer-79bqk\" (UID: \"a919b128-8dca-4cee-81a2-51a65300820c\") " pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:20:32.880813 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:32.880332 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 12:20:32.880813 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:32.880396 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a919b128-8dca-4cee-81a2-51a65300820c-original-pull-secret podName:a919b128-8dca-4cee-81a2-51a65300820c nodeName:}" failed. No retries permitted until 2026-07-15 12:20:36.880376797 +0000 UTC m=+10.021172909 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a919b128-8dca-4cee-81a2-51a65300820c-original-pull-secret") pod "global-pull-secret-syncer-79bqk" (UID: "a919b128-8dca-4cee-81a2-51a65300820c") : object "kube-system"/"original-pull-secret" not registered Jul 15 12:20:32.981800 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:32.981579 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/332b1403-fae5-4270-b17f-5b7597d59006-metrics-certs\") pod \"network-metrics-daemon-h7bxb\" (UID: \"332b1403-fae5-4270-b17f-5b7597d59006\") " pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:20:32.981800 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:32.981732 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 12:20:32.981800 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:32.981794 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/332b1403-fae5-4270-b17f-5b7597d59006-metrics-certs podName:332b1403-fae5-4270-b17f-5b7597d59006 nodeName:}" failed. No retries permitted until 2026-07-15 12:20:36.981775647 +0000 UTC m=+10.122571739 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/332b1403-fae5-4270-b17f-5b7597d59006-metrics-certs") pod "network-metrics-daemon-h7bxb" (UID: "332b1403-fae5-4270-b17f-5b7597d59006") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 12:20:33.082868 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:33.082832 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vprwx\" (UniqueName: \"kubernetes.io/projected/b8db43c0-4eee-4627-b30d-53206ad7666a-kube-api-access-vprwx\") pod \"network-check-target-c94x9\" (UID: \"b8db43c0-4eee-4627-b30d-53206ad7666a\") " pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:20:33.083085 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:33.082991 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 12:20:33.083085 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:33.083013 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 12:20:33.083085 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:33.083026 2560 projected.go:196] Error preparing data for projected volume kube-api-access-vprwx for pod openshift-network-diagnostics/network-check-target-c94x9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 12:20:33.083085 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:33.083083 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8db43c0-4eee-4627-b30d-53206ad7666a-kube-api-access-vprwx podName:b8db43c0-4eee-4627-b30d-53206ad7666a nodeName:}" failed. No retries permitted until 2026-07-15 12:20:37.083065848 +0000 UTC m=+10.223861960 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vprwx" (UniqueName: "kubernetes.io/projected/b8db43c0-4eee-4627-b30d-53206ad7666a-kube-api-access-vprwx") pod "network-check-target-c94x9" (UID: "b8db43c0-4eee-4627-b30d-53206ad7666a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 12:20:33.405439 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:33.405406 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:20:33.406885 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:33.405914 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h7bxb" podUID="332b1403-fae5-4270-b17f-5b7597d59006" Jul 15 12:20:33.416807 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:33.416759 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:20:33.416807 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:33.416772 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:20:33.416976 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:33.416921 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79bqk" podUID="a919b128-8dca-4cee-81a2-51a65300820c" Jul 15 12:20:33.417370 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:33.417026 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c94x9" podUID="b8db43c0-4eee-4627-b30d-53206ad7666a" Jul 15 12:20:35.403789 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:35.403303 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:20:35.403789 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:35.403452 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h7bxb" podUID="332b1403-fae5-4270-b17f-5b7597d59006" Jul 15 12:20:35.403789 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:35.403469 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:20:35.403789 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:35.403561 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79bqk" podUID="a919b128-8dca-4cee-81a2-51a65300820c" Jul 15 12:20:35.403789 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:35.403608 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:20:35.403789 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:35.403684 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c94x9" podUID="b8db43c0-4eee-4627-b30d-53206ad7666a" Jul 15 12:20:36.915876 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:36.915298 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a919b128-8dca-4cee-81a2-51a65300820c-original-pull-secret\") pod \"global-pull-secret-syncer-79bqk\" (UID: \"a919b128-8dca-4cee-81a2-51a65300820c\") " pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:20:36.915876 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:36.915470 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 12:20:36.915876 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:36.915533 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a919b128-8dca-4cee-81a2-51a65300820c-original-pull-secret podName:a919b128-8dca-4cee-81a2-51a65300820c nodeName:}" failed. No retries permitted until 2026-07-15 12:20:44.915516082 +0000 UTC m=+18.056312187 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a919b128-8dca-4cee-81a2-51a65300820c-original-pull-secret") pod "global-pull-secret-syncer-79bqk" (UID: "a919b128-8dca-4cee-81a2-51a65300820c") : object "kube-system"/"original-pull-secret" not registered Jul 15 12:20:37.016130 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:37.016097 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/332b1403-fae5-4270-b17f-5b7597d59006-metrics-certs\") pod \"network-metrics-daemon-h7bxb\" (UID: \"332b1403-fae5-4270-b17f-5b7597d59006\") " pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:20:37.016299 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:37.016241 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 12:20:37.016373 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:37.016303 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/332b1403-fae5-4270-b17f-5b7597d59006-metrics-certs podName:332b1403-fae5-4270-b17f-5b7597d59006 nodeName:}" failed. No retries permitted until 2026-07-15 12:20:45.016284737 +0000 UTC m=+18.157080832 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/332b1403-fae5-4270-b17f-5b7597d59006-metrics-certs") pod "network-metrics-daemon-h7bxb" (UID: "332b1403-fae5-4270-b17f-5b7597d59006") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 12:20:37.116730 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:37.116682 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vprwx\" (UniqueName: \"kubernetes.io/projected/b8db43c0-4eee-4627-b30d-53206ad7666a-kube-api-access-vprwx\") pod \"network-check-target-c94x9\" (UID: \"b8db43c0-4eee-4627-b30d-53206ad7666a\") " pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:20:37.116934 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:37.116885 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 12:20:37.116934 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:37.116912 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 12:20:37.116934 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:37.116924 2560 projected.go:196] Error preparing data for projected volume kube-api-access-vprwx for pod openshift-network-diagnostics/network-check-target-c94x9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 12:20:37.117090 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:37.116986 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8db43c0-4eee-4627-b30d-53206ad7666a-kube-api-access-vprwx podName:b8db43c0-4eee-4627-b30d-53206ad7666a nodeName:}" failed. No retries permitted until 2026-07-15 12:20:45.116966526 +0000 UTC m=+18.257762637 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vprwx" (UniqueName: "kubernetes.io/projected/b8db43c0-4eee-4627-b30d-53206ad7666a-kube-api-access-vprwx") pod "network-check-target-c94x9" (UID: "b8db43c0-4eee-4627-b30d-53206ad7666a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 12:20:37.404054 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:37.403971 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:20:37.404220 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:37.404105 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:20:37.404220 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:37.404150 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:20:37.404327 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:37.404219 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c94x9" podUID="b8db43c0-4eee-4627-b30d-53206ad7666a" Jul 15 12:20:37.404327 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:37.404227 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79bqk" podUID="a919b128-8dca-4cee-81a2-51a65300820c" Jul 15 12:20:37.404430 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:37.404337 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h7bxb" podUID="332b1403-fae5-4270-b17f-5b7597d59006" Jul 15 12:20:39.402770 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:39.402655 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:20:39.402770 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:39.402705 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:20:39.403311 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:39.402803 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h7bxb" podUID="332b1403-fae5-4270-b17f-5b7597d59006" Jul 15 12:20:39.403311 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:39.402897 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79bqk" podUID="a919b128-8dca-4cee-81a2-51a65300820c" Jul 15 12:20:39.403311 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:39.402960 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:20:39.403311 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:39.403117 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c94x9" podUID="b8db43c0-4eee-4627-b30d-53206ad7666a" Jul 15 12:20:41.403588 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:41.403462 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:20:41.403588 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:41.403482 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:20:41.403588 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:41.403577 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c94x9" podUID="b8db43c0-4eee-4627-b30d-53206ad7666a" Jul 15 12:20:41.404138 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:41.403823 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:20:41.404138 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:41.403945 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h7bxb" podUID="332b1403-fae5-4270-b17f-5b7597d59006" Jul 15 12:20:41.404138 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:41.404067 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79bqk" podUID="a919b128-8dca-4cee-81a2-51a65300820c" Jul 15 12:20:43.403265 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:43.403228 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:20:43.403736 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:43.403228 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:20:43.403736 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:43.403379 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h7bxb" podUID="332b1403-fae5-4270-b17f-5b7597d59006" Jul 15 12:20:43.403736 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:43.403425 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:20:43.403736 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:43.403510 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c94x9" podUID="b8db43c0-4eee-4627-b30d-53206ad7666a" Jul 15 12:20:43.403736 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:43.403596 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79bqk" podUID="a919b128-8dca-4cee-81a2-51a65300820c" Jul 15 12:20:44.979221 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:44.979188 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a919b128-8dca-4cee-81a2-51a65300820c-original-pull-secret\") pod \"global-pull-secret-syncer-79bqk\" (UID: \"a919b128-8dca-4cee-81a2-51a65300820c\") " pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:20:44.979677 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:44.979316 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 12:20:44.979677 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:44.979384 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a919b128-8dca-4cee-81a2-51a65300820c-original-pull-secret podName:a919b128-8dca-4cee-81a2-51a65300820c nodeName:}" failed. No retries permitted until 2026-07-15 12:21:00.979365601 +0000 UTC m=+34.120161692 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a919b128-8dca-4cee-81a2-51a65300820c-original-pull-secret") pod "global-pull-secret-syncer-79bqk" (UID: "a919b128-8dca-4cee-81a2-51a65300820c") : object "kube-system"/"original-pull-secret" not registered Jul 15 12:20:45.079436 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:45.079410 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/332b1403-fae5-4270-b17f-5b7597d59006-metrics-certs\") pod \"network-metrics-daemon-h7bxb\" (UID: \"332b1403-fae5-4270-b17f-5b7597d59006\") " pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:20:45.079616 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:45.079592 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 12:20:45.079708 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:45.079693 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/332b1403-fae5-4270-b17f-5b7597d59006-metrics-certs podName:332b1403-fae5-4270-b17f-5b7597d59006 nodeName:}" failed. No retries permitted until 2026-07-15 12:21:01.079670605 +0000 UTC m=+34.220466697 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/332b1403-fae5-4270-b17f-5b7597d59006-metrics-certs") pod "network-metrics-daemon-h7bxb" (UID: "332b1403-fae5-4270-b17f-5b7597d59006") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 12:20:45.179965 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:45.179935 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vprwx\" (UniqueName: \"kubernetes.io/projected/b8db43c0-4eee-4627-b30d-53206ad7666a-kube-api-access-vprwx\") pod \"network-check-target-c94x9\" (UID: \"b8db43c0-4eee-4627-b30d-53206ad7666a\") " pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:20:45.180126 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:45.180079 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 12:20:45.180126 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:45.180099 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 12:20:45.180126 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:45.180110 2560 projected.go:196] Error preparing data for projected volume kube-api-access-vprwx for pod openshift-network-diagnostics/network-check-target-c94x9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 12:20:45.180258 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:45.180158 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8db43c0-4eee-4627-b30d-53206ad7666a-kube-api-access-vprwx podName:b8db43c0-4eee-4627-b30d-53206ad7666a nodeName:}" failed. No retries permitted until 2026-07-15 12:21:01.180142748 +0000 UTC m=+34.320938840 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vprwx" (UniqueName: "kubernetes.io/projected/b8db43c0-4eee-4627-b30d-53206ad7666a-kube-api-access-vprwx") pod "network-check-target-c94x9" (UID: "b8db43c0-4eee-4627-b30d-53206ad7666a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 12:20:45.403021 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:45.402941 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:20:45.403021 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:45.402964 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:20:45.403222 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:45.402941 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:20:45.403222 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:45.403074 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h7bxb" podUID="332b1403-fae5-4270-b17f-5b7597d59006" Jul 15 12:20:45.403222 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:45.403161 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79bqk" podUID="a919b128-8dca-4cee-81a2-51a65300820c" Jul 15 12:20:45.403351 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:45.403237 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c94x9" podUID="b8db43c0-4eee-4627-b30d-53206ad7666a" Jul 15 12:20:47.403860 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:47.403818 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:20:47.404398 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:47.403926 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:20:47.404398 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:47.403945 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:20:47.404398 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:47.403987 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h7bxb" podUID="332b1403-fae5-4270-b17f-5b7597d59006" Jul 15 12:20:47.404398 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:47.404027 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79bqk" podUID="a919b128-8dca-4cee-81a2-51a65300820c" Jul 15 12:20:47.404398 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:47.404088 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c94x9" podUID="b8db43c0-4eee-4627-b30d-53206ad7666a" Jul 15 12:20:49.403326 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:49.403288 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:20:49.403813 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:49.403289 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:20:49.403813 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:49.403396 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h7bxb" podUID="332b1403-fae5-4270-b17f-5b7597d59006" Jul 15 12:20:49.403813 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:49.403421 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:20:49.403813 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:49.403491 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79bqk" podUID="a919b128-8dca-4cee-81a2-51a65300820c" Jul 15 12:20:49.403813 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:49.403554 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c94x9" podUID="b8db43c0-4eee-4627-b30d-53206ad7666a" Jul 15 12:20:49.630340 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:49.630043 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rw5hk" event={"ID":"7c84d9e6-e6f4-4bc0-a95e-707e501a2d80","Type":"ContainerStarted","Data":"78dc5f105c13959bc28976c849e9bfb187a9c8c69fc8a84d6a6c2026356c044e"} Jul 15 12:20:49.636722 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:49.636667 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-whxjk" event={"ID":"62753f0f-8783-4ec9-b5f2-1e7663da727e","Type":"ContainerStarted","Data":"bc0cfb3837ad608671db9c0fa9f7ceb1196abbb32049db43fd901e0e057fbc3f"} Jul 15 12:20:49.653995 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:49.653787 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-188.ec2.internal" podStartSLOduration=21.653748919 podStartE2EDuration="21.653748919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:20:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:20:32.608150636 +0000 UTC m=+5.748946751" watchObservedRunningTime="2026-07-15 12:20:49.653748919 +0000 UTC m=+22.794545044" Jul 15 12:20:49.654367 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:49.654238 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-rw5hk" podStartSLOduration=11.963262644 podStartE2EDuration="22.654219918s" totalImagesPullingTime="10.690957274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:20:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:20:49.65333807 +0000 UTC m=+22.794134269" watchObservedRunningTime="2026-07-15 12:20:49.654219918 +0000 UTC m=+22.795016030" Jul 15 12:20:49.718549 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:49.718503 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-lcf4n" podStartSLOduration=3.769350609 podStartE2EDuration="22.718487384s" totalImagesPullingTime="18.949136775s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:20:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:20:49.687832821 +0000 UTC m=+22.828628931" watchObservedRunningTime="2026-07-15 12:20:49.718487384 +0000 UTC m=+22.859283494" Jul 15 12:20:49.719012 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:49.718988 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-whxjk" podStartSLOduration=3.7872402750000003 podStartE2EDuration="22.718980753s" totalImagesPullingTime="18.931740478s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:20:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:20:49.71830942 +0000 UTC m=+22.859105532" watchObservedRunningTime="2026-07-15 12:20:49.718980753 +0000 UTC m=+22.859776866" Jul 15 12:20:49.977977 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:49.977945 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rw5hk_7c84d9e6-e6f4-4bc0-a95e-707e501a2d80/konnectivity-agent/0.log" limit=1024 Jul 15 12:20:50.067859 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:50.067721 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rw5hk_7c84d9e6-e6f4-4bc0-a95e-707e501a2d80/konnectivity-agent/0.log" limit=1024 Jul 15 12:20:50.168379 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:50.168350 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rw5hk_7c84d9e6-e6f4-4bc0-a95e-707e501a2d80/konnectivity-agent/0.log" limit=1024 Jul 15 12:20:50.269195 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:50.269160 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rw5hk_7c84d9e6-e6f4-4bc0-a95e-707e501a2d80/konnectivity-agent/0.log" limit=1024 Jul 15 12:20:50.643592 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:50.643558 2560 generic.go:356] "Generic (PLEG): container finished" podID="7028a878-0c31-4518-ac5b-b63dc858a010" containerID="e94d6448ca79cf5119e3b633c58210f81515ae5e632fb7aac0c4de811cc3f5af" exitCode=0 Jul 15 12:20:50.644483 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:50.643627 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-694pn" event={"ID":"7028a878-0c31-4518-ac5b-b63dc858a010","Type":"ContainerDied","Data":"e94d6448ca79cf5119e3b633c58210f81515ae5e632fb7aac0c4de811cc3f5af"} Jul 15 12:20:50.645156 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:50.645079 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-m7tbh" event={"ID":"05b26d09-b957-4b76-80fd-b30046fbbfc5","Type":"ContainerStarted","Data":"2d57d28549e8617b476ed79ff82aca7f16e11a87b3b18e23d2a554cf5bf60d9a"} Jul 15 12:20:50.646590 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:50.646561 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zvlp" event={"ID":"610463f4-1622-4297-9bd5-cf12fc02ed55","Type":"ContainerStarted","Data":"7b6b5b27fa6d62b2d454a2cc22f9d8c92305d1530fb564061c2940844c5385c0"} Jul 15 12:20:50.648000 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:50.647980 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lcf4n" event={"ID":"de344d40-415e-49c1-b0eb-67c0292a59fa","Type":"ContainerStarted","Data":"1aec4b3430c0f587f368628b08b66f505415b8682eb2815acff069fbf7c033e9"} Jul 15 12:20:50.650696 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:50.650680 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bg9fm_808669ad-89bb-473a-86e9-7562fcd99736/ovn-acl-logging/0.log" Jul 15 12:20:50.651004 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:50.650986 2560 generic.go:356] "Generic (PLEG): container finished" podID="808669ad-89bb-473a-86e9-7562fcd99736" containerID="65899ee1bf658515d7c48db5c08883f5099b9c20f2db31f21d1e236ec7cb7850" exitCode=1 Jul 15 12:20:50.651068 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:50.651052 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" event={"ID":"808669ad-89bb-473a-86e9-7562fcd99736","Type":"ContainerStarted","Data":"99f7e1f119c5ee63155174cff65aa0409d385880f89e46b43e8946046beb6525"} Jul 15 12:20:50.651123 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:50.651110 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" event={"ID":"808669ad-89bb-473a-86e9-7562fcd99736","Type":"ContainerStarted","Data":"76e92ad5d540259db264eef550f8864b0ae37cc4ceef8414f5592982520c6e27"} Jul 15 12:20:50.651166 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:50.651160 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" event={"ID":"808669ad-89bb-473a-86e9-7562fcd99736","Type":"ContainerStarted","Data":"8caae9401273c443c2ea7e98a1a001026da2772805011bb1c9164913e80a9569"} Jul 15 12:20:50.651199 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:50.651189 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" event={"ID":"808669ad-89bb-473a-86e9-7562fcd99736","Type":"ContainerStarted","Data":"1c432e8e52a130d7828639bebe327c57cd3461335de0ee2b49c121745f385112"} Jul 15 12:20:50.651242 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:50.651230 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" event={"ID":"808669ad-89bb-473a-86e9-7562fcd99736","Type":"ContainerDied","Data":"65899ee1bf658515d7c48db5c08883f5099b9c20f2db31f21d1e236ec7cb7850"} Jul 15 12:20:50.651286 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:50.651273 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" event={"ID":"808669ad-89bb-473a-86e9-7562fcd99736","Type":"ContainerStarted","Data":"08d578c7144af9f897fbeabc73f41beaaa31cd4722865fd1ebc690ec8899d5d2"} Jul 15 12:20:50.652269 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:50.652245 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6xkhw" event={"ID":"74f12cb2-ed42-4446-bb5d-25cb7a992fa3","Type":"ContainerStarted","Data":"66416b6b6578021f5195ad9775ae7c1ee19e6768719001f3d8fb81b175480c19"} Jul 15 12:20:50.685551 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:50.685501 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-m7tbh" podStartSLOduration=4.762217517 podStartE2EDuration="23.685487681s" totalImagesPullingTime="18.923270164s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:20:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:20:50.685076436 +0000 UTC m=+23.825872749" watchObservedRunningTime="2026-07-15 12:20:50.685487681 +0000 UTC m=+23.826283792" Jul 15 12:20:50.709546 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:50.709499 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-6xkhw" podStartSLOduration=4.796459141 podStartE2EDuration="23.709484802s" totalImagesPullingTime="18.913025661s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:20:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:20:50.70935805 +0000 UTC m=+23.850154161" watchObservedRunningTime="2026-07-15 12:20:50.709484802 +0000 UTC m=+23.850280913" Jul 15 12:20:50.822396 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:50.822357 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rw5hk_7c84d9e6-e6f4-4bc0-a95e-707e501a2d80/konnectivity-agent/0.log" limit=1024 Jul 15 12:20:50.923265 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:50.923231 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rw5hk_7c84d9e6-e6f4-4bc0-a95e-707e501a2d80/konnectivity-agent/0.log" limit=1024 Jul 15 12:20:50.991241 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:50.991218 2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 15 12:20:51.307766 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:51.307599 2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T12:20:50.991235574Z","UUID":"d8474352-bb75-4f4c-87f6-78518ebc1e01","Handler":null,"Name":"","Endpoint":""} Jul 15 12:20:51.309333 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:51.309303 2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 15 12:20:51.309451 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:51.309350 2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 15 12:20:51.402928 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:51.402879 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:20:51.403167 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:51.403027 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79bqk" podUID="a919b128-8dca-4cee-81a2-51a65300820c" Jul 15 12:20:51.403167 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:51.402879 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:20:51.403167 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:51.402879 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:20:51.403167 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:51.403135 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c94x9" podUID="b8db43c0-4eee-4627-b30d-53206ad7666a" Jul 15 12:20:51.403381 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:51.403250 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h7bxb" podUID="332b1403-fae5-4270-b17f-5b7597d59006" Jul 15 12:20:51.656973 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:51.656882 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zvlp" event={"ID":"610463f4-1622-4297-9bd5-cf12fc02ed55","Type":"ContainerStarted","Data":"715ff729d65ab2afbe5e3934cf447da36fac1731564317d497e8778fcd5fa6a6"} Jul 15 12:20:51.658514 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:51.658484 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9c5zn" event={"ID":"37807770-29e4-460a-bea4-efc59d333d9a","Type":"ContainerStarted","Data":"50d54a41a549e09a3a4bd8b0f05de849cfb5664479c6e6c37456ef1a544e340f"} Jul 15 12:20:51.675216 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:51.675169 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9c5zn" podStartSLOduration=5.748690788 podStartE2EDuration="24.675155131s" totalImagesPullingTime="18.926464343s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:20:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:20:51.674967727 +0000 UTC m=+24.815763842" watchObservedRunningTime="2026-07-15 12:20:51.675155131 +0000 UTC m=+24.815951247" Jul 15 12:20:51.797037 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:51.796999 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rw5hk_7c84d9e6-e6f4-4bc0-a95e-707e501a2d80/konnectivity-agent/0.log" limit=1024 Jul 15 12:20:51.932425 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:51.932329 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rw5hk_7c84d9e6-e6f4-4bc0-a95e-707e501a2d80/konnectivity-agent/0.log" limit=1024 Jul 15 12:20:52.081713 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:52.081676 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rw5hk_7c84d9e6-e6f4-4bc0-a95e-707e501a2d80/konnectivity-agent/0.log" limit=1024 Jul 15 12:20:52.182851 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:52.182782 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rw5hk_7c84d9e6-e6f4-4bc0-a95e-707e501a2d80/konnectivity-agent/0.log" limit=1024 Jul 15 12:20:52.664471 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:52.664401 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bg9fm_808669ad-89bb-473a-86e9-7562fcd99736/ovn-acl-logging/0.log" Jul 15 12:20:52.664951 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:52.664891 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" event={"ID":"808669ad-89bb-473a-86e9-7562fcd99736","Type":"ContainerStarted","Data":"7b611fa0997f4f41a59f1b2433781e7421043995ff3b68825cb256799a4ebc3e"} Jul 15 12:20:52.667261 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:52.667239 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zvlp" event={"ID":"610463f4-1622-4297-9bd5-cf12fc02ed55","Type":"ContainerStarted","Data":"07c4eb4d1611708dc8a5efc299288f8949301a5e26ce3c410328160b413fd828"} Jul 15 12:20:52.685490 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:52.685448 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zvlp" podStartSLOduration=4.453561413 podStartE2EDuration="25.685430021s" totalImagesPullingTime="21.231868608s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:20:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:20:52.684909727 +0000 UTC m=+25.825705863" watchObservedRunningTime="2026-07-15 12:20:52.685430021 +0000 UTC m=+25.826226135" Jul 15 12:20:52.840598 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:52.840392 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-rw5hk" Jul 15 12:20:53.402870 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:53.402829 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:20:53.402870 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:53.402859 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:20:53.403107 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:53.402829 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:20:53.403107 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:53.402942 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h7bxb" podUID="332b1403-fae5-4270-b17f-5b7597d59006" Jul 15 12:20:53.403107 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:53.403004 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c94x9" podUID="b8db43c0-4eee-4627-b30d-53206ad7666a" Jul 15 12:20:53.403107 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:53.403086 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79bqk" podUID="a919b128-8dca-4cee-81a2-51a65300820c" Jul 15 12:20:54.306204 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:54.306164 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-rw5hk" Jul 15 12:20:54.306914 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:54.306895 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-rw5hk" Jul 15 12:20:54.676182 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:54.676160 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bg9fm_808669ad-89bb-473a-86e9-7562fcd99736/ovn-acl-logging/0.log" Jul 15 12:20:54.676716 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:54.676608 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" event={"ID":"808669ad-89bb-473a-86e9-7562fcd99736","Type":"ContainerStarted","Data":"432cda8bd6b098d8d79591701212e3c74c2b007319bc57e15080f286cb11b29a"} Jul 15 12:20:54.677203 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:54.677164 2560 scope.go:117] "RemoveContainer" containerID="65899ee1bf658515d7c48db5c08883f5099b9c20f2db31f21d1e236ec7cb7850" Jul 15 12:20:54.677520 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:54.677462 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-rw5hk" Jul 15 12:20:55.403060 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:55.402877 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:20:55.403740 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:55.402877 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:20:55.403740 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:55.403155 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79bqk" podUID="a919b128-8dca-4cee-81a2-51a65300820c" Jul 15 12:20:55.403740 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:55.402938 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:20:55.403740 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:55.403219 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h7bxb" podUID="332b1403-fae5-4270-b17f-5b7597d59006" Jul 15 12:20:55.403740 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:55.403323 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c94x9" podUID="b8db43c0-4eee-4627-b30d-53206ad7666a" Jul 15 12:20:55.681995 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:55.681924 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bg9fm_808669ad-89bb-473a-86e9-7562fcd99736/ovn-acl-logging/0.log" Jul 15 12:20:55.682319 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:55.682293 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" event={"ID":"808669ad-89bb-473a-86e9-7562fcd99736","Type":"ContainerStarted","Data":"da2d92be86906ff4963f40b266e0efe646e4403104dd85396ac5de97a81df64d"} Jul 15 12:20:55.682384 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:55.682371 2560 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 15 12:20:55.682626 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:55.682607 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:55.682734 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:55.682692 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:55.684248 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:55.684224 2560 generic.go:356] "Generic (PLEG): container finished" podID="7028a878-0c31-4518-ac5b-b63dc858a010" containerID="e48a00b7310b0e8d8b77d1c43fd9982a38fddd514d63c62d0e6fcbb1c0d63292" exitCode=0 Jul 15 12:20:55.684354 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:55.684291 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-694pn" event={"ID":"7028a878-0c31-4518-ac5b-b63dc858a010","Type":"ContainerDied","Data":"e48a00b7310b0e8d8b77d1c43fd9982a38fddd514d63c62d0e6fcbb1c0d63292"} Jul 15 12:20:55.697940 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:55.697921 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:55.698022 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:55.698012 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:55.748487 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:55.748447 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" podStartSLOduration=9.722633106 podStartE2EDuration="28.748431663s" totalImagesPullingTime="19.025798557s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:20:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:20:55.719787352 +0000 UTC m=+28.860583481" watchObservedRunningTime="2026-07-15 12:20:55.748431663 +0000 UTC m=+28.889227773" Jul 15 12:20:56.686873 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:56.686845 2560 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 15 12:20:56.987158 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:56.986917 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-h7bxb"] Jul 15 12:20:56.987288 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:56.987273 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:20:56.987409 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:56.987391 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h7bxb" podUID="332b1403-fae5-4270-b17f-5b7597d59006" Jul 15 12:20:56.996164 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:56.996140 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-c94x9"] Jul 15 12:20:56.996259 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:56.996223 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:20:56.996303 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:56.996286 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c94x9" podUID="b8db43c0-4eee-4627-b30d-53206ad7666a" Jul 15 12:20:57.009565 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:57.009541 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-79bqk"] Jul 15 12:20:57.009656 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:57.009618 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:20:57.009728 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:57.009708 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79bqk" podUID="a919b128-8dca-4cee-81a2-51a65300820c" Jul 15 12:20:57.274548 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:57.274477 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:20:57.691055 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:57.690975 2560 generic.go:356] "Generic (PLEG): container finished" podID="7028a878-0c31-4518-ac5b-b63dc858a010" containerID="7c3fc45dc14e0129cbb887b8b15966c56e4deb5f650622364f73de91fed5be1e" exitCode=0 Jul 15 12:20:57.691412 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:57.691064 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-694pn" event={"ID":"7028a878-0c31-4518-ac5b-b63dc858a010","Type":"ContainerDied","Data":"7c3fc45dc14e0129cbb887b8b15966c56e4deb5f650622364f73de91fed5be1e"} Jul 15 12:20:58.403032 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:58.402999 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:20:58.403032 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:58.403029 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:20:58.403314 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:58.403109 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c94x9" podUID="b8db43c0-4eee-4627-b30d-53206ad7666a" Jul 15 12:20:58.403314 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:58.403160 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h7bxb" podUID="332b1403-fae5-4270-b17f-5b7597d59006" Jul 15 12:20:58.403314 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:58.403200 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:20:58.403314 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:20:58.403261 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79bqk" podUID="a919b128-8dca-4cee-81a2-51a65300820c" Jul 15 12:20:59.700185 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:59.700148 2560 generic.go:356] "Generic (PLEG): container finished" podID="7028a878-0c31-4518-ac5b-b63dc858a010" containerID="00de2c580d5c437e01e189f70466bf066b07cfd44044e8c756098faf7c65cf64" exitCode=0 Jul 15 12:20:59.700839 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:20:59.700224 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-694pn" event={"ID":"7028a878-0c31-4518-ac5b-b63dc858a010","Type":"ContainerDied","Data":"00de2c580d5c437e01e189f70466bf066b07cfd44044e8c756098faf7c65cf64"} Jul 15 12:21:00.402531 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:00.402499 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:21:00.402732 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:00.402499 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:21:00.402732 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:00.402649 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79bqk" podUID="a919b128-8dca-4cee-81a2-51a65300820c" Jul 15 12:21:00.402837 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:00.402730 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h7bxb" podUID="332b1403-fae5-4270-b17f-5b7597d59006" Jul 15 12:21:00.402837 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:00.402499 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:21:00.402837 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:00.402808 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c94x9" podUID="b8db43c0-4eee-4627-b30d-53206ad7666a" Jul 15 12:21:00.994843 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:00.994753 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a919b128-8dca-4cee-81a2-51a65300820c-original-pull-secret\") pod \"global-pull-secret-syncer-79bqk\" (UID: \"a919b128-8dca-4cee-81a2-51a65300820c\") " pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:21:00.995318 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:00.994916 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 12:21:00.995318 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:00.994985 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a919b128-8dca-4cee-81a2-51a65300820c-original-pull-secret podName:a919b128-8dca-4cee-81a2-51a65300820c nodeName:}" failed. No retries permitted until 2026-07-15 12:21:32.994966455 +0000 UTC m=+66.135762563 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a919b128-8dca-4cee-81a2-51a65300820c-original-pull-secret") pod "global-pull-secret-syncer-79bqk" (UID: "a919b128-8dca-4cee-81a2-51a65300820c") : object "kube-system"/"original-pull-secret" not registered Jul 15 12:21:01.095232 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:01.095199 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/332b1403-fae5-4270-b17f-5b7597d59006-metrics-certs\") pod \"network-metrics-daemon-h7bxb\" (UID: \"332b1403-fae5-4270-b17f-5b7597d59006\") " pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:21:01.095370 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:01.095301 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 12:21:01.095370 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:01.095353 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/332b1403-fae5-4270-b17f-5b7597d59006-metrics-certs podName:332b1403-fae5-4270-b17f-5b7597d59006 nodeName:}" failed. No retries permitted until 2026-07-15 12:21:33.095338911 +0000 UTC m=+66.236135001 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/332b1403-fae5-4270-b17f-5b7597d59006-metrics-certs") pod "network-metrics-daemon-h7bxb" (UID: "332b1403-fae5-4270-b17f-5b7597d59006") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 12:21:01.195764 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:01.195726 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vprwx\" (UniqueName: \"kubernetes.io/projected/b8db43c0-4eee-4627-b30d-53206ad7666a-kube-api-access-vprwx\") pod \"network-check-target-c94x9\" (UID: \"b8db43c0-4eee-4627-b30d-53206ad7666a\") " pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:21:01.195947 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:01.195926 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 12:21:01.196013 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:01.195955 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 12:21:01.196013 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:01.195968 2560 projected.go:196] Error preparing data for projected volume kube-api-access-vprwx for pod openshift-network-diagnostics/network-check-target-c94x9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 12:21:01.196107 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:01.196028 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8db43c0-4eee-4627-b30d-53206ad7666a-kube-api-access-vprwx podName:b8db43c0-4eee-4627-b30d-53206ad7666a nodeName:}" failed. No retries permitted until 2026-07-15 12:21:33.196007555 +0000 UTC m=+66.336803647 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vprwx" (UniqueName: "kubernetes.io/projected/b8db43c0-4eee-4627-b30d-53206ad7666a-kube-api-access-vprwx") pod "network-check-target-c94x9" (UID: "b8db43c0-4eee-4627-b30d-53206ad7666a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 12:21:02.402687 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:02.402647 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:21:02.402687 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:02.402677 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:21:02.403167 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:02.402676 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:21:02.403167 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:02.402787 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c94x9" podUID="b8db43c0-4eee-4627-b30d-53206ad7666a" Jul 15 12:21:02.403167 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:02.402852 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79bqk" podUID="a919b128-8dca-4cee-81a2-51a65300820c" Jul 15 12:21:02.403167 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:02.402931 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h7bxb" podUID="332b1403-fae5-4270-b17f-5b7597d59006" Jul 15 12:21:04.402513 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:04.402483 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:21:04.402513 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:04.402495 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:21:04.403010 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:04.402595 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c94x9" podUID="b8db43c0-4eee-4627-b30d-53206ad7666a" Jul 15 12:21:04.403010 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:04.402646 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:21:04.403010 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:04.402726 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h7bxb" podUID="332b1403-fae5-4270-b17f-5b7597d59006" Jul 15 12:21:04.403010 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:04.402788 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79bqk" podUID="a919b128-8dca-4cee-81a2-51a65300820c" Jul 15 12:21:04.715386 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:04.715308 2560 generic.go:356] "Generic (PLEG): container finished" podID="7028a878-0c31-4518-ac5b-b63dc858a010" containerID="48a6e3ace7244bfbe099f029d9326f74736a1d8def812fe2a7d768ade3a2700c" exitCode=0 Jul 15 12:21:04.715524 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:04.715388 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-694pn" event={"ID":"7028a878-0c31-4518-ac5b-b63dc858a010","Type":"ContainerDied","Data":"48a6e3ace7244bfbe099f029d9326f74736a1d8def812fe2a7d768ade3a2700c"} Jul 15 12:21:05.723214 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:05.723034 2560 generic.go:356] "Generic (PLEG): container finished" podID="7028a878-0c31-4518-ac5b-b63dc858a010" containerID="d06c346034d7239b43fdad17a903340e86b2c2b7c9c025b8b4a4fdb0b16ffa58" exitCode=0 Jul 15 12:21:05.723571 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:05.723112 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-694pn" event={"ID":"7028a878-0c31-4518-ac5b-b63dc858a010","Type":"ContainerDied","Data":"d06c346034d7239b43fdad17a903340e86b2c2b7c9c025b8b4a4fdb0b16ffa58"} Jul 15 12:21:06.403083 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:06.403060 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:21:06.403227 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:06.403060 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:21:06.403290 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:06.403244 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c94x9" podUID="b8db43c0-4eee-4627-b30d-53206ad7666a" Jul 15 12:21:06.403290 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:06.403152 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79bqk" podUID="a919b128-8dca-4cee-81a2-51a65300820c" Jul 15 12:21:06.403290 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:06.403060 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:21:06.403422 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:06.403358 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h7bxb" podUID="332b1403-fae5-4270-b17f-5b7597d59006" Jul 15 12:21:06.728596 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:06.728519 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-694pn" event={"ID":"7028a878-0c31-4518-ac5b-b63dc858a010","Type":"ContainerStarted","Data":"55a84de77ce11484e72a86016224bffc05e3fa78ab90fd197f07faf195657534"} Jul 15 12:21:06.761065 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:06.761019 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-694pn" podStartSLOduration=10.728900869 podStartE2EDuration="39.761001859s" totalImagesPullingTime="29.03210099s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:20:27 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:21:06.760578853 +0000 UTC m=+39.901374976" watchObservedRunningTime="2026-07-15 12:21:06.761001859 +0000 UTC m=+39.901797986" Jul 15 12:21:08.403244 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:08.403209 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:21:08.403632 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:08.403308 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:21:08.403632 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:08.403313 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79bqk" podUID="a919b128-8dca-4cee-81a2-51a65300820c" Jul 15 12:21:08.403632 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:08.403320 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:21:08.403632 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:08.403384 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c94x9" podUID="b8db43c0-4eee-4627-b30d-53206ad7666a" Jul 15 12:21:08.403632 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:08.403452 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h7bxb" podUID="332b1403-fae5-4270-b17f-5b7597d59006" Jul 15 12:21:10.402583 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:10.402549 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:21:10.403051 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:10.402683 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c94x9" podUID="b8db43c0-4eee-4627-b30d-53206ad7666a" Jul 15 12:21:10.403051 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:10.402716 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:21:10.403051 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:10.402725 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:21:10.403051 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:10.402805 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79bqk" podUID="a919b128-8dca-4cee-81a2-51a65300820c" Jul 15 12:21:10.403051 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:10.402869 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h7bxb" podUID="332b1403-fae5-4270-b17f-5b7597d59006" Jul 15 12:21:12.402766 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:12.402739 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:21:12.403118 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:12.402739 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:21:12.403118 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:12.402857 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h7bxb" podUID="332b1403-fae5-4270-b17f-5b7597d59006" Jul 15 12:21:12.403118 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:12.402739 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:21:12.403118 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:12.402904 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79bqk" podUID="a919b128-8dca-4cee-81a2-51a65300820c" Jul 15 12:21:12.403118 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:12.402969 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c94x9" podUID="b8db43c0-4eee-4627-b30d-53206ad7666a" Jul 15 12:21:14.403401 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:14.403374 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:21:14.403856 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:14.403374 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:21:14.403856 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:14.403475 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h7bxb" podUID="332b1403-fae5-4270-b17f-5b7597d59006" Jul 15 12:21:14.403856 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:14.403561 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79bqk" podUID="a919b128-8dca-4cee-81a2-51a65300820c" Jul 15 12:21:14.403856 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:14.403374 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:21:14.403856 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:14.403629 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c94x9" podUID="b8db43c0-4eee-4627-b30d-53206ad7666a" Jul 15 12:21:16.403411 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:16.403371 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:21:16.403411 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:16.403410 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:21:16.403827 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:16.403380 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:21:16.403827 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:16.403473 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c94x9" podUID="b8db43c0-4eee-4627-b30d-53206ad7666a" Jul 15 12:21:16.403827 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:16.403555 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79bqk" podUID="a919b128-8dca-4cee-81a2-51a65300820c" Jul 15 12:21:16.403827 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:16.403658 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h7bxb" podUID="332b1403-fae5-4270-b17f-5b7597d59006" Jul 15 12:21:18.403024 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:18.402990 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:21:18.403409 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:18.403090 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:21:18.403409 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:18.403103 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:21:18.403409 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:18.403103 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79bqk" podUID="a919b128-8dca-4cee-81a2-51a65300820c" Jul 15 12:21:18.403409 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:18.403176 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c94x9" podUID="b8db43c0-4eee-4627-b30d-53206ad7666a" Jul 15 12:21:18.403409 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:18.403238 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h7bxb" podUID="332b1403-fae5-4270-b17f-5b7597d59006" Jul 15 12:21:19.172824 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.172797 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-188.ec2.internal" event="NodeReady" Jul 15 12:21:19.172964 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.172900 2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 15 12:21:19.234090 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.234023 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5dff78dbc9-kf6kz"] Jul 15 12:21:19.246247 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.246229 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5dff78dbc9-kf6kz" Jul 15 12:21:19.253729 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.253710 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4bnx5\"" Jul 15 12:21:19.253897 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.253883 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 15 12:21:19.254207 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.254194 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 15 12:21:19.259738 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.259721 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 15 12:21:19.264677 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.264656 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 15 12:21:19.296692 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.296666 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5dff78dbc9-kf6kz"] Jul 15 12:21:19.297172 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.297157 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rt7x5"] Jul 15 12:21:19.318202 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.318182 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5cpsl"] Jul 15 12:21:19.318305 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.318289 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rt7x5" Jul 15 12:21:19.323598 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.323581 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 15 12:21:19.323930 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.323910 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 15 12:21:19.324223 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.324206 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bx6xw\"" Jul 15 12:21:19.324368 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.324349 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 15 12:21:19.329812 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.329795 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f8d0552b-2236-45a7-b3bb-57681252e2e5-ca-trust-extracted\") pod \"image-registry-5dff78dbc9-kf6kz\" (UID: \"f8d0552b-2236-45a7-b3bb-57681252e2e5\") " pod="openshift-image-registry/image-registry-5dff78dbc9-kf6kz" Jul 15 12:21:19.329888 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.329820 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f8d0552b-2236-45a7-b3bb-57681252e2e5-installation-pull-secrets\") pod \"image-registry-5dff78dbc9-kf6kz\" (UID: \"f8d0552b-2236-45a7-b3bb-57681252e2e5\") " pod="openshift-image-registry/image-registry-5dff78dbc9-kf6kz" Jul 15 12:21:19.329888 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.329837 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8d0552b-2236-45a7-b3bb-57681252e2e5-bound-sa-token\") pod \"image-registry-5dff78dbc9-kf6kz\" (UID: \"f8d0552b-2236-45a7-b3bb-57681252e2e5\") " pod="openshift-image-registry/image-registry-5dff78dbc9-kf6kz" Jul 15 12:21:19.329888 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.329873 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f8d0552b-2236-45a7-b3bb-57681252e2e5-image-registry-private-configuration\") pod \"image-registry-5dff78dbc9-kf6kz\" (UID: \"f8d0552b-2236-45a7-b3bb-57681252e2e5\") " pod="openshift-image-registry/image-registry-5dff78dbc9-kf6kz" Jul 15 12:21:19.329991 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.329938 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f8d0552b-2236-45a7-b3bb-57681252e2e5-registry-certificates\") pod \"image-registry-5dff78dbc9-kf6kz\" (UID: \"f8d0552b-2236-45a7-b3bb-57681252e2e5\") " pod="openshift-image-registry/image-registry-5dff78dbc9-kf6kz" Jul 15 12:21:19.329991 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.329978 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8d0552b-2236-45a7-b3bb-57681252e2e5-trusted-ca\") pod \"image-registry-5dff78dbc9-kf6kz\" (UID: \"f8d0552b-2236-45a7-b3bb-57681252e2e5\") " pod="openshift-image-registry/image-registry-5dff78dbc9-kf6kz" Jul 15 12:21:19.330053 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.330012 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8d0552b-2236-45a7-b3bb-57681252e2e5-registry-tls\") pod \"image-registry-5dff78dbc9-kf6kz\" (UID: \"f8d0552b-2236-45a7-b3bb-57681252e2e5\") " pod="openshift-image-registry/image-registry-5dff78dbc9-kf6kz" Jul 15 12:21:19.330092 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.330050 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2zrq\" (UniqueName: \"kubernetes.io/projected/f8d0552b-2236-45a7-b3bb-57681252e2e5-kube-api-access-p2zrq\") pod \"image-registry-5dff78dbc9-kf6kz\" (UID: \"f8d0552b-2236-45a7-b3bb-57681252e2e5\") " pod="openshift-image-registry/image-registry-5dff78dbc9-kf6kz" Jul 15 12:21:19.338354 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.338304 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5cpsl" Jul 15 12:21:19.352548 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.352529 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 15 12:21:19.352627 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.352529 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 15 12:21:19.357870 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.357857 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hw8dx\"" Jul 15 12:21:19.374386 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.374362 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rt7x5"] Jul 15 12:21:19.392316 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.392297 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5cpsl"] Jul 15 12:21:19.430320 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.430299 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f8d0552b-2236-45a7-b3bb-57681252e2e5-ca-trust-extracted\") pod \"image-registry-5dff78dbc9-kf6kz\" (UID: \"f8d0552b-2236-45a7-b3bb-57681252e2e5\") " pod="openshift-image-registry/image-registry-5dff78dbc9-kf6kz" Jul 15 12:21:19.430711 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.430326 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f8d0552b-2236-45a7-b3bb-57681252e2e5-installation-pull-secrets\") pod \"image-registry-5dff78dbc9-kf6kz\" (UID: \"f8d0552b-2236-45a7-b3bb-57681252e2e5\") " pod="openshift-image-registry/image-registry-5dff78dbc9-kf6kz" Jul 15 12:21:19.430711 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.430345 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8d0552b-2236-45a7-b3bb-57681252e2e5-bound-sa-token\") pod \"image-registry-5dff78dbc9-kf6kz\" (UID: \"f8d0552b-2236-45a7-b3bb-57681252e2e5\") " pod="openshift-image-registry/image-registry-5dff78dbc9-kf6kz" Jul 15 12:21:19.430711 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.430369 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f742cf0-bd7c-4cf1-9e4a-1214def20eff-tmp-dir\") pod \"dns-default-5cpsl\" (UID: \"5f742cf0-bd7c-4cf1-9e4a-1214def20eff\") " pod="openshift-dns/dns-default-5cpsl" Jul 15 12:21:19.430711 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.430409 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f8d0552b-2236-45a7-b3bb-57681252e2e5-image-registry-private-configuration\") pod \"image-registry-5dff78dbc9-kf6kz\" (UID: \"f8d0552b-2236-45a7-b3bb-57681252e2e5\") " pod="openshift-image-registry/image-registry-5dff78dbc9-kf6kz" Jul 15 12:21:19.430711 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.430456 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f8d0552b-2236-45a7-b3bb-57681252e2e5-registry-certificates\") pod \"image-registry-5dff78dbc9-kf6kz\" (UID: \"f8d0552b-2236-45a7-b3bb-57681252e2e5\") " pod="openshift-image-registry/image-registry-5dff78dbc9-kf6kz" Jul 15 12:21:19.430711 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.430537 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8d0552b-2236-45a7-b3bb-57681252e2e5-trusted-ca\") pod \"image-registry-5dff78dbc9-kf6kz\" (UID: \"f8d0552b-2236-45a7-b3bb-57681252e2e5\") " pod="openshift-image-registry/image-registry-5dff78dbc9-kf6kz" Jul 15 12:21:19.430711 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.430565 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbcsn\" (UniqueName: \"kubernetes.io/projected/4efdeca1-8ed8-43bb-9ec0-d7a54a30e3f9-kube-api-access-zbcsn\") pod \"ingress-canary-rt7x5\" (UID: \"4efdeca1-8ed8-43bb-9ec0-d7a54a30e3f9\") " pod="openshift-ingress-canary/ingress-canary-rt7x5" Jul 15 12:21:19.430711 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.430591 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5f742cf0-bd7c-4cf1-9e4a-1214def20eff-config-volume\") pod \"dns-default-5cpsl\" (UID: \"5f742cf0-bd7c-4cf1-9e4a-1214def20eff\") " pod="openshift-dns/dns-default-5cpsl" Jul 15 12:21:19.430711 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.430613 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4wqt\" (UniqueName: \"kubernetes.io/projected/5f742cf0-bd7c-4cf1-9e4a-1214def20eff-kube-api-access-h4wqt\") pod \"dns-default-5cpsl\" (UID: \"5f742cf0-bd7c-4cf1-9e4a-1214def20eff\") " pod="openshift-dns/dns-default-5cpsl" Jul 15 12:21:19.431175 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.430753 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f8d0552b-2236-45a7-b3bb-57681252e2e5-ca-trust-extracted\") pod \"image-registry-5dff78dbc9-kf6kz\" (UID: \"f8d0552b-2236-45a7-b3bb-57681252e2e5\") " pod="openshift-image-registry/image-registry-5dff78dbc9-kf6kz" Jul 15 12:21:19.431175 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.430785 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8d0552b-2236-45a7-b3bb-57681252e2e5-registry-tls\") pod \"image-registry-5dff78dbc9-kf6kz\" (UID: \"f8d0552b-2236-45a7-b3bb-57681252e2e5\") " pod="openshift-image-registry/image-registry-5dff78dbc9-kf6kz" Jul 15 12:21:19.431175 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.430830 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2zrq\" (UniqueName: \"kubernetes.io/projected/f8d0552b-2236-45a7-b3bb-57681252e2e5-kube-api-access-p2zrq\") pod \"image-registry-5dff78dbc9-kf6kz\" (UID: \"f8d0552b-2236-45a7-b3bb-57681252e2e5\") " pod="openshift-image-registry/image-registry-5dff78dbc9-kf6kz" Jul 15 12:21:19.431175 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.430856 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f742cf0-bd7c-4cf1-9e4a-1214def20eff-metrics-tls\") pod \"dns-default-5cpsl\" (UID: \"5f742cf0-bd7c-4cf1-9e4a-1214def20eff\") " pod="openshift-dns/dns-default-5cpsl" Jul 15 12:21:19.431175 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.430881 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4efdeca1-8ed8-43bb-9ec0-d7a54a30e3f9-cert\") pod \"ingress-canary-rt7x5\" (UID: \"4efdeca1-8ed8-43bb-9ec0-d7a54a30e3f9\") " pod="openshift-ingress-canary/ingress-canary-rt7x5" Jul 15 12:21:19.431418 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.431303 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f8d0552b-2236-45a7-b3bb-57681252e2e5-registry-certificates\") pod \"image-registry-5dff78dbc9-kf6kz\" (UID: \"f8d0552b-2236-45a7-b3bb-57681252e2e5\") " pod="openshift-image-registry/image-registry-5dff78dbc9-kf6kz" Jul 15 12:21:19.431594 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.431574 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8d0552b-2236-45a7-b3bb-57681252e2e5-trusted-ca\") pod \"image-registry-5dff78dbc9-kf6kz\" (UID: \"f8d0552b-2236-45a7-b3bb-57681252e2e5\") " pod="openshift-image-registry/image-registry-5dff78dbc9-kf6kz" Jul 15 12:21:19.434728 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.434700 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f8d0552b-2236-45a7-b3bb-57681252e2e5-installation-pull-secrets\") pod \"image-registry-5dff78dbc9-kf6kz\" (UID: \"f8d0552b-2236-45a7-b3bb-57681252e2e5\") " pod="openshift-image-registry/image-registry-5dff78dbc9-kf6kz" Jul 15 12:21:19.434800 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.434725 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f8d0552b-2236-45a7-b3bb-57681252e2e5-image-registry-private-configuration\") pod \"image-registry-5dff78dbc9-kf6kz\" (UID: \"f8d0552b-2236-45a7-b3bb-57681252e2e5\") " pod="openshift-image-registry/image-registry-5dff78dbc9-kf6kz" Jul 15 12:21:19.434800 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.434731 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8d0552b-2236-45a7-b3bb-57681252e2e5-registry-tls\") pod \"image-registry-5dff78dbc9-kf6kz\" (UID: \"f8d0552b-2236-45a7-b3bb-57681252e2e5\") " pod="openshift-image-registry/image-registry-5dff78dbc9-kf6kz" Jul 15 12:21:19.448486 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.448469 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2zrq\" (UniqueName: \"kubernetes.io/projected/f8d0552b-2236-45a7-b3bb-57681252e2e5-kube-api-access-p2zrq\") pod \"image-registry-5dff78dbc9-kf6kz\" (UID: \"f8d0552b-2236-45a7-b3bb-57681252e2e5\") " pod="openshift-image-registry/image-registry-5dff78dbc9-kf6kz" Jul 15 12:21:19.451451 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.451431 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8d0552b-2236-45a7-b3bb-57681252e2e5-bound-sa-token\") pod \"image-registry-5dff78dbc9-kf6kz\" (UID: \"f8d0552b-2236-45a7-b3bb-57681252e2e5\") " pod="openshift-image-registry/image-registry-5dff78dbc9-kf6kz" Jul 15 12:21:19.531330 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.531276 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f742cf0-bd7c-4cf1-9e4a-1214def20eff-metrics-tls\") pod \"dns-default-5cpsl\" (UID: \"5f742cf0-bd7c-4cf1-9e4a-1214def20eff\") " pod="openshift-dns/dns-default-5cpsl" Jul 15 12:21:19.531330 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.531305 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4efdeca1-8ed8-43bb-9ec0-d7a54a30e3f9-cert\") pod \"ingress-canary-rt7x5\" (UID: \"4efdeca1-8ed8-43bb-9ec0-d7a54a30e3f9\") " pod="openshift-ingress-canary/ingress-canary-rt7x5" Jul 15 12:21:19.531474 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.531335 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f742cf0-bd7c-4cf1-9e4a-1214def20eff-tmp-dir\") pod \"dns-default-5cpsl\" (UID: \"5f742cf0-bd7c-4cf1-9e4a-1214def20eff\") " pod="openshift-dns/dns-default-5cpsl" Jul 15 12:21:19.531533 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.531516 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbcsn\" (UniqueName: \"kubernetes.io/projected/4efdeca1-8ed8-43bb-9ec0-d7a54a30e3f9-kube-api-access-zbcsn\") pod \"ingress-canary-rt7x5\" (UID: \"4efdeca1-8ed8-43bb-9ec0-d7a54a30e3f9\") " pod="openshift-ingress-canary/ingress-canary-rt7x5" Jul 15 12:21:19.531587 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.531545 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5f742cf0-bd7c-4cf1-9e4a-1214def20eff-config-volume\") pod \"dns-default-5cpsl\" (UID: \"5f742cf0-bd7c-4cf1-9e4a-1214def20eff\") " pod="openshift-dns/dns-default-5cpsl" Jul 15 12:21:19.531587 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.531561 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4wqt\" (UniqueName: \"kubernetes.io/projected/5f742cf0-bd7c-4cf1-9e4a-1214def20eff-kube-api-access-h4wqt\") pod \"dns-default-5cpsl\" (UID: \"5f742cf0-bd7c-4cf1-9e4a-1214def20eff\") " pod="openshift-dns/dns-default-5cpsl" Jul 15 12:21:19.531993 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.531973 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5f742cf0-bd7c-4cf1-9e4a-1214def20eff-config-volume\") pod \"dns-default-5cpsl\" (UID: \"5f742cf0-bd7c-4cf1-9e4a-1214def20eff\") " pod="openshift-dns/dns-default-5cpsl" Jul 15 12:21:19.533443 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.533416 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f742cf0-bd7c-4cf1-9e4a-1214def20eff-metrics-tls\") pod \"dns-default-5cpsl\" (UID: \"5f742cf0-bd7c-4cf1-9e4a-1214def20eff\") " pod="openshift-dns/dns-default-5cpsl" Jul 15 12:21:19.533519 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.533488 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4efdeca1-8ed8-43bb-9ec0-d7a54a30e3f9-cert\") pod \"ingress-canary-rt7x5\" (UID: \"4efdeca1-8ed8-43bb-9ec0-d7a54a30e3f9\") " pod="openshift-ingress-canary/ingress-canary-rt7x5" Jul 15 12:21:19.542266 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.542249 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f742cf0-bd7c-4cf1-9e4a-1214def20eff-tmp-dir\") pod \"dns-default-5cpsl\" (UID: \"5f742cf0-bd7c-4cf1-9e4a-1214def20eff\") " pod="openshift-dns/dns-default-5cpsl" Jul 15 12:21:19.554401 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.554383 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5dff78dbc9-kf6kz" Jul 15 12:21:19.568500 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.568480 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4wqt\" (UniqueName: \"kubernetes.io/projected/5f742cf0-bd7c-4cf1-9e4a-1214def20eff-kube-api-access-h4wqt\") pod \"dns-default-5cpsl\" (UID: \"5f742cf0-bd7c-4cf1-9e4a-1214def20eff\") " pod="openshift-dns/dns-default-5cpsl" Jul 15 12:21:19.568701 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.568684 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbcsn\" (UniqueName: \"kubernetes.io/projected/4efdeca1-8ed8-43bb-9ec0-d7a54a30e3f9-kube-api-access-zbcsn\") pod \"ingress-canary-rt7x5\" (UID: \"4efdeca1-8ed8-43bb-9ec0-d7a54a30e3f9\") " pod="openshift-ingress-canary/ingress-canary-rt7x5" Jul 15 12:21:19.626516 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.626474 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rt7x5" Jul 15 12:21:19.646995 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.646965 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5cpsl" Jul 15 12:21:19.777161 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.777136 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5dff78dbc9-kf6kz"] Jul 15 12:21:19.781000 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:21:19.780976 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8d0552b_2236_45a7_b3bb_57681252e2e5.slice/crio-944c36a0caf041adb2e03922ce5e08f3c134e62cb1617073a79bdaabc517a085 WatchSource:0}: Error finding container 944c36a0caf041adb2e03922ce5e08f3c134e62cb1617073a79bdaabc517a085: Status 404 returned error can't find the container with id 944c36a0caf041adb2e03922ce5e08f3c134e62cb1617073a79bdaabc517a085 Jul 15 12:21:19.820149 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.820126 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rt7x5"] Jul 15 12:21:19.829602 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:21:19.829581 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4efdeca1_8ed8_43bb_9ec0_d7a54a30e3f9.slice/crio-64311f87618f2a433a1b397b51bce95889cb510d5f4744d72c6820a7253cf371 WatchSource:0}: Error finding container 64311f87618f2a433a1b397b51bce95889cb510d5f4744d72c6820a7253cf371: Status 404 returned error can't find the container with id 64311f87618f2a433a1b397b51bce95889cb510d5f4744d72c6820a7253cf371 Jul 15 12:21:19.841351 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:21:19.841325 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f742cf0_bd7c_4cf1_9e4a_1214def20eff.slice/crio-4578b78c01b4747057b3d16007721103a6920cc82afceb0ee2e51061c2c1cb7c WatchSource:0}: Error finding container 4578b78c01b4747057b3d16007721103a6920cc82afceb0ee2e51061c2c1cb7c: Status 404 returned error can't find the container with id 4578b78c01b4747057b3d16007721103a6920cc82afceb0ee2e51061c2c1cb7c Jul 15 12:21:19.842853 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:19.842756 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5cpsl"] Jul 15 12:21:20.403443 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:20.403414 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:21:20.403595 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:20.403414 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:21:20.403595 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:20.403536 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:21:20.409617 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:20.409590 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pcnmv\"" Jul 15 12:21:20.409617 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:20.409608 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 12:21:20.410539 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:20.410519 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 12:21:20.410731 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:20.410565 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nnkkc\"" Jul 15 12:21:20.410731 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:20.410519 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 12:21:20.410883 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:20.410864 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 12:21:20.761279 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:20.761198 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rt7x5" event={"ID":"4efdeca1-8ed8-43bb-9ec0-d7a54a30e3f9","Type":"ContainerStarted","Data":"64311f87618f2a433a1b397b51bce95889cb510d5f4744d72c6820a7253cf371"} Jul 15 12:21:20.763524 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:20.763496 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5dff78dbc9-kf6kz" event={"ID":"f8d0552b-2236-45a7-b3bb-57681252e2e5","Type":"ContainerStarted","Data":"eef2a1de0dcc24b3b83b5ee5f13620a78aa512818df2b84b568ce546b9c9857f"} Jul 15 12:21:20.763659 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:20.763545 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5dff78dbc9-kf6kz" event={"ID":"f8d0552b-2236-45a7-b3bb-57681252e2e5","Type":"ContainerStarted","Data":"944c36a0caf041adb2e03922ce5e08f3c134e62cb1617073a79bdaabc517a085"} Jul 15 12:21:20.763859 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:20.763841 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5dff78dbc9-kf6kz" Jul 15 12:21:20.766940 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:20.766907 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5cpsl" event={"ID":"5f742cf0-bd7c-4cf1-9e4a-1214def20eff","Type":"ContainerStarted","Data":"4578b78c01b4747057b3d16007721103a6920cc82afceb0ee2e51061c2c1cb7c"} Jul 15 12:21:20.797607 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:20.797551 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5dff78dbc9-kf6kz" podStartSLOduration=2.797531259 podStartE2EDuration="2.797531259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:21:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:21:20.796871538 +0000 UTC m=+53.937667712" watchObservedRunningTime="2026-07-15 12:21:20.797531259 +0000 UTC m=+53.938327374" Jul 15 12:21:23.776534 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:23.776495 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5cpsl" event={"ID":"5f742cf0-bd7c-4cf1-9e4a-1214def20eff","Type":"ContainerStarted","Data":"cd803716f3deb57b1b9e188c068f887579a62d7080647a2bfae549359000f171"} Jul 15 12:21:23.777030 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:23.776551 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5cpsl" event={"ID":"5f742cf0-bd7c-4cf1-9e4a-1214def20eff","Type":"ContainerStarted","Data":"de1c28fc55b57c8e5e437dcfd7af8e5ef01ce5b3d446b747d2da6d2103b75279"} Jul 15 12:21:23.777030 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:23.776580 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5cpsl" Jul 15 12:21:23.777959 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:23.777936 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rt7x5" event={"ID":"4efdeca1-8ed8-43bb-9ec0-d7a54a30e3f9","Type":"ContainerStarted","Data":"b07ebbec9349f3c8f03ca974ea0166acfd632a9dfc24750f04bf3219e05fe510"} Jul 15 12:21:23.823686 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:23.823622 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5cpsl" podStartSLOduration=1.742315251 podStartE2EDuration="4.823607888s" totalImagesPullingTime="3.081292637s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:21:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:21:23.821229294 +0000 UTC m=+56.962025424" watchObservedRunningTime="2026-07-15 12:21:23.823607888 +0000 UTC m=+56.964403977" Jul 15 12:21:23.851895 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:23.851849 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rt7x5" podStartSLOduration=1.747261886 podStartE2EDuration="4.851834669s" totalImagesPullingTime="3.104572783s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:21:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:21:23.851149934 +0000 UTC m=+56.991946046" watchObservedRunningTime="2026-07-15 12:21:23.851834669 +0000 UTC m=+56.992630757" Jul 15 12:21:28.711381 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:28.711353 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bg9fm" Jul 15 12:21:31.085539 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:31.085506 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7g6wj"] Jul 15 12:21:31.089036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:31.089017 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7g6wj" Jul 15 12:21:31.094113 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:31.094084 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 15 12:21:31.094378 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:31.094365 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 15 12:21:31.095170 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:31.095144 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 15 12:21:31.095266 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:31.095182 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 15 12:21:31.095266 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:31.095189 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8wtrb\"" Jul 15 12:21:31.107056 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:31.107030 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7g6wj"] Jul 15 12:21:31.118466 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:31.118441 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c883a479-e3e8-45b1-bcc3-027f8637183e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7g6wj\" (UID: \"c883a479-e3e8-45b1-bcc3-027f8637183e\") " pod="openshift-insights/insights-runtime-extractor-7g6wj" Jul 15 12:21:31.118573 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:31.118475 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c883a479-e3e8-45b1-bcc3-027f8637183e-data-volume\") pod \"insights-runtime-extractor-7g6wj\" (UID: \"c883a479-e3e8-45b1-bcc3-027f8637183e\") " pod="openshift-insights/insights-runtime-extractor-7g6wj" Jul 15 12:21:31.118573 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:31.118492 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zd8p\" (UniqueName: \"kubernetes.io/projected/c883a479-e3e8-45b1-bcc3-027f8637183e-kube-api-access-5zd8p\") pod \"insights-runtime-extractor-7g6wj\" (UID: \"c883a479-e3e8-45b1-bcc3-027f8637183e\") " pod="openshift-insights/insights-runtime-extractor-7g6wj" Jul 15 12:21:31.118703 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:31.118608 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c883a479-e3e8-45b1-bcc3-027f8637183e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7g6wj\" (UID: \"c883a479-e3e8-45b1-bcc3-027f8637183e\") " pod="openshift-insights/insights-runtime-extractor-7g6wj" Jul 15 12:21:31.118703 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:31.118676 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c883a479-e3e8-45b1-bcc3-027f8637183e-crio-socket\") pod \"insights-runtime-extractor-7g6wj\" (UID: \"c883a479-e3e8-45b1-bcc3-027f8637183e\") " pod="openshift-insights/insights-runtime-extractor-7g6wj" Jul 15 12:21:31.219853 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:31.219818 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c883a479-e3e8-45b1-bcc3-027f8637183e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7g6wj\" (UID: \"c883a479-e3e8-45b1-bcc3-027f8637183e\") " pod="openshift-insights/insights-runtime-extractor-7g6wj" Jul 15 12:21:31.219853 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:31.219859 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c883a479-e3e8-45b1-bcc3-027f8637183e-crio-socket\") pod \"insights-runtime-extractor-7g6wj\" (UID: \"c883a479-e3e8-45b1-bcc3-027f8637183e\") " pod="openshift-insights/insights-runtime-extractor-7g6wj" Jul 15 12:21:31.220087 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:31.219885 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c883a479-e3e8-45b1-bcc3-027f8637183e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7g6wj\" (UID: \"c883a479-e3e8-45b1-bcc3-027f8637183e\") " pod="openshift-insights/insights-runtime-extractor-7g6wj" Jul 15 12:21:31.220087 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:31.219901 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c883a479-e3e8-45b1-bcc3-027f8637183e-data-volume\") pod \"insights-runtime-extractor-7g6wj\" (UID: \"c883a479-e3e8-45b1-bcc3-027f8637183e\") " pod="openshift-insights/insights-runtime-extractor-7g6wj" Jul 15 12:21:31.220087 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:31.219918 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zd8p\" (UniqueName: \"kubernetes.io/projected/c883a479-e3e8-45b1-bcc3-027f8637183e-kube-api-access-5zd8p\") pod \"insights-runtime-extractor-7g6wj\" (UID: \"c883a479-e3e8-45b1-bcc3-027f8637183e\") " pod="openshift-insights/insights-runtime-extractor-7g6wj" Jul 15 12:21:31.220087 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:31.219972 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 12:21:31.220087 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:31.219983 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 12:21:31.220087 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:31.219969 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c883a479-e3e8-45b1-bcc3-027f8637183e-crio-socket\") pod \"insights-runtime-extractor-7g6wj\" (UID: \"c883a479-e3e8-45b1-bcc3-027f8637183e\") " pod="openshift-insights/insights-runtime-extractor-7g6wj" Jul 15 12:21:31.220087 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:31.220042 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c883a479-e3e8-45b1-bcc3-027f8637183e-kube-rbac-proxy-cm podName:c883a479-e3e8-45b1-bcc3-027f8637183e nodeName:}" failed. No retries permitted until 2026-07-15 12:21:31.720021759 +0000 UTC m=+64.860817851 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c883a479-e3e8-45b1-bcc3-027f8637183e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7g6wj" (UID: "c883a479-e3e8-45b1-bcc3-027f8637183e") : configmap "kube-rbac-proxy" not found Jul 15 12:21:31.220087 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:31.220060 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c883a479-e3e8-45b1-bcc3-027f8637183e-insights-runtime-extractor-tls podName:c883a479-e3e8-45b1-bcc3-027f8637183e nodeName:}" failed. No retries permitted until 2026-07-15 12:21:31.72005342 +0000 UTC m=+64.860849508 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c883a479-e3e8-45b1-bcc3-027f8637183e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7g6wj" (UID: "c883a479-e3e8-45b1-bcc3-027f8637183e") : secret "insights-runtime-extractor-tls" not found Jul 15 12:21:31.220338 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:31.220232 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c883a479-e3e8-45b1-bcc3-027f8637183e-data-volume\") pod \"insights-runtime-extractor-7g6wj\" (UID: \"c883a479-e3e8-45b1-bcc3-027f8637183e\") " pod="openshift-insights/insights-runtime-extractor-7g6wj" Jul 15 12:21:31.233504 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:31.233484 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zd8p\" (UniqueName: \"kubernetes.io/projected/c883a479-e3e8-45b1-bcc3-027f8637183e-kube-api-access-5zd8p\") pod \"insights-runtime-extractor-7g6wj\" (UID: \"c883a479-e3e8-45b1-bcc3-027f8637183e\") " pod="openshift-insights/insights-runtime-extractor-7g6wj" Jul 15 12:21:31.651769 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:31.651741 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5cpsl_5f742cf0-bd7c-4cf1-9e4a-1214def20eff/dns/0.log" Jul 15 12:21:31.724387 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:31.724358 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c883a479-e3e8-45b1-bcc3-027f8637183e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7g6wj\" (UID: \"c883a479-e3e8-45b1-bcc3-027f8637183e\") " pod="openshift-insights/insights-runtime-extractor-7g6wj" Jul 15 12:21:31.724551 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:31.724407 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c883a479-e3e8-45b1-bcc3-027f8637183e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7g6wj\" (UID: \"c883a479-e3e8-45b1-bcc3-027f8637183e\") " pod="openshift-insights/insights-runtime-extractor-7g6wj" Jul 15 12:21:31.724551 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:31.724494 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 12:21:31.724551 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:31.724517 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 12:21:31.724551 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:31.724549 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c883a479-e3e8-45b1-bcc3-027f8637183e-kube-rbac-proxy-cm podName:c883a479-e3e8-45b1-bcc3-027f8637183e nodeName:}" failed. No retries permitted until 2026-07-15 12:21:32.724537583 +0000 UTC m=+65.865333671 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c883a479-e3e8-45b1-bcc3-027f8637183e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7g6wj" (UID: "c883a479-e3e8-45b1-bcc3-027f8637183e") : configmap "kube-rbac-proxy" not found Jul 15 12:21:31.724731 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:31.724576 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c883a479-e3e8-45b1-bcc3-027f8637183e-insights-runtime-extractor-tls podName:c883a479-e3e8-45b1-bcc3-027f8637183e nodeName:}" failed. No retries permitted until 2026-07-15 12:21:32.724560355 +0000 UTC m=+65.865356466 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c883a479-e3e8-45b1-bcc3-027f8637183e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7g6wj" (UID: "c883a479-e3e8-45b1-bcc3-027f8637183e") : secret "insights-runtime-extractor-tls" not found Jul 15 12:21:31.850871 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:31.850845 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5cpsl_5f742cf0-bd7c-4cf1-9e4a-1214def20eff/kube-rbac-proxy/0.log" Jul 15 12:21:32.732563 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:32.732526 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c883a479-e3e8-45b1-bcc3-027f8637183e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7g6wj\" (UID: \"c883a479-e3e8-45b1-bcc3-027f8637183e\") " pod="openshift-insights/insights-runtime-extractor-7g6wj" Jul 15 12:21:32.732927 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:32.732581 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c883a479-e3e8-45b1-bcc3-027f8637183e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7g6wj\" (UID: \"c883a479-e3e8-45b1-bcc3-027f8637183e\") " pod="openshift-insights/insights-runtime-extractor-7g6wj" Jul 15 12:21:32.732927 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:32.732677 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 12:21:32.732927 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:32.732690 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 12:21:32.732927 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:32.732729 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c883a479-e3e8-45b1-bcc3-027f8637183e-kube-rbac-proxy-cm podName:c883a479-e3e8-45b1-bcc3-027f8637183e nodeName:}" failed. No retries permitted until 2026-07-15 12:21:34.732715627 +0000 UTC m=+67.873511716 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c883a479-e3e8-45b1-bcc3-027f8637183e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7g6wj" (UID: "c883a479-e3e8-45b1-bcc3-027f8637183e") : configmap "kube-rbac-proxy" not found Jul 15 12:21:32.732927 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:32.732742 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c883a479-e3e8-45b1-bcc3-027f8637183e-insights-runtime-extractor-tls podName:c883a479-e3e8-45b1-bcc3-027f8637183e nodeName:}" failed. No retries permitted until 2026-07-15 12:21:34.732736536 +0000 UTC m=+67.873532625 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c883a479-e3e8-45b1-bcc3-027f8637183e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7g6wj" (UID: "c883a479-e3e8-45b1-bcc3-027f8637183e") : secret "insights-runtime-extractor-tls" not found Jul 15 12:21:32.848950 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:32.848924 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6xkhw_74f12cb2-ed42-4446-bb5d-25cb7a992fa3/dns-node-resolver/0.log" Jul 15 12:21:33.035537 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:33.035444 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a919b128-8dca-4cee-81a2-51a65300820c-original-pull-secret\") pod \"global-pull-secret-syncer-79bqk\" (UID: \"a919b128-8dca-4cee-81a2-51a65300820c\") " pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:21:33.038164 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:33.038143 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 12:21:33.047853 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:33.047820 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a919b128-8dca-4cee-81a2-51a65300820c-original-pull-secret\") pod \"global-pull-secret-syncer-79bqk\" (UID: \"a919b128-8dca-4cee-81a2-51a65300820c\") " pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:21:33.136717 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:33.136682 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/332b1403-fae5-4270-b17f-5b7597d59006-metrics-certs\") pod \"network-metrics-daemon-h7bxb\" (UID: \"332b1403-fae5-4270-b17f-5b7597d59006\") " pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:21:33.139321 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:33.139304 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 12:21:33.152485 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:33.149157 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/332b1403-fae5-4270-b17f-5b7597d59006-metrics-certs\") pod \"network-metrics-daemon-h7bxb\" (UID: \"332b1403-fae5-4270-b17f-5b7597d59006\") " pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:21:33.237978 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:33.237937 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vprwx\" (UniqueName: \"kubernetes.io/projected/b8db43c0-4eee-4627-b30d-53206ad7666a-kube-api-access-vprwx\") pod \"network-check-target-c94x9\" (UID: \"b8db43c0-4eee-4627-b30d-53206ad7666a\") " pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:21:33.240876 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:33.240858 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 12:21:33.255042 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:33.255026 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 12:21:33.261900 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:33.261882 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vprwx\" (UniqueName: \"kubernetes.io/projected/b8db43c0-4eee-4627-b30d-53206ad7666a-kube-api-access-vprwx\") pod \"network-check-target-c94x9\" (UID: \"b8db43c0-4eee-4627-b30d-53206ad7666a\") " pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:21:33.313918 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:33.313892 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79bqk" Jul 15 12:21:33.323619 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:33.323599 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pcnmv\"" Jul 15 12:21:33.330020 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:33.330005 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nnkkc\"" Jul 15 12:21:33.332097 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:33.332079 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h7bxb" Jul 15 12:21:33.338054 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:33.338021 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:21:33.449927 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:33.449898 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5dff78dbc9-kf6kz_f8d0552b-2236-45a7-b3bb-57681252e2e5/registry/0.log" Jul 15 12:21:33.477225 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:33.477196 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-79bqk"] Jul 15 12:21:33.489994 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:21:33.489972 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda919b128_8dca_4cee_81a2_51a65300820c.slice/crio-3f15d8f91adaafec9085240830a5987bcba03916f2c7779ac2eeeb69c11c2709 WatchSource:0}: Error finding container 3f15d8f91adaafec9085240830a5987bcba03916f2c7779ac2eeeb69c11c2709: Status 404 returned error can't find the container with id 3f15d8f91adaafec9085240830a5987bcba03916f2c7779ac2eeeb69c11c2709 Jul 15 12:21:33.495100 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:21:33.495079 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod332b1403_fae5_4270_b17f_5b7597d59006.slice/crio-7c9f1d8c79e87a32f5f620f3fca05f6607c9be4aa071a1d4fb7546cfd718c3ed WatchSource:0}: Error finding container 7c9f1d8c79e87a32f5f620f3fca05f6607c9be4aa071a1d4fb7546cfd718c3ed: Status 404 returned error can't find the container with id 7c9f1d8c79e87a32f5f620f3fca05f6607c9be4aa071a1d4fb7546cfd718c3ed Jul 15 12:21:33.497516 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:33.497482 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-h7bxb"] Jul 15 12:21:33.527816 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:33.527791 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-c94x9"] Jul 15 12:21:33.530024 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:21:33.530002 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8db43c0_4eee_4627_b30d_53206ad7666a.slice/crio-85b32788ed0038761b79de3ff1bfce239e0089ea7d45acbc7bc2547aa212cc1f WatchSource:0}: Error finding container 85b32788ed0038761b79de3ff1bfce239e0089ea7d45acbc7bc2547aa212cc1f: Status 404 returned error can't find the container with id 85b32788ed0038761b79de3ff1bfce239e0089ea7d45acbc7bc2547aa212cc1f Jul 15 12:21:33.783511 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:33.783430 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5cpsl" Jul 15 12:21:33.803353 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:33.803325 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-h7bxb" event={"ID":"332b1403-fae5-4270-b17f-5b7597d59006","Type":"ContainerStarted","Data":"7c9f1d8c79e87a32f5f620f3fca05f6607c9be4aa071a1d4fb7546cfd718c3ed"} Jul 15 12:21:33.804390 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:33.804369 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-c94x9" event={"ID":"b8db43c0-4eee-4627-b30d-53206ad7666a","Type":"ContainerStarted","Data":"85b32788ed0038761b79de3ff1bfce239e0089ea7d45acbc7bc2547aa212cc1f"} Jul 15 12:21:33.805386 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:33.805366 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-79bqk" event={"ID":"a919b128-8dca-4cee-81a2-51a65300820c","Type":"ContainerStarted","Data":"3f15d8f91adaafec9085240830a5987bcba03916f2c7779ac2eeeb69c11c2709"} Jul 15 12:21:34.250889 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:34.250838 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-m7tbh_05b26d09-b957-4b76-80fd-b30046fbbfc5/node-ca/0.log" Jul 15 12:21:34.749980 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:34.749951 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c883a479-e3e8-45b1-bcc3-027f8637183e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7g6wj\" (UID: \"c883a479-e3e8-45b1-bcc3-027f8637183e\") " pod="openshift-insights/insights-runtime-extractor-7g6wj" Jul 15 12:21:34.750104 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:34.750028 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c883a479-e3e8-45b1-bcc3-027f8637183e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7g6wj\" (UID: \"c883a479-e3e8-45b1-bcc3-027f8637183e\") " pod="openshift-insights/insights-runtime-extractor-7g6wj" Jul 15 12:21:34.750596 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:34.750547 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c883a479-e3e8-45b1-bcc3-027f8637183e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7g6wj\" (UID: \"c883a479-e3e8-45b1-bcc3-027f8637183e\") " pod="openshift-insights/insights-runtime-extractor-7g6wj" Jul 15 12:21:34.754746 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:34.754722 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c883a479-e3e8-45b1-bcc3-027f8637183e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7g6wj\" (UID: \"c883a479-e3e8-45b1-bcc3-027f8637183e\") " pod="openshift-insights/insights-runtime-extractor-7g6wj" Jul 15 12:21:34.811338 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:34.811240 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-h7bxb" event={"ID":"332b1403-fae5-4270-b17f-5b7597d59006","Type":"ContainerStarted","Data":"e2e7197dfcecc973531a461f3b0a22af9fdf45bf0b03293f650b086bafce756b"} Jul 15 12:21:34.811338 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:34.811303 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-h7bxb" event={"ID":"332b1403-fae5-4270-b17f-5b7597d59006","Type":"ContainerStarted","Data":"5ed4085718198120c39919149b6b976eccd7e70776721ac913ca63459b169dc7"} Jul 15 12:21:34.830585 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:34.830531 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-h7bxb" podStartSLOduration=66.781813808 podStartE2EDuration="1m7.830511183s" totalImagesPullingTime="1.048697375s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:20:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:21:34.828797818 +0000 UTC m=+67.969593930" watchObservedRunningTime="2026-07-15 12:21:34.830511183 +0000 UTC m=+67.971307296" Jul 15 12:21:34.848200 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:34.848156 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rt7x5_4efdeca1-8ed8-43bb-9ec0-d7a54a30e3f9/serve-healthcheck-canary/0.log" Jul 15 12:21:34.997697 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:34.997586 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7g6wj" Jul 15 12:21:35.153865 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:35.153783 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7g6wj"] Jul 15 12:21:35.158447 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:21:35.158422 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc883a479_e3e8_45b1_bcc3_027f8637183e.slice/crio-1af05cf00ff46081009fa1db07a01928c395d3e9e53939b96977734a6a4f0676 WatchSource:0}: Error finding container 1af05cf00ff46081009fa1db07a01928c395d3e9e53939b96977734a6a4f0676: Status 404 returned error can't find the container with id 1af05cf00ff46081009fa1db07a01928c395d3e9e53939b96977734a6a4f0676 Jul 15 12:21:35.817387 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:35.817306 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7g6wj" event={"ID":"c883a479-e3e8-45b1-bcc3-027f8637183e","Type":"ContainerStarted","Data":"1af05cf00ff46081009fa1db07a01928c395d3e9e53939b96977734a6a4f0676"} Jul 15 12:21:41.774827 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:41.774747 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5dff78dbc9-kf6kz" Jul 15 12:21:41.842098 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:41.841845 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-c94x9" event={"ID":"b8db43c0-4eee-4627-b30d-53206ad7666a","Type":"ContainerStarted","Data":"9235d5296e08a6e4532e7a26d52b3aa9f43e4cfcc32932650c30df986f0f9d11"} Jul 15 12:21:41.842587 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:41.842561 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:21:41.844906 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:41.844876 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-79bqk" event={"ID":"a919b128-8dca-4cee-81a2-51a65300820c","Type":"ContainerStarted","Data":"1eb2531ac783e8c9d24a25c20911e2b55484795fed07ec7d1712d41790b52c7d"} Jul 15 12:21:41.846690 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:41.846661 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7g6wj" event={"ID":"c883a479-e3e8-45b1-bcc3-027f8637183e","Type":"ContainerStarted","Data":"2b67a492e51cae8e74592196377f39bb3dfcc073f430dcfa30dca74e2f5102c4"} Jul 15 12:21:41.876707 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:41.876655 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-c94x9" podStartSLOduration=67.394713427 podStartE2EDuration="1m14.876621338s" totalImagesPullingTime="7.481907911s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:20:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:21:41.874690208 +0000 UTC m=+75.015486347" watchObservedRunningTime="2026-07-15 12:21:41.876621338 +0000 UTC m=+75.017417449" Jul 15 12:21:41.902463 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:41.902416 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-79bqk" podStartSLOduration=67.058105794 podStartE2EDuration="1m14.902396993s" totalImagesPullingTime="7.844291199s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:20:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:21:41.901553713 +0000 UTC m=+75.042349825" watchObservedRunningTime="2026-07-15 12:21:41.902396993 +0000 UTC m=+75.043193106" Jul 15 12:21:44.858886 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:44.858850 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7g6wj" event={"ID":"c883a479-e3e8-45b1-bcc3-027f8637183e","Type":"ContainerStarted","Data":"954e7b0fdd076252e0e6af4989539f87e9e6eeff9e8ef240b94fc6c18b3f2947"} Jul 15 12:21:48.874406 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:48.874369 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7g6wj" event={"ID":"c883a479-e3e8-45b1-bcc3-027f8637183e","Type":"ContainerStarted","Data":"529ae9d1909e725c1729a9c8c643cebe114fdd3aa03b124680e3fc6e0ed576eb"} Jul 15 12:21:48.899497 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:48.899446 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7g6wj" podStartSLOduration=5.141328798 podStartE2EDuration="17.899429142s" totalImagesPullingTime="12.758100344s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:21:31 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:21:48.898427722 +0000 UTC m=+82.039223832" watchObservedRunningTime="2026-07-15 12:21:48.899429142 +0000 UTC m=+82.040225254" Jul 15 12:21:55.739076 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.739039 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-qr5rr"] Jul 15 12:21:55.742245 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.742226 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-qr5rr" Jul 15 12:21:55.745823 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.745799 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-v2m22\"" Jul 15 12:21:55.745946 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.745800 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 15 12:21:55.745946 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.745894 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 15 12:21:55.746090 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.746076 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 15 12:21:55.746261 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.746241 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 15 12:21:55.747551 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.747527 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 15 12:21:55.748477 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.748462 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-96pbm"] Jul 15 12:21:55.750618 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.750596 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-96pbm" Jul 15 12:21:55.751542 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.751524 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-vrglv"] Jul 15 12:21:55.753533 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.753517 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-vrglv" Jul 15 12:21:55.753714 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.753694 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-g745z\"" Jul 15 12:21:55.753714 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.753706 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 15 12:21:55.753849 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.753711 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 15 12:21:55.754752 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.754736 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 15 12:21:55.757265 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.757246 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 15 12:21:55.757349 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.757267 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-w7d2b\"" Jul 15 12:21:55.757349 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.757323 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 15 12:21:55.757458 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.757404 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 15 12:21:55.779224 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.779206 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-qr5rr"] Jul 15 12:21:55.785837 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.785815 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-vrglv"] Jul 15 12:21:55.911541 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.911510 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/268eb654-ee5f-408e-bf43-7476e962fdf4-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-qr5rr\" (UID: \"268eb654-ee5f-408e-bf43-7476e962fdf4\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-qr5rr" Jul 15 12:21:55.911541 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.911544 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cad0427d-526a-408e-8735-9dd31593821e-root\") pod \"node-exporter-96pbm\" (UID: \"cad0427d-526a-408e-8735-9dd31593821e\") " pod="openshift-monitoring/node-exporter-96pbm" Jul 15 12:21:55.911789 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.911572 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a104d556-6b6d-4d2c-b54d-5c4567fa05cf-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-vrglv\" (UID: \"a104d556-6b6d-4d2c-b54d-5c4567fa05cf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vrglv" Jul 15 12:21:55.911789 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.911594 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a104d556-6b6d-4d2c-b54d-5c4567fa05cf-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-vrglv\" (UID: \"a104d556-6b6d-4d2c-b54d-5c4567fa05cf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vrglv" Jul 15 12:21:55.911789 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.911668 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/268eb654-ee5f-408e-bf43-7476e962fdf4-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-qr5rr\" (UID: \"268eb654-ee5f-408e-bf43-7476e962fdf4\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-qr5rr" Jul 15 12:21:55.911789 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.911702 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cad0427d-526a-408e-8735-9dd31593821e-node-exporter-textfile\") pod \"node-exporter-96pbm\" (UID: \"cad0427d-526a-408e-8735-9dd31593821e\") " pod="openshift-monitoring/node-exporter-96pbm" Jul 15 12:21:55.911789 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.911722 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cad0427d-526a-408e-8735-9dd31593821e-metrics-client-ca\") pod \"node-exporter-96pbm\" (UID: \"cad0427d-526a-408e-8735-9dd31593821e\") " pod="openshift-monitoring/node-exporter-96pbm" Jul 15 12:21:55.911789 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.911739 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/268eb654-ee5f-408e-bf43-7476e962fdf4-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-qr5rr\" (UID: \"268eb654-ee5f-408e-bf43-7476e962fdf4\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-qr5rr" Jul 15 12:21:55.911989 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.911799 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a104d556-6b6d-4d2c-b54d-5c4567fa05cf-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-vrglv\" (UID: \"a104d556-6b6d-4d2c-b54d-5c4567fa05cf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vrglv" Jul 15 12:21:55.911989 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.911840 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zssz\" (UniqueName: \"kubernetes.io/projected/cad0427d-526a-408e-8735-9dd31593821e-kube-api-access-4zssz\") pod \"node-exporter-96pbm\" (UID: \"cad0427d-526a-408e-8735-9dd31593821e\") " pod="openshift-monitoring/node-exporter-96pbm" Jul 15 12:21:55.911989 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.911865 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cad0427d-526a-408e-8735-9dd31593821e-sys\") pod \"node-exporter-96pbm\" (UID: \"cad0427d-526a-408e-8735-9dd31593821e\") " pod="openshift-monitoring/node-exporter-96pbm" Jul 15 12:21:55.911989 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.911894 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a104d556-6b6d-4d2c-b54d-5c4567fa05cf-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-vrglv\" (UID: \"a104d556-6b6d-4d2c-b54d-5c4567fa05cf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vrglv" Jul 15 12:21:55.911989 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.911922 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cad0427d-526a-408e-8735-9dd31593821e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-96pbm\" (UID: \"cad0427d-526a-408e-8735-9dd31593821e\") " pod="openshift-monitoring/node-exporter-96pbm" Jul 15 12:21:55.911989 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.911942 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cad0427d-526a-408e-8735-9dd31593821e-node-exporter-wtmp\") pod \"node-exporter-96pbm\" (UID: \"cad0427d-526a-408e-8735-9dd31593821e\") " pod="openshift-monitoring/node-exporter-96pbm" Jul 15 12:21:55.911989 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.911968 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cad0427d-526a-408e-8735-9dd31593821e-node-exporter-tls\") pod \"node-exporter-96pbm\" (UID: \"cad0427d-526a-408e-8735-9dd31593821e\") " pod="openshift-monitoring/node-exporter-96pbm" Jul 15 12:21:55.912215 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.911995 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a104d556-6b6d-4d2c-b54d-5c4567fa05cf-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-vrglv\" (UID: \"a104d556-6b6d-4d2c-b54d-5c4567fa05cf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vrglv" Jul 15 12:21:55.912215 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.912011 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns5wd\" (UniqueName: \"kubernetes.io/projected/a104d556-6b6d-4d2c-b54d-5c4567fa05cf-kube-api-access-ns5wd\") pod \"kube-state-metrics-785778ff74-vrglv\" (UID: \"a104d556-6b6d-4d2c-b54d-5c4567fa05cf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vrglv" Jul 15 12:21:55.912215 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.912026 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cad0427d-526a-408e-8735-9dd31593821e-node-exporter-accelerators-collector-config\") pod \"node-exporter-96pbm\" (UID: \"cad0427d-526a-408e-8735-9dd31593821e\") " pod="openshift-monitoring/node-exporter-96pbm" Jul 15 12:21:55.912215 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:55.912047 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7zsx\" (UniqueName: \"kubernetes.io/projected/268eb654-ee5f-408e-bf43-7476e962fdf4-kube-api-access-t7zsx\") pod \"openshift-state-metrics-84bd68b896-qr5rr\" (UID: \"268eb654-ee5f-408e-bf43-7476e962fdf4\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-qr5rr" Jul 15 12:21:56.013153 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.013059 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a104d556-6b6d-4d2c-b54d-5c4567fa05cf-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-vrglv\" (UID: \"a104d556-6b6d-4d2c-b54d-5c4567fa05cf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vrglv" Jul 15 12:21:56.013153 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.013098 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zssz\" (UniqueName: \"kubernetes.io/projected/cad0427d-526a-408e-8735-9dd31593821e-kube-api-access-4zssz\") pod \"node-exporter-96pbm\" (UID: \"cad0427d-526a-408e-8735-9dd31593821e\") " pod="openshift-monitoring/node-exporter-96pbm" Jul 15 12:21:56.013153 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.013123 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cad0427d-526a-408e-8735-9dd31593821e-sys\") pod \"node-exporter-96pbm\" (UID: \"cad0427d-526a-408e-8735-9dd31593821e\") " pod="openshift-monitoring/node-exporter-96pbm" Jul 15 12:21:56.013153 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.013147 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a104d556-6b6d-4d2c-b54d-5c4567fa05cf-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-vrglv\" (UID: \"a104d556-6b6d-4d2c-b54d-5c4567fa05cf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vrglv" Jul 15 12:21:56.013477 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.013168 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cad0427d-526a-408e-8735-9dd31593821e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-96pbm\" (UID: \"cad0427d-526a-408e-8735-9dd31593821e\") " pod="openshift-monitoring/node-exporter-96pbm" Jul 15 12:21:56.013477 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.013187 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cad0427d-526a-408e-8735-9dd31593821e-node-exporter-wtmp\") pod \"node-exporter-96pbm\" (UID: \"cad0427d-526a-408e-8735-9dd31593821e\") " pod="openshift-monitoring/node-exporter-96pbm" Jul 15 12:21:56.013477 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.013212 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cad0427d-526a-408e-8735-9dd31593821e-node-exporter-tls\") pod \"node-exporter-96pbm\" (UID: \"cad0427d-526a-408e-8735-9dd31593821e\") " pod="openshift-monitoring/node-exporter-96pbm" Jul 15 12:21:56.013477 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.013273 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cad0427d-526a-408e-8735-9dd31593821e-sys\") pod \"node-exporter-96pbm\" (UID: \"cad0427d-526a-408e-8735-9dd31593821e\") " pod="openshift-monitoring/node-exporter-96pbm" Jul 15 12:21:56.013477 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.013324 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a104d556-6b6d-4d2c-b54d-5c4567fa05cf-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-vrglv\" (UID: \"a104d556-6b6d-4d2c-b54d-5c4567fa05cf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vrglv" Jul 15 12:21:56.013477 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.013357 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns5wd\" (UniqueName: \"kubernetes.io/projected/a104d556-6b6d-4d2c-b54d-5c4567fa05cf-kube-api-access-ns5wd\") pod \"kube-state-metrics-785778ff74-vrglv\" (UID: \"a104d556-6b6d-4d2c-b54d-5c4567fa05cf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vrglv" Jul 15 12:21:56.013477 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.013379 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cad0427d-526a-408e-8735-9dd31593821e-node-exporter-wtmp\") pod \"node-exporter-96pbm\" (UID: \"cad0427d-526a-408e-8735-9dd31593821e\") " pod="openshift-monitoring/node-exporter-96pbm" Jul 15 12:21:56.013477 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.013382 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cad0427d-526a-408e-8735-9dd31593821e-node-exporter-accelerators-collector-config\") pod \"node-exporter-96pbm\" (UID: \"cad0427d-526a-408e-8735-9dd31593821e\") " pod="openshift-monitoring/node-exporter-96pbm" Jul 15 12:21:56.013477 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.013422 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7zsx\" (UniqueName: \"kubernetes.io/projected/268eb654-ee5f-408e-bf43-7476e962fdf4-kube-api-access-t7zsx\") pod \"openshift-state-metrics-84bd68b896-qr5rr\" (UID: \"268eb654-ee5f-408e-bf43-7476e962fdf4\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-qr5rr" Jul 15 12:21:56.013477 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.013460 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/268eb654-ee5f-408e-bf43-7476e962fdf4-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-qr5rr\" (UID: \"268eb654-ee5f-408e-bf43-7476e962fdf4\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-qr5rr" Jul 15 12:21:56.013897 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.013493 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cad0427d-526a-408e-8735-9dd31593821e-root\") pod \"node-exporter-96pbm\" (UID: \"cad0427d-526a-408e-8735-9dd31593821e\") " pod="openshift-monitoring/node-exporter-96pbm" Jul 15 12:21:56.013897 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.013521 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a104d556-6b6d-4d2c-b54d-5c4567fa05cf-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-vrglv\" (UID: \"a104d556-6b6d-4d2c-b54d-5c4567fa05cf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vrglv" Jul 15 12:21:56.013897 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.013544 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a104d556-6b6d-4d2c-b54d-5c4567fa05cf-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-vrglv\" (UID: \"a104d556-6b6d-4d2c-b54d-5c4567fa05cf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vrglv" Jul 15 12:21:56.013897 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.013587 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/268eb654-ee5f-408e-bf43-7476e962fdf4-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-qr5rr\" (UID: \"268eb654-ee5f-408e-bf43-7476e962fdf4\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-qr5rr" Jul 15 12:21:56.013897 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.013630 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cad0427d-526a-408e-8735-9dd31593821e-node-exporter-textfile\") pod \"node-exporter-96pbm\" (UID: \"cad0427d-526a-408e-8735-9dd31593821e\") " pod="openshift-monitoring/node-exporter-96pbm" Jul 15 12:21:56.013897 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.013632 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a104d556-6b6d-4d2c-b54d-5c4567fa05cf-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-vrglv\" (UID: \"a104d556-6b6d-4d2c-b54d-5c4567fa05cf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vrglv" Jul 15 12:21:56.013897 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.013678 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cad0427d-526a-408e-8735-9dd31593821e-metrics-client-ca\") pod \"node-exporter-96pbm\" (UID: \"cad0427d-526a-408e-8735-9dd31593821e\") " pod="openshift-monitoring/node-exporter-96pbm" Jul 15 12:21:56.013897 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.013707 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/268eb654-ee5f-408e-bf43-7476e962fdf4-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-qr5rr\" (UID: \"268eb654-ee5f-408e-bf43-7476e962fdf4\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-qr5rr" Jul 15 12:21:56.013897 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.013729 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cad0427d-526a-408e-8735-9dd31593821e-root\") pod \"node-exporter-96pbm\" (UID: \"cad0427d-526a-408e-8735-9dd31593821e\") " pod="openshift-monitoring/node-exporter-96pbm" Jul 15 12:21:56.014320 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.014015 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cad0427d-526a-408e-8735-9dd31593821e-node-exporter-accelerators-collector-config\") pod \"node-exporter-96pbm\" (UID: \"cad0427d-526a-408e-8735-9dd31593821e\") " pod="openshift-monitoring/node-exporter-96pbm" Jul 15 12:21:56.014320 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.014247 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cad0427d-526a-408e-8735-9dd31593821e-metrics-client-ca\") pod \"node-exporter-96pbm\" (UID: \"cad0427d-526a-408e-8735-9dd31593821e\") " pod="openshift-monitoring/node-exporter-96pbm" Jul 15 12:21:56.014552 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.014487 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a104d556-6b6d-4d2c-b54d-5c4567fa05cf-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-vrglv\" (UID: \"a104d556-6b6d-4d2c-b54d-5c4567fa05cf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vrglv" Jul 15 12:21:56.014552 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.014496 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cad0427d-526a-408e-8735-9dd31593821e-node-exporter-textfile\") pod \"node-exporter-96pbm\" (UID: \"cad0427d-526a-408e-8735-9dd31593821e\") " pod="openshift-monitoring/node-exporter-96pbm" Jul 15 12:21:56.014758 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:56.014565 2560 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 15 12:21:56.014758 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.014633 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/268eb654-ee5f-408e-bf43-7476e962fdf4-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-qr5rr\" (UID: \"268eb654-ee5f-408e-bf43-7476e962fdf4\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-qr5rr" Jul 15 12:21:56.014758 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.014564 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a104d556-6b6d-4d2c-b54d-5c4567fa05cf-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-vrglv\" (UID: \"a104d556-6b6d-4d2c-b54d-5c4567fa05cf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vrglv" Jul 15 12:21:56.014758 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:21:56.014708 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a104d556-6b6d-4d2c-b54d-5c4567fa05cf-kube-state-metrics-tls podName:a104d556-6b6d-4d2c-b54d-5c4567fa05cf nodeName:}" failed. No retries permitted until 2026-07-15 12:21:56.51468528 +0000 UTC m=+89.655481385 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/a104d556-6b6d-4d2c-b54d-5c4567fa05cf-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-vrglv" (UID: "a104d556-6b6d-4d2c-b54d-5c4567fa05cf") : secret "kube-state-metrics-tls" not found Jul 15 12:21:56.016059 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.016034 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a104d556-6b6d-4d2c-b54d-5c4567fa05cf-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-vrglv\" (UID: \"a104d556-6b6d-4d2c-b54d-5c4567fa05cf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vrglv" Jul 15 12:21:56.016244 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.016223 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cad0427d-526a-408e-8735-9dd31593821e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-96pbm\" (UID: \"cad0427d-526a-408e-8735-9dd31593821e\") " pod="openshift-monitoring/node-exporter-96pbm" Jul 15 12:21:56.016865 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.016845 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cad0427d-526a-408e-8735-9dd31593821e-node-exporter-tls\") pod \"node-exporter-96pbm\" (UID: \"cad0427d-526a-408e-8735-9dd31593821e\") " pod="openshift-monitoring/node-exporter-96pbm" Jul 15 12:21:56.016865 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.016857 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/268eb654-ee5f-408e-bf43-7476e962fdf4-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-qr5rr\" (UID: \"268eb654-ee5f-408e-bf43-7476e962fdf4\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-qr5rr" Jul 15 12:21:56.016865 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.016860 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/268eb654-ee5f-408e-bf43-7476e962fdf4-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-qr5rr\" (UID: \"268eb654-ee5f-408e-bf43-7476e962fdf4\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-qr5rr" Jul 15 12:21:56.035729 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.035708 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zssz\" (UniqueName: \"kubernetes.io/projected/cad0427d-526a-408e-8735-9dd31593821e-kube-api-access-4zssz\") pod \"node-exporter-96pbm\" (UID: \"cad0427d-526a-408e-8735-9dd31593821e\") " pod="openshift-monitoring/node-exporter-96pbm" Jul 15 12:21:56.037665 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.037626 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns5wd\" (UniqueName: \"kubernetes.io/projected/a104d556-6b6d-4d2c-b54d-5c4567fa05cf-kube-api-access-ns5wd\") pod \"kube-state-metrics-785778ff74-vrglv\" (UID: \"a104d556-6b6d-4d2c-b54d-5c4567fa05cf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vrglv" Jul 15 12:21:56.042399 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.042378 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7zsx\" (UniqueName: \"kubernetes.io/projected/268eb654-ee5f-408e-bf43-7476e962fdf4-kube-api-access-t7zsx\") pod \"openshift-state-metrics-84bd68b896-qr5rr\" (UID: \"268eb654-ee5f-408e-bf43-7476e962fdf4\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-qr5rr" Jul 15 12:21:56.051158 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.051141 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-qr5rr" Jul 15 12:21:56.059728 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.059713 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-96pbm" Jul 15 12:21:56.068199 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:21:56.068176 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcad0427d_526a_408e_8735_9dd31593821e.slice/crio-0dc55d2528301efe9c3257e9d80300a1dcf86c1fa9a34a48a3e6f96f12c0f4ac WatchSource:0}: Error finding container 0dc55d2528301efe9c3257e9d80300a1dcf86c1fa9a34a48a3e6f96f12c0f4ac: Status 404 returned error can't find the container with id 0dc55d2528301efe9c3257e9d80300a1dcf86c1fa9a34a48a3e6f96f12c0f4ac Jul 15 12:21:56.191174 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.191143 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-qr5rr"] Jul 15 12:21:56.194543 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:21:56.194519 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod268eb654_ee5f_408e_bf43_7476e962fdf4.slice/crio-14d7829ff1041d37207cdaf642b6289fe1de13fa3b333f42bd8204b567394a01 WatchSource:0}: Error finding container 14d7829ff1041d37207cdaf642b6289fe1de13fa3b333f42bd8204b567394a01: Status 404 returned error can't find the container with id 14d7829ff1041d37207cdaf642b6289fe1de13fa3b333f42bd8204b567394a01 Jul 15 12:21:56.518270 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.518230 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a104d556-6b6d-4d2c-b54d-5c4567fa05cf-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-vrglv\" (UID: \"a104d556-6b6d-4d2c-b54d-5c4567fa05cf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vrglv" Jul 15 12:21:56.520484 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.520465 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a104d556-6b6d-4d2c-b54d-5c4567fa05cf-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-vrglv\" (UID: \"a104d556-6b6d-4d2c-b54d-5c4567fa05cf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vrglv" Jul 15 12:21:56.664705 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.664675 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-vrglv" Jul 15 12:21:56.793843 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.793802 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-vrglv"] Jul 15 12:21:56.796709 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:21:56.796681 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda104d556_6b6d_4d2c_b54d_5c4567fa05cf.slice/crio-488daf869b7df2b053a21fedb1e7f78da34988852d50c8b4b59dc0d43233726b WatchSource:0}: Error finding container 488daf869b7df2b053a21fedb1e7f78da34988852d50c8b4b59dc0d43233726b: Status 404 returned error can't find the container with id 488daf869b7df2b053a21fedb1e7f78da34988852d50c8b4b59dc0d43233726b Jul 15 12:21:56.900770 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.900735 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-96pbm" event={"ID":"cad0427d-526a-408e-8735-9dd31593821e","Type":"ContainerStarted","Data":"0dc55d2528301efe9c3257e9d80300a1dcf86c1fa9a34a48a3e6f96f12c0f4ac"} Jul 15 12:21:56.901740 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.901713 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-vrglv" event={"ID":"a104d556-6b6d-4d2c-b54d-5c4567fa05cf","Type":"ContainerStarted","Data":"488daf869b7df2b053a21fedb1e7f78da34988852d50c8b4b59dc0d43233726b"} Jul 15 12:21:56.903379 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.903352 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-qr5rr" event={"ID":"268eb654-ee5f-408e-bf43-7476e962fdf4","Type":"ContainerStarted","Data":"849e5b3174ff71cc929b40b78dde3c81af51866e94a620f369a831f8523961fb"} Jul 15 12:21:56.903451 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.903422 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-qr5rr" event={"ID":"268eb654-ee5f-408e-bf43-7476e962fdf4","Type":"ContainerStarted","Data":"a9c74ac388baa16ad0aa6c8cfcbe0ce919d74b17b9a905fa5294f444b95b165c"} Jul 15 12:21:56.903451 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:56.903441 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-qr5rr" event={"ID":"268eb654-ee5f-408e-bf43-7476e962fdf4","Type":"ContainerStarted","Data":"14d7829ff1041d37207cdaf642b6289fe1de13fa3b333f42bd8204b567394a01"} Jul 15 12:21:57.763308 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:57.763219 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-776d57b8cc-mh8ll"] Jul 15 12:21:57.766408 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:57.766378 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" Jul 15 12:21:57.769504 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:57.769308 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 15 12:21:57.770762 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:57.769882 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-qzvtz\"" Jul 15 12:21:57.770762 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:57.770142 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-84e2h85rltfhj\"" Jul 15 12:21:57.770762 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:57.770364 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 15 12:21:57.770762 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:57.770550 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 15 12:21:57.771023 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:57.770806 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 15 12:21:57.771023 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:57.770806 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 15 12:21:57.779431 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:57.779403 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-776d57b8cc-mh8ll"] Jul 15 12:21:57.909902 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:57.909868 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-qr5rr" event={"ID":"268eb654-ee5f-408e-bf43-7476e962fdf4","Type":"ContainerStarted","Data":"255f3120a2b3c094ee42fa4caa6c175d470a07d5f344c111dd7f61779db8116e"} Jul 15 12:21:57.928966 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:57.928932 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38e690a5-da92-4aa4-9d66-b4de8a6748f6-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-776d57b8cc-mh8ll\" (UID: \"38e690a5-da92-4aa4-9d66-b4de8a6748f6\") " pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" Jul 15 12:21:57.929124 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:57.928974 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/38e690a5-da92-4aa4-9d66-b4de8a6748f6-secret-thanos-querier-tls\") pod \"thanos-querier-776d57b8cc-mh8ll\" (UID: \"38e690a5-da92-4aa4-9d66-b4de8a6748f6\") " pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" Jul 15 12:21:57.929124 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:57.929021 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzjqg\" (UniqueName: \"kubernetes.io/projected/38e690a5-da92-4aa4-9d66-b4de8a6748f6-kube-api-access-xzjqg\") pod \"thanos-querier-776d57b8cc-mh8ll\" (UID: \"38e690a5-da92-4aa4-9d66-b4de8a6748f6\") " pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" Jul 15 12:21:57.929124 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:57.929073 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38e690a5-da92-4aa4-9d66-b4de8a6748f6-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-776d57b8cc-mh8ll\" (UID: \"38e690a5-da92-4aa4-9d66-b4de8a6748f6\") " pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" Jul 15 12:21:57.929124 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:57.929093 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/38e690a5-da92-4aa4-9d66-b4de8a6748f6-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-776d57b8cc-mh8ll\" (UID: \"38e690a5-da92-4aa4-9d66-b4de8a6748f6\") " pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" Jul 15 12:21:57.929124 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:57.929114 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/38e690a5-da92-4aa4-9d66-b4de8a6748f6-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-776d57b8cc-mh8ll\" (UID: \"38e690a5-da92-4aa4-9d66-b4de8a6748f6\") " pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" Jul 15 12:21:57.929318 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:57.929138 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/38e690a5-da92-4aa4-9d66-b4de8a6748f6-secret-grpc-tls\") pod \"thanos-querier-776d57b8cc-mh8ll\" (UID: \"38e690a5-da92-4aa4-9d66-b4de8a6748f6\") " pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" Jul 15 12:21:57.929318 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:57.929211 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38e690a5-da92-4aa4-9d66-b4de8a6748f6-metrics-client-ca\") pod \"thanos-querier-776d57b8cc-mh8ll\" (UID: \"38e690a5-da92-4aa4-9d66-b4de8a6748f6\") " pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" Jul 15 12:21:57.942316 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:57.942257 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-qr5rr" podStartSLOduration=1.838992725 podStartE2EDuration="2.942235342s" totalImagesPullingTime="1.103242617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:21:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:21:57.939861508 +0000 UTC m=+91.080657632" watchObservedRunningTime="2026-07-15 12:21:57.942235342 +0000 UTC m=+91.083031455" Jul 15 12:21:58.029668 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:58.029566 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38e690a5-da92-4aa4-9d66-b4de8a6748f6-metrics-client-ca\") pod \"thanos-querier-776d57b8cc-mh8ll\" (UID: \"38e690a5-da92-4aa4-9d66-b4de8a6748f6\") " pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" Jul 15 12:21:58.029836 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:58.029673 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38e690a5-da92-4aa4-9d66-b4de8a6748f6-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-776d57b8cc-mh8ll\" (UID: \"38e690a5-da92-4aa4-9d66-b4de8a6748f6\") " pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" Jul 15 12:21:58.029836 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:58.029707 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/38e690a5-da92-4aa4-9d66-b4de8a6748f6-secret-thanos-querier-tls\") pod \"thanos-querier-776d57b8cc-mh8ll\" (UID: \"38e690a5-da92-4aa4-9d66-b4de8a6748f6\") " pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" Jul 15 12:21:58.029836 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:58.029756 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzjqg\" (UniqueName: \"kubernetes.io/projected/38e690a5-da92-4aa4-9d66-b4de8a6748f6-kube-api-access-xzjqg\") pod \"thanos-querier-776d57b8cc-mh8ll\" (UID: \"38e690a5-da92-4aa4-9d66-b4de8a6748f6\") " pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" Jul 15 12:21:58.029984 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:58.029850 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38e690a5-da92-4aa4-9d66-b4de8a6748f6-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-776d57b8cc-mh8ll\" (UID: \"38e690a5-da92-4aa4-9d66-b4de8a6748f6\") " pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" Jul 15 12:21:58.029984 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:58.029897 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/38e690a5-da92-4aa4-9d66-b4de8a6748f6-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-776d57b8cc-mh8ll\" (UID: \"38e690a5-da92-4aa4-9d66-b4de8a6748f6\") " pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" Jul 15 12:21:58.029984 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:58.029935 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/38e690a5-da92-4aa4-9d66-b4de8a6748f6-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-776d57b8cc-mh8ll\" (UID: \"38e690a5-da92-4aa4-9d66-b4de8a6748f6\") " pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" Jul 15 12:21:58.029984 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:58.029968 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/38e690a5-da92-4aa4-9d66-b4de8a6748f6-secret-grpc-tls\") pod \"thanos-querier-776d57b8cc-mh8ll\" (UID: \"38e690a5-da92-4aa4-9d66-b4de8a6748f6\") " pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" Jul 15 12:21:58.030536 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:58.030496 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38e690a5-da92-4aa4-9d66-b4de8a6748f6-metrics-client-ca\") pod \"thanos-querier-776d57b8cc-mh8ll\" (UID: \"38e690a5-da92-4aa4-9d66-b4de8a6748f6\") " pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" Jul 15 12:21:58.033012 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:58.032986 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/38e690a5-da92-4aa4-9d66-b4de8a6748f6-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-776d57b8cc-mh8ll\" (UID: \"38e690a5-da92-4aa4-9d66-b4de8a6748f6\") " pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" Jul 15 12:21:58.033117 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:58.032994 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/38e690a5-da92-4aa4-9d66-b4de8a6748f6-secret-grpc-tls\") pod \"thanos-querier-776d57b8cc-mh8ll\" (UID: \"38e690a5-da92-4aa4-9d66-b4de8a6748f6\") " pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" Jul 15 12:21:58.033117 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:58.033034 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38e690a5-da92-4aa4-9d66-b4de8a6748f6-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-776d57b8cc-mh8ll\" (UID: \"38e690a5-da92-4aa4-9d66-b4de8a6748f6\") " pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" Jul 15 12:21:58.033277 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:58.033252 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/38e690a5-da92-4aa4-9d66-b4de8a6748f6-secret-thanos-querier-tls\") pod \"thanos-querier-776d57b8cc-mh8ll\" (UID: \"38e690a5-da92-4aa4-9d66-b4de8a6748f6\") " pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" Jul 15 12:21:58.033413 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:58.033358 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38e690a5-da92-4aa4-9d66-b4de8a6748f6-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-776d57b8cc-mh8ll\" (UID: \"38e690a5-da92-4aa4-9d66-b4de8a6748f6\") " pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" Jul 15 12:21:58.033601 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:58.033580 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/38e690a5-da92-4aa4-9d66-b4de8a6748f6-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-776d57b8cc-mh8ll\" (UID: \"38e690a5-da92-4aa4-9d66-b4de8a6748f6\") " pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" Jul 15 12:21:58.048319 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:58.048289 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzjqg\" (UniqueName: \"kubernetes.io/projected/38e690a5-da92-4aa4-9d66-b4de8a6748f6-kube-api-access-xzjqg\") pod \"thanos-querier-776d57b8cc-mh8ll\" (UID: \"38e690a5-da92-4aa4-9d66-b4de8a6748f6\") " pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" Jul 15 12:21:58.078940 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:58.078904 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" Jul 15 12:21:58.658062 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:58.658022 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-776d57b8cc-mh8ll"] Jul 15 12:21:58.696147 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:21:58.696107 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38e690a5_da92_4aa4_9d66_b4de8a6748f6.slice/crio-228d89e6f9c0160864479b22ed947fcdcb7ab640cc0ce99aee62f460d19bcac6 WatchSource:0}: Error finding container 228d89e6f9c0160864479b22ed947fcdcb7ab640cc0ce99aee62f460d19bcac6: Status 404 returned error can't find the container with id 228d89e6f9c0160864479b22ed947fcdcb7ab640cc0ce99aee62f460d19bcac6 Jul 15 12:21:58.917043 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:58.917011 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" event={"ID":"38e690a5-da92-4aa4-9d66-b4de8a6748f6","Type":"ContainerStarted","Data":"228d89e6f9c0160864479b22ed947fcdcb7ab640cc0ce99aee62f460d19bcac6"} Jul 15 12:21:58.918480 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:58.918457 2560 generic.go:356] "Generic (PLEG): container finished" podID="cad0427d-526a-408e-8735-9dd31593821e" containerID="0db0a68e9f788e5aad33a7ca3c1776878d24e4e82a1e6385921184e6b488a1d7" exitCode=0 Jul 15 12:21:58.918589 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:58.918539 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-96pbm" event={"ID":"cad0427d-526a-408e-8735-9dd31593821e","Type":"ContainerDied","Data":"0db0a68e9f788e5aad33a7ca3c1776878d24e4e82a1e6385921184e6b488a1d7"} Jul 15 12:21:58.920964 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:58.920941 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-vrglv" event={"ID":"a104d556-6b6d-4d2c-b54d-5c4567fa05cf","Type":"ContainerStarted","Data":"fb920f3d3be7e574e5993425f1d73f2bcf788a8024158d0fc2d8fca718477db8"} Jul 15 12:21:58.921079 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:58.920994 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-vrglv" event={"ID":"a104d556-6b6d-4d2c-b54d-5c4567fa05cf","Type":"ContainerStarted","Data":"509d9be247a838b82ffda11f99e9bc69762ad5791882aa21104014aed94abace"} Jul 15 12:21:58.921079 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:58.921015 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-vrglv" event={"ID":"a104d556-6b6d-4d2c-b54d-5c4567fa05cf","Type":"ContainerStarted","Data":"d512e01a5656bf4d7acfd948816bcdc8b4112e7e94a93977521a9d411e1cba0e"} Jul 15 12:21:58.972220 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:58.972161 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-vrglv" podStartSLOduration=2.280344442 podStartE2EDuration="3.972138688s" totalImagesPullingTime="1.691794246s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:21:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:21:58.971772896 +0000 UTC m=+92.112569030" watchObservedRunningTime="2026-07-15 12:21:58.972138688 +0000 UTC m=+92.112934799" Jul 15 12:21:59.927920 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:59.927872 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-96pbm" event={"ID":"cad0427d-526a-408e-8735-9dd31593821e","Type":"ContainerStarted","Data":"39d63570f7ae5df3aaa50e221ea36b4977b59c1892d40a68866cf9bcbeb2b7d1"} Jul 15 12:21:59.928382 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:59.927948 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-96pbm" event={"ID":"cad0427d-526a-408e-8735-9dd31593821e","Type":"ContainerStarted","Data":"83e608744c7230aaeffc359e5fc6337e1c7278240a53b48fbcad7b08c52858d0"} Jul 15 12:21:59.950824 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:21:59.950771 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-96pbm" podStartSLOduration=2.452803572 podStartE2EDuration="4.950752718s" totalImagesPullingTime="2.419586454s" totalInitContainerRuntime="78.362692ms" isStatefulPod=true podCreationTimestamp="2026-07-15 12:21:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:21:59.949188998 +0000 UTC m=+93.089985111" watchObservedRunningTime="2026-07-15 12:21:59.950752718 +0000 UTC m=+93.091548854" Jul 15 12:22:00.360945 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:00.360909 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-788f455ccc-8tj2l"] Jul 15 12:22:00.364081 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:00.364060 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-788f455ccc-8tj2l" Jul 15 12:22:00.366698 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:00.366612 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 15 12:22:00.366698 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:00.366672 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-c76u465mdup4s\"" Jul 15 12:22:00.366874 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:00.366708 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 15 12:22:00.367435 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:00.367401 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 15 12:22:00.367897 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:00.367769 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-qnglc\"" Jul 15 12:22:00.367897 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:00.367790 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 15 12:22:00.384654 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:00.384614 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-788f455ccc-8tj2l"] Jul 15 12:22:00.449969 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:00.449933 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d958af7b-382f-499e-86ac-0f7ea1f3c2dd-audit-log\") pod \"metrics-server-788f455ccc-8tj2l\" (UID: \"d958af7b-382f-499e-86ac-0f7ea1f3c2dd\") " pod="openshift-monitoring/metrics-server-788f455ccc-8tj2l" Jul 15 12:22:00.449969 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:00.449970 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d958af7b-382f-499e-86ac-0f7ea1f3c2dd-secret-metrics-server-tls\") pod \"metrics-server-788f455ccc-8tj2l\" (UID: \"d958af7b-382f-499e-86ac-0f7ea1f3c2dd\") " pod="openshift-monitoring/metrics-server-788f455ccc-8tj2l" Jul 15 12:22:00.450157 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:00.450020 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d958af7b-382f-499e-86ac-0f7ea1f3c2dd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-788f455ccc-8tj2l\" (UID: \"d958af7b-382f-499e-86ac-0f7ea1f3c2dd\") " pod="openshift-monitoring/metrics-server-788f455ccc-8tj2l" Jul 15 12:22:00.450157 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:00.450108 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d958af7b-382f-499e-86ac-0f7ea1f3c2dd-client-ca-bundle\") pod \"metrics-server-788f455ccc-8tj2l\" (UID: \"d958af7b-382f-499e-86ac-0f7ea1f3c2dd\") " pod="openshift-monitoring/metrics-server-788f455ccc-8tj2l" Jul 15 12:22:00.450252 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:00.450178 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twsdg\" (UniqueName: \"kubernetes.io/projected/d958af7b-382f-499e-86ac-0f7ea1f3c2dd-kube-api-access-twsdg\") pod \"metrics-server-788f455ccc-8tj2l\" (UID: \"d958af7b-382f-499e-86ac-0f7ea1f3c2dd\") " pod="openshift-monitoring/metrics-server-788f455ccc-8tj2l" Jul 15 12:22:00.450252 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:00.450214 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d958af7b-382f-499e-86ac-0f7ea1f3c2dd-metrics-server-audit-profiles\") pod \"metrics-server-788f455ccc-8tj2l\" (UID: \"d958af7b-382f-499e-86ac-0f7ea1f3c2dd\") " pod="openshift-monitoring/metrics-server-788f455ccc-8tj2l" Jul 15 12:22:00.450252 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:00.450246 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d958af7b-382f-499e-86ac-0f7ea1f3c2dd-secret-metrics-server-client-certs\") pod \"metrics-server-788f455ccc-8tj2l\" (UID: \"d958af7b-382f-499e-86ac-0f7ea1f3c2dd\") " pod="openshift-monitoring/metrics-server-788f455ccc-8tj2l" Jul 15 12:22:00.551272 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:00.551233 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d958af7b-382f-499e-86ac-0f7ea1f3c2dd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-788f455ccc-8tj2l\" (UID: \"d958af7b-382f-499e-86ac-0f7ea1f3c2dd\") " pod="openshift-monitoring/metrics-server-788f455ccc-8tj2l" Jul 15 12:22:00.551454 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:00.551283 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d958af7b-382f-499e-86ac-0f7ea1f3c2dd-client-ca-bundle\") pod \"metrics-server-788f455ccc-8tj2l\" (UID: \"d958af7b-382f-499e-86ac-0f7ea1f3c2dd\") " pod="openshift-monitoring/metrics-server-788f455ccc-8tj2l" Jul 15 12:22:00.551454 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:00.551311 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twsdg\" (UniqueName: \"kubernetes.io/projected/d958af7b-382f-499e-86ac-0f7ea1f3c2dd-kube-api-access-twsdg\") pod \"metrics-server-788f455ccc-8tj2l\" (UID: \"d958af7b-382f-499e-86ac-0f7ea1f3c2dd\") " pod="openshift-monitoring/metrics-server-788f455ccc-8tj2l" Jul 15 12:22:00.551454 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:00.551338 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d958af7b-382f-499e-86ac-0f7ea1f3c2dd-metrics-server-audit-profiles\") pod \"metrics-server-788f455ccc-8tj2l\" (UID: \"d958af7b-382f-499e-86ac-0f7ea1f3c2dd\") " pod="openshift-monitoring/metrics-server-788f455ccc-8tj2l" Jul 15 12:22:00.551454 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:00.551365 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d958af7b-382f-499e-86ac-0f7ea1f3c2dd-secret-metrics-server-client-certs\") pod \"metrics-server-788f455ccc-8tj2l\" (UID: \"d958af7b-382f-499e-86ac-0f7ea1f3c2dd\") " pod="openshift-monitoring/metrics-server-788f455ccc-8tj2l" Jul 15 12:22:00.551454 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:00.551400 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d958af7b-382f-499e-86ac-0f7ea1f3c2dd-audit-log\") pod \"metrics-server-788f455ccc-8tj2l\" (UID: \"d958af7b-382f-499e-86ac-0f7ea1f3c2dd\") " pod="openshift-monitoring/metrics-server-788f455ccc-8tj2l" Jul 15 12:22:00.551454 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:00.551426 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d958af7b-382f-499e-86ac-0f7ea1f3c2dd-secret-metrics-server-tls\") pod \"metrics-server-788f455ccc-8tj2l\" (UID: \"d958af7b-382f-499e-86ac-0f7ea1f3c2dd\") " pod="openshift-monitoring/metrics-server-788f455ccc-8tj2l" Jul 15 12:22:00.551952 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:00.551890 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d958af7b-382f-499e-86ac-0f7ea1f3c2dd-audit-log\") pod \"metrics-server-788f455ccc-8tj2l\" (UID: \"d958af7b-382f-499e-86ac-0f7ea1f3c2dd\") " pod="openshift-monitoring/metrics-server-788f455ccc-8tj2l" Jul 15 12:22:00.552115 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:00.552087 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d958af7b-382f-499e-86ac-0f7ea1f3c2dd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-788f455ccc-8tj2l\" (UID: \"d958af7b-382f-499e-86ac-0f7ea1f3c2dd\") " pod="openshift-monitoring/metrics-server-788f455ccc-8tj2l" Jul 15 12:22:00.552505 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:00.552479 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d958af7b-382f-499e-86ac-0f7ea1f3c2dd-metrics-server-audit-profiles\") pod \"metrics-server-788f455ccc-8tj2l\" (UID: \"d958af7b-382f-499e-86ac-0f7ea1f3c2dd\") " pod="openshift-monitoring/metrics-server-788f455ccc-8tj2l" Jul 15 12:22:00.554325 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:00.554281 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d958af7b-382f-499e-86ac-0f7ea1f3c2dd-client-ca-bundle\") pod \"metrics-server-788f455ccc-8tj2l\" (UID: \"d958af7b-382f-499e-86ac-0f7ea1f3c2dd\") " pod="openshift-monitoring/metrics-server-788f455ccc-8tj2l" Jul 15 12:22:00.554507 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:00.554487 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d958af7b-382f-499e-86ac-0f7ea1f3c2dd-secret-metrics-server-tls\") pod \"metrics-server-788f455ccc-8tj2l\" (UID: \"d958af7b-382f-499e-86ac-0f7ea1f3c2dd\") " pod="openshift-monitoring/metrics-server-788f455ccc-8tj2l" Jul 15 12:22:00.554615 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:00.554596 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d958af7b-382f-499e-86ac-0f7ea1f3c2dd-secret-metrics-server-client-certs\") pod \"metrics-server-788f455ccc-8tj2l\" (UID: \"d958af7b-382f-499e-86ac-0f7ea1f3c2dd\") " pod="openshift-monitoring/metrics-server-788f455ccc-8tj2l" Jul 15 12:22:00.560557 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:00.560530 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twsdg\" (UniqueName: \"kubernetes.io/projected/d958af7b-382f-499e-86ac-0f7ea1f3c2dd-kube-api-access-twsdg\") pod \"metrics-server-788f455ccc-8tj2l\" (UID: \"d958af7b-382f-499e-86ac-0f7ea1f3c2dd\") " pod="openshift-monitoring/metrics-server-788f455ccc-8tj2l" Jul 15 12:22:00.675444 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:00.675349 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-788f455ccc-8tj2l" Jul 15 12:22:00.867811 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:00.867772 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-788f455ccc-8tj2l"] Jul 15 12:22:00.878719 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:22:00.878688 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd958af7b_382f_499e_86ac_0f7ea1f3c2dd.slice/crio-3cc4c3a221afcb8236cb00809b7e8f3110cd29524c3f60aeec124e6c28def0fb WatchSource:0}: Error finding container 3cc4c3a221afcb8236cb00809b7e8f3110cd29524c3f60aeec124e6c28def0fb: Status 404 returned error can't find the container with id 3cc4c3a221afcb8236cb00809b7e8f3110cd29524c3f60aeec124e6c28def0fb Jul 15 12:22:00.933626 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:00.933604 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" event={"ID":"38e690a5-da92-4aa4-9d66-b4de8a6748f6","Type":"ContainerStarted","Data":"da0fef589b9357f430d716af0081a8b9370efb6d9dc4966df1bc356015b1cd89"} Jul 15 12:22:00.933991 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:00.933799 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" event={"ID":"38e690a5-da92-4aa4-9d66-b4de8a6748f6","Type":"ContainerStarted","Data":"3faaabe0306079a9407b90c664e5f04282b25e3a3e7993b244884ac98c534635"} Jul 15 12:22:00.935307 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:00.935279 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-788f455ccc-8tj2l" event={"ID":"d958af7b-382f-499e-86ac-0f7ea1f3c2dd","Type":"ContainerStarted","Data":"3cc4c3a221afcb8236cb00809b7e8f3110cd29524c3f60aeec124e6c28def0fb"} Jul 15 12:22:01.024677 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.024634 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-78c4b465b9-9qbqf"] Jul 15 12:22:01.028016 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.028000 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-78c4b465b9-9qbqf" Jul 15 12:22:01.030806 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.030779 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 15 12:22:01.030910 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.030890 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 15 12:22:01.031110 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.031088 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 15 12:22:01.031110 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.031100 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-krz4k\"" Jul 15 12:22:01.031345 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.031330 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 15 12:22:01.033528 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.033500 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 15 12:22:01.038866 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.038845 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 15 12:22:01.048138 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.048117 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-78c4b465b9-9qbqf"] Jul 15 12:22:01.156831 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.156803 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/498584a2-e27b-48c5-9eb0-14c6245268f2-metrics-client-ca\") pod \"telemeter-client-78c4b465b9-9qbqf\" (UID: \"498584a2-e27b-48c5-9eb0-14c6245268f2\") " pod="openshift-monitoring/telemeter-client-78c4b465b9-9qbqf" Jul 15 12:22:01.156974 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.156842 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/498584a2-e27b-48c5-9eb0-14c6245268f2-telemeter-client-tls\") pod \"telemeter-client-78c4b465b9-9qbqf\" (UID: \"498584a2-e27b-48c5-9eb0-14c6245268f2\") " pod="openshift-monitoring/telemeter-client-78c4b465b9-9qbqf" Jul 15 12:22:01.156974 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.156869 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/498584a2-e27b-48c5-9eb0-14c6245268f2-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-78c4b465b9-9qbqf\" (UID: \"498584a2-e27b-48c5-9eb0-14c6245268f2\") " pod="openshift-monitoring/telemeter-client-78c4b465b9-9qbqf" Jul 15 12:22:01.156974 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.156947 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vm9lw\" (UniqueName: \"kubernetes.io/projected/498584a2-e27b-48c5-9eb0-14c6245268f2-kube-api-access-vm9lw\") pod \"telemeter-client-78c4b465b9-9qbqf\" (UID: \"498584a2-e27b-48c5-9eb0-14c6245268f2\") " pod="openshift-monitoring/telemeter-client-78c4b465b9-9qbqf" Jul 15 12:22:01.157085 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.157000 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/498584a2-e27b-48c5-9eb0-14c6245268f2-secret-telemeter-client\") pod \"telemeter-client-78c4b465b9-9qbqf\" (UID: \"498584a2-e27b-48c5-9eb0-14c6245268f2\") " pod="openshift-monitoring/telemeter-client-78c4b465b9-9qbqf" Jul 15 12:22:01.157085 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.157032 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/498584a2-e27b-48c5-9eb0-14c6245268f2-serving-certs-ca-bundle\") pod \"telemeter-client-78c4b465b9-9qbqf\" (UID: \"498584a2-e27b-48c5-9eb0-14c6245268f2\") " pod="openshift-monitoring/telemeter-client-78c4b465b9-9qbqf" Jul 15 12:22:01.157085 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.157051 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/498584a2-e27b-48c5-9eb0-14c6245268f2-telemeter-trusted-ca-bundle\") pod \"telemeter-client-78c4b465b9-9qbqf\" (UID: \"498584a2-e27b-48c5-9eb0-14c6245268f2\") " pod="openshift-monitoring/telemeter-client-78c4b465b9-9qbqf" Jul 15 12:22:01.157085 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.157074 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/498584a2-e27b-48c5-9eb0-14c6245268f2-federate-client-tls\") pod \"telemeter-client-78c4b465b9-9qbqf\" (UID: \"498584a2-e27b-48c5-9eb0-14c6245268f2\") " pod="openshift-monitoring/telemeter-client-78c4b465b9-9qbqf" Jul 15 12:22:01.257415 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.257352 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/498584a2-e27b-48c5-9eb0-14c6245268f2-federate-client-tls\") pod \"telemeter-client-78c4b465b9-9qbqf\" (UID: \"498584a2-e27b-48c5-9eb0-14c6245268f2\") " pod="openshift-monitoring/telemeter-client-78c4b465b9-9qbqf" Jul 15 12:22:01.257415 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.257390 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/498584a2-e27b-48c5-9eb0-14c6245268f2-metrics-client-ca\") pod \"telemeter-client-78c4b465b9-9qbqf\" (UID: \"498584a2-e27b-48c5-9eb0-14c6245268f2\") " pod="openshift-monitoring/telemeter-client-78c4b465b9-9qbqf" Jul 15 12:22:01.257415 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.257413 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/498584a2-e27b-48c5-9eb0-14c6245268f2-telemeter-client-tls\") pod \"telemeter-client-78c4b465b9-9qbqf\" (UID: \"498584a2-e27b-48c5-9eb0-14c6245268f2\") " pod="openshift-monitoring/telemeter-client-78c4b465b9-9qbqf" Jul 15 12:22:01.257634 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.257438 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/498584a2-e27b-48c5-9eb0-14c6245268f2-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-78c4b465b9-9qbqf\" (UID: \"498584a2-e27b-48c5-9eb0-14c6245268f2\") " pod="openshift-monitoring/telemeter-client-78c4b465b9-9qbqf" Jul 15 12:22:01.257634 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.257460 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vm9lw\" (UniqueName: \"kubernetes.io/projected/498584a2-e27b-48c5-9eb0-14c6245268f2-kube-api-access-vm9lw\") pod \"telemeter-client-78c4b465b9-9qbqf\" (UID: \"498584a2-e27b-48c5-9eb0-14c6245268f2\") " pod="openshift-monitoring/telemeter-client-78c4b465b9-9qbqf" Jul 15 12:22:01.257634 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.257495 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/498584a2-e27b-48c5-9eb0-14c6245268f2-secret-telemeter-client\") pod \"telemeter-client-78c4b465b9-9qbqf\" (UID: \"498584a2-e27b-48c5-9eb0-14c6245268f2\") " pod="openshift-monitoring/telemeter-client-78c4b465b9-9qbqf" Jul 15 12:22:01.257634 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.257521 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/498584a2-e27b-48c5-9eb0-14c6245268f2-serving-certs-ca-bundle\") pod \"telemeter-client-78c4b465b9-9qbqf\" (UID: \"498584a2-e27b-48c5-9eb0-14c6245268f2\") " pod="openshift-monitoring/telemeter-client-78c4b465b9-9qbqf" Jul 15 12:22:01.257634 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.257543 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/498584a2-e27b-48c5-9eb0-14c6245268f2-telemeter-trusted-ca-bundle\") pod \"telemeter-client-78c4b465b9-9qbqf\" (UID: \"498584a2-e27b-48c5-9eb0-14c6245268f2\") " pod="openshift-monitoring/telemeter-client-78c4b465b9-9qbqf" Jul 15 12:22:01.258300 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.258269 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/498584a2-e27b-48c5-9eb0-14c6245268f2-metrics-client-ca\") pod \"telemeter-client-78c4b465b9-9qbqf\" (UID: \"498584a2-e27b-48c5-9eb0-14c6245268f2\") " pod="openshift-monitoring/telemeter-client-78c4b465b9-9qbqf" Jul 15 12:22:01.258426 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.258403 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/498584a2-e27b-48c5-9eb0-14c6245268f2-telemeter-trusted-ca-bundle\") pod \"telemeter-client-78c4b465b9-9qbqf\" (UID: \"498584a2-e27b-48c5-9eb0-14c6245268f2\") " pod="openshift-monitoring/telemeter-client-78c4b465b9-9qbqf" Jul 15 12:22:01.258532 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.258510 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/498584a2-e27b-48c5-9eb0-14c6245268f2-serving-certs-ca-bundle\") pod \"telemeter-client-78c4b465b9-9qbqf\" (UID: \"498584a2-e27b-48c5-9eb0-14c6245268f2\") " pod="openshift-monitoring/telemeter-client-78c4b465b9-9qbqf" Jul 15 12:22:01.259952 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.259930 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/498584a2-e27b-48c5-9eb0-14c6245268f2-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-78c4b465b9-9qbqf\" (UID: \"498584a2-e27b-48c5-9eb0-14c6245268f2\") " pod="openshift-monitoring/telemeter-client-78c4b465b9-9qbqf" Jul 15 12:22:01.260078 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.260056 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/498584a2-e27b-48c5-9eb0-14c6245268f2-federate-client-tls\") pod \"telemeter-client-78c4b465b9-9qbqf\" (UID: \"498584a2-e27b-48c5-9eb0-14c6245268f2\") " pod="openshift-monitoring/telemeter-client-78c4b465b9-9qbqf" Jul 15 12:22:01.260127 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.260093 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/498584a2-e27b-48c5-9eb0-14c6245268f2-telemeter-client-tls\") pod \"telemeter-client-78c4b465b9-9qbqf\" (UID: \"498584a2-e27b-48c5-9eb0-14c6245268f2\") " pod="openshift-monitoring/telemeter-client-78c4b465b9-9qbqf" Jul 15 12:22:01.260127 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.260098 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/498584a2-e27b-48c5-9eb0-14c6245268f2-secret-telemeter-client\") pod \"telemeter-client-78c4b465b9-9qbqf\" (UID: \"498584a2-e27b-48c5-9eb0-14c6245268f2\") " pod="openshift-monitoring/telemeter-client-78c4b465b9-9qbqf" Jul 15 12:22:01.268203 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.268186 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vm9lw\" (UniqueName: \"kubernetes.io/projected/498584a2-e27b-48c5-9eb0-14c6245268f2-kube-api-access-vm9lw\") pod \"telemeter-client-78c4b465b9-9qbqf\" (UID: \"498584a2-e27b-48c5-9eb0-14c6245268f2\") " pod="openshift-monitoring/telemeter-client-78c4b465b9-9qbqf" Jul 15 12:22:01.337266 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.337238 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-78c4b465b9-9qbqf" Jul 15 12:22:01.468844 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.468821 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-78c4b465b9-9qbqf"] Jul 15 12:22:01.471150 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:22:01.471126 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod498584a2_e27b_48c5_9eb0_14c6245268f2.slice/crio-fefdf6e4ade4d0d327c22f6f4e6f467ee026817cedafd6643eac785e395424cf WatchSource:0}: Error finding container fefdf6e4ade4d0d327c22f6f4e6f467ee026817cedafd6643eac785e395424cf: Status 404 returned error can't find the container with id fefdf6e4ade4d0d327c22f6f4e6f467ee026817cedafd6643eac785e395424cf Jul 15 12:22:01.748120 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.748084 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-fbfb49797-rt7xd"] Jul 15 12:22:01.751945 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.751915 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fbfb49797-rt7xd" Jul 15 12:22:01.759620 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.759593 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 15 12:22:01.761434 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.760915 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 15 12:22:01.761434 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.761021 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 15 12:22:01.761434 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.761243 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 15 12:22:01.761434 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.761290 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 15 12:22:01.761434 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.761338 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 15 12:22:01.761768 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.761469 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 15 12:22:01.762991 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.762973 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-w4w5h\"" Jul 15 12:22:01.767655 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.767617 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 15 12:22:01.777038 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.776975 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fbfb49797-rt7xd"] Jul 15 12:22:01.864090 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.864055 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84854dc7-c8c3-4750-8b06-186a1e49c138-oauth-serving-cert\") pod \"console-fbfb49797-rt7xd\" (UID: \"84854dc7-c8c3-4750-8b06-186a1e49c138\") " pod="openshift-console/console-fbfb49797-rt7xd" Jul 15 12:22:01.864276 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.864114 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84854dc7-c8c3-4750-8b06-186a1e49c138-console-serving-cert\") pod \"console-fbfb49797-rt7xd\" (UID: \"84854dc7-c8c3-4750-8b06-186a1e49c138\") " pod="openshift-console/console-fbfb49797-rt7xd" Jul 15 12:22:01.864276 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.864223 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84854dc7-c8c3-4750-8b06-186a1e49c138-service-ca\") pod \"console-fbfb49797-rt7xd\" (UID: \"84854dc7-c8c3-4750-8b06-186a1e49c138\") " pod="openshift-console/console-fbfb49797-rt7xd" Jul 15 12:22:01.864276 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.864259 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84854dc7-c8c3-4750-8b06-186a1e49c138-console-oauth-config\") pod \"console-fbfb49797-rt7xd\" (UID: \"84854dc7-c8c3-4750-8b06-186a1e49c138\") " pod="openshift-console/console-fbfb49797-rt7xd" Jul 15 12:22:01.864444 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.864304 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84854dc7-c8c3-4750-8b06-186a1e49c138-console-config\") pod \"console-fbfb49797-rt7xd\" (UID: \"84854dc7-c8c3-4750-8b06-186a1e49c138\") " pod="openshift-console/console-fbfb49797-rt7xd" Jul 15 12:22:01.864444 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.864337 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84854dc7-c8c3-4750-8b06-186a1e49c138-trusted-ca-bundle\") pod \"console-fbfb49797-rt7xd\" (UID: \"84854dc7-c8c3-4750-8b06-186a1e49c138\") " pod="openshift-console/console-fbfb49797-rt7xd" Jul 15 12:22:01.864444 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.864366 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpqs8\" (UniqueName: \"kubernetes.io/projected/84854dc7-c8c3-4750-8b06-186a1e49c138-kube-api-access-xpqs8\") pod \"console-fbfb49797-rt7xd\" (UID: \"84854dc7-c8c3-4750-8b06-186a1e49c138\") " pod="openshift-console/console-fbfb49797-rt7xd" Jul 15 12:22:01.946818 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.946781 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" event={"ID":"38e690a5-da92-4aa4-9d66-b4de8a6748f6","Type":"ContainerStarted","Data":"e652d8f23bf2693ccde4d0bc4c7a6044c71e2538ec55f485a41895276d7cc9bb"} Jul 15 12:22:01.948217 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.948190 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78c4b465b9-9qbqf" event={"ID":"498584a2-e27b-48c5-9eb0-14c6245268f2","Type":"ContainerStarted","Data":"fefdf6e4ade4d0d327c22f6f4e6f467ee026817cedafd6643eac785e395424cf"} Jul 15 12:22:01.965830 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.965802 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84854dc7-c8c3-4750-8b06-186a1e49c138-service-ca\") pod \"console-fbfb49797-rt7xd\" (UID: \"84854dc7-c8c3-4750-8b06-186a1e49c138\") " pod="openshift-console/console-fbfb49797-rt7xd" Jul 15 12:22:01.965926 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.965844 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84854dc7-c8c3-4750-8b06-186a1e49c138-console-oauth-config\") pod \"console-fbfb49797-rt7xd\" (UID: \"84854dc7-c8c3-4750-8b06-186a1e49c138\") " pod="openshift-console/console-fbfb49797-rt7xd" Jul 15 12:22:01.965926 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.965879 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84854dc7-c8c3-4750-8b06-186a1e49c138-console-config\") pod \"console-fbfb49797-rt7xd\" (UID: \"84854dc7-c8c3-4750-8b06-186a1e49c138\") " pod="openshift-console/console-fbfb49797-rt7xd" Jul 15 12:22:01.965926 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.965905 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84854dc7-c8c3-4750-8b06-186a1e49c138-trusted-ca-bundle\") pod \"console-fbfb49797-rt7xd\" (UID: \"84854dc7-c8c3-4750-8b06-186a1e49c138\") " pod="openshift-console/console-fbfb49797-rt7xd" Jul 15 12:22:01.966093 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.965929 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpqs8\" (UniqueName: \"kubernetes.io/projected/84854dc7-c8c3-4750-8b06-186a1e49c138-kube-api-access-xpqs8\") pod \"console-fbfb49797-rt7xd\" (UID: \"84854dc7-c8c3-4750-8b06-186a1e49c138\") " pod="openshift-console/console-fbfb49797-rt7xd" Jul 15 12:22:01.966093 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.965968 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84854dc7-c8c3-4750-8b06-186a1e49c138-oauth-serving-cert\") pod \"console-fbfb49797-rt7xd\" (UID: \"84854dc7-c8c3-4750-8b06-186a1e49c138\") " pod="openshift-console/console-fbfb49797-rt7xd" Jul 15 12:22:01.966093 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.966009 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84854dc7-c8c3-4750-8b06-186a1e49c138-console-serving-cert\") pod \"console-fbfb49797-rt7xd\" (UID: \"84854dc7-c8c3-4750-8b06-186a1e49c138\") " pod="openshift-console/console-fbfb49797-rt7xd" Jul 15 12:22:01.967058 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.967005 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84854dc7-c8c3-4750-8b06-186a1e49c138-console-config\") pod \"console-fbfb49797-rt7xd\" (UID: \"84854dc7-c8c3-4750-8b06-186a1e49c138\") " pod="openshift-console/console-fbfb49797-rt7xd" Jul 15 12:22:01.967379 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.967302 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84854dc7-c8c3-4750-8b06-186a1e49c138-trusted-ca-bundle\") pod \"console-fbfb49797-rt7xd\" (UID: \"84854dc7-c8c3-4750-8b06-186a1e49c138\") " pod="openshift-console/console-fbfb49797-rt7xd" Jul 15 12:22:01.967804 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.967783 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84854dc7-c8c3-4750-8b06-186a1e49c138-oauth-serving-cert\") pod \"console-fbfb49797-rt7xd\" (UID: \"84854dc7-c8c3-4750-8b06-186a1e49c138\") " pod="openshift-console/console-fbfb49797-rt7xd" Jul 15 12:22:01.967898 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.967863 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84854dc7-c8c3-4750-8b06-186a1e49c138-service-ca\") pod \"console-fbfb49797-rt7xd\" (UID: \"84854dc7-c8c3-4750-8b06-186a1e49c138\") " pod="openshift-console/console-fbfb49797-rt7xd" Jul 15 12:22:01.969123 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.969082 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84854dc7-c8c3-4750-8b06-186a1e49c138-console-serving-cert\") pod \"console-fbfb49797-rt7xd\" (UID: \"84854dc7-c8c3-4750-8b06-186a1e49c138\") " pod="openshift-console/console-fbfb49797-rt7xd" Jul 15 12:22:01.969387 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.969367 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84854dc7-c8c3-4750-8b06-186a1e49c138-console-oauth-config\") pod \"console-fbfb49797-rt7xd\" (UID: \"84854dc7-c8c3-4750-8b06-186a1e49c138\") " pod="openshift-console/console-fbfb49797-rt7xd" Jul 15 12:22:01.996066 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:01.995929 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpqs8\" (UniqueName: \"kubernetes.io/projected/84854dc7-c8c3-4750-8b06-186a1e49c138-kube-api-access-xpqs8\") pod \"console-fbfb49797-rt7xd\" (UID: \"84854dc7-c8c3-4750-8b06-186a1e49c138\") " pod="openshift-console/console-fbfb49797-rt7xd" Jul 15 12:22:02.064704 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:02.064575 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fbfb49797-rt7xd" Jul 15 12:22:02.221391 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:02.221346 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fbfb49797-rt7xd"] Jul 15 12:22:02.528210 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:22:02.528181 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84854dc7_c8c3_4750_8b06_186a1e49c138.slice/crio-9d1f8bde7792d76df9e4d6457745b7f64b8ac41630123ae5d2e61ddbcccc380b WatchSource:0}: Error finding container 9d1f8bde7792d76df9e4d6457745b7f64b8ac41630123ae5d2e61ddbcccc380b: Status 404 returned error can't find the container with id 9d1f8bde7792d76df9e4d6457745b7f64b8ac41630123ae5d2e61ddbcccc380b Jul 15 12:22:02.955770 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:02.955733 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" event={"ID":"38e690a5-da92-4aa4-9d66-b4de8a6748f6","Type":"ContainerStarted","Data":"c372c107e6062700d5a4aed47d7d0423f3c672f1e3b54c01f21579b952dbe800"} Jul 15 12:22:02.956210 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:02.955817 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" event={"ID":"38e690a5-da92-4aa4-9d66-b4de8a6748f6","Type":"ContainerStarted","Data":"9da28a6e2023b7afa5098506ace21a035c7197fc42bee6d0391892f29a08a738"} Jul 15 12:22:02.956210 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:02.955865 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" event={"ID":"38e690a5-da92-4aa4-9d66-b4de8a6748f6","Type":"ContainerStarted","Data":"e53dc995cc01e537a021914bcb45f4cd0673440eeb7aa60fa919b2ca262f2f95"} Jul 15 12:22:02.956210 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:02.955914 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" Jul 15 12:22:02.957382 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:02.957354 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fbfb49797-rt7xd" event={"ID":"84854dc7-c8c3-4750-8b06-186a1e49c138","Type":"ContainerStarted","Data":"9d1f8bde7792d76df9e4d6457745b7f64b8ac41630123ae5d2e61ddbcccc380b"} Jul 15 12:22:02.958790 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:02.958770 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-788f455ccc-8tj2l" event={"ID":"d958af7b-382f-499e-86ac-0f7ea1f3c2dd","Type":"ContainerStarted","Data":"5e50e71a2af3712409d8d70267027bad60dcf9d046fe4f3ef251f996970da24b"} Jul 15 12:22:02.997838 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:02.997780 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" podStartSLOduration=2.9982530179999998 podStartE2EDuration="5.997754177s" totalImagesPullingTime="2.999501159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:21:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:22:02.994384835 +0000 UTC m=+96.135180957" watchObservedRunningTime="2026-07-15 12:22:02.997754177 +0000 UTC m=+96.138550291" Jul 15 12:22:03.964780 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:03.964695 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78c4b465b9-9qbqf" event={"ID":"498584a2-e27b-48c5-9eb0-14c6245268f2","Type":"ContainerStarted","Data":"e234bd56bd6dbdd4d7f540c62bc93b84ab2b76f626874e6b18adf3624dc02f26"} Jul 15 12:22:05.977860 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:05.977824 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78c4b465b9-9qbqf" event={"ID":"498584a2-e27b-48c5-9eb0-14c6245268f2","Type":"ContainerStarted","Data":"63c890d41ccbb5b1f0ea43e2d9df3954db9ec7967892c2b292ec8f4ec1b43ec7"} Jul 15 12:22:05.978310 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:05.977889 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78c4b465b9-9qbqf" event={"ID":"498584a2-e27b-48c5-9eb0-14c6245268f2","Type":"ContainerStarted","Data":"79588569e6d517da2a67e7bb28ee2b8d83c906c505c509173dc7b3d65fb478a8"} Jul 15 12:22:05.979323 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:05.979302 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fbfb49797-rt7xd" event={"ID":"84854dc7-c8c3-4750-8b06-186a1e49c138","Type":"ContainerStarted","Data":"840338f67584b63151c386b2e5854b0ed9f3b00f2762745067d6d3e9f0222521"} Jul 15 12:22:06.010003 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:06.009905 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-788f455ccc-8tj2l" podStartSLOduration=4.318010044 podStartE2EDuration="6.009889029s" totalImagesPullingTime="1.691878985s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:22:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:22:03.024808409 +0000 UTC m=+96.165604547" watchObservedRunningTime="2026-07-15 12:22:06.009889029 +0000 UTC m=+99.150685141" Jul 15 12:22:06.010168 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:06.010118 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-78c4b465b9-9qbqf" podStartSLOduration=1.817288897 podStartE2EDuration="6.010111925s" totalImagesPullingTime="4.192823028s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:22:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:22:06.00910686 +0000 UTC m=+99.149902971" watchObservedRunningTime="2026-07-15 12:22:06.010111925 +0000 UTC m=+99.150908016" Jul 15 12:22:06.033238 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:06.033185 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-fbfb49797-rt7xd" podStartSLOduration=1.820378039 podStartE2EDuration="5.033169772s" totalImagesPullingTime="3.212791733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:22:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:22:06.031662591 +0000 UTC m=+99.172458714" watchObservedRunningTime="2026-07-15 12:22:06.033169772 +0000 UTC m=+99.173965883" Jul 15 12:22:06.878305 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:06.878271 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-fbfb49797-rt7xd"] Jul 15 12:22:08.971218 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:08.971189 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-776d57b8cc-mh8ll" Jul 15 12:22:12.065125 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:12.065096 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-fbfb49797-rt7xd" Jul 15 12:22:13.856087 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:13.856057 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-c94x9" Jul 15 12:22:20.675690 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:20.675630 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-788f455ccc-8tj2l" Jul 15 12:22:20.676078 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:20.675744 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-788f455ccc-8tj2l" Jul 15 12:22:33.005776 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:33.005714 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-fbfb49797-rt7xd" podUID="84854dc7-c8c3-4750-8b06-186a1e49c138" containerName="console" containerID="cri-o://840338f67584b63151c386b2e5854b0ed9f3b00f2762745067d6d3e9f0222521" gracePeriod=15 Jul 15 12:22:33.770053 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:33.770028 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fbfb49797-rt7xd" Jul 15 12:22:33.832617 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:33.832590 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84854dc7-c8c3-4750-8b06-186a1e49c138-service-ca\") pod \"84854dc7-c8c3-4750-8b06-186a1e49c138\" (UID: \"84854dc7-c8c3-4750-8b06-186a1e49c138\") " Jul 15 12:22:33.832831 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:33.832655 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84854dc7-c8c3-4750-8b06-186a1e49c138-trusted-ca-bundle\") pod \"84854dc7-c8c3-4750-8b06-186a1e49c138\" (UID: \"84854dc7-c8c3-4750-8b06-186a1e49c138\") " Jul 15 12:22:33.832831 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:33.832692 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84854dc7-c8c3-4750-8b06-186a1e49c138-console-oauth-config\") pod \"84854dc7-c8c3-4750-8b06-186a1e49c138\" (UID: \"84854dc7-c8c3-4750-8b06-186a1e49c138\") " Jul 15 12:22:33.832831 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:33.832719 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84854dc7-c8c3-4750-8b06-186a1e49c138-console-config\") pod \"84854dc7-c8c3-4750-8b06-186a1e49c138\" (UID: \"84854dc7-c8c3-4750-8b06-186a1e49c138\") " Jul 15 12:22:33.832831 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:33.832743 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xpqs8\" (UniqueName: \"kubernetes.io/projected/84854dc7-c8c3-4750-8b06-186a1e49c138-kube-api-access-xpqs8\") pod \"84854dc7-c8c3-4750-8b06-186a1e49c138\" (UID: \"84854dc7-c8c3-4750-8b06-186a1e49c138\") " Jul 15 12:22:33.832831 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:33.832761 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84854dc7-c8c3-4750-8b06-186a1e49c138-oauth-serving-cert\") pod \"84854dc7-c8c3-4750-8b06-186a1e49c138\" (UID: \"84854dc7-c8c3-4750-8b06-186a1e49c138\") " Jul 15 12:22:33.832831 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:33.832816 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84854dc7-c8c3-4750-8b06-186a1e49c138-console-serving-cert\") pod \"84854dc7-c8c3-4750-8b06-186a1e49c138\" (UID: \"84854dc7-c8c3-4750-8b06-186a1e49c138\") " Jul 15 12:22:33.833157 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:33.833008 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84854dc7-c8c3-4750-8b06-186a1e49c138-service-ca" (OuterVolumeSpecName: "service-ca") pod "84854dc7-c8c3-4750-8b06-186a1e49c138" (UID: "84854dc7-c8c3-4750-8b06-186a1e49c138"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:22:33.833217 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:33.833191 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84854dc7-c8c3-4750-8b06-186a1e49c138-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "84854dc7-c8c3-4750-8b06-186a1e49c138" (UID: "84854dc7-c8c3-4750-8b06-186a1e49c138"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:22:33.833320 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:33.833223 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84854dc7-c8c3-4750-8b06-186a1e49c138-console-config" (OuterVolumeSpecName: "console-config") pod "84854dc7-c8c3-4750-8b06-186a1e49c138" (UID: "84854dc7-c8c3-4750-8b06-186a1e49c138"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:22:33.833411 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:33.833379 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84854dc7-c8c3-4750-8b06-186a1e49c138-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "84854dc7-c8c3-4750-8b06-186a1e49c138" (UID: "84854dc7-c8c3-4750-8b06-186a1e49c138"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:22:33.835170 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:33.835146 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84854dc7-c8c3-4750-8b06-186a1e49c138-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "84854dc7-c8c3-4750-8b06-186a1e49c138" (UID: "84854dc7-c8c3-4750-8b06-186a1e49c138"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:22:33.835285 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:33.835193 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84854dc7-c8c3-4750-8b06-186a1e49c138-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "84854dc7-c8c3-4750-8b06-186a1e49c138" (UID: "84854dc7-c8c3-4750-8b06-186a1e49c138"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:22:33.835285 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:33.835243 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84854dc7-c8c3-4750-8b06-186a1e49c138-kube-api-access-xpqs8" (OuterVolumeSpecName: "kube-api-access-xpqs8") pod "84854dc7-c8c3-4750-8b06-186a1e49c138" (UID: "84854dc7-c8c3-4750-8b06-186a1e49c138"). InnerVolumeSpecName "kube-api-access-xpqs8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:22:33.934203 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:33.934142 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84854dc7-c8c3-4750-8b06-186a1e49c138-console-oauth-config\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:22:33.934203 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:33.934166 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84854dc7-c8c3-4750-8b06-186a1e49c138-console-config\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:22:33.934203 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:33.934175 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xpqs8\" (UniqueName: \"kubernetes.io/projected/84854dc7-c8c3-4750-8b06-186a1e49c138-kube-api-access-xpqs8\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:22:33.934203 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:33.934184 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84854dc7-c8c3-4750-8b06-186a1e49c138-oauth-serving-cert\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:22:33.934203 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:33.934193 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84854dc7-c8c3-4750-8b06-186a1e49c138-console-serving-cert\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:22:33.934203 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:33.934201 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84854dc7-c8c3-4750-8b06-186a1e49c138-service-ca\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:22:33.934448 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:33.934209 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84854dc7-c8c3-4750-8b06-186a1e49c138-trusted-ca-bundle\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:22:34.081212 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:34.081177 2560 generic.go:356] "Generic (PLEG): container finished" podID="84854dc7-c8c3-4750-8b06-186a1e49c138" containerID="840338f67584b63151c386b2e5854b0ed9f3b00f2762745067d6d3e9f0222521" exitCode=0 Jul 15 12:22:34.081571 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:34.081228 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fbfb49797-rt7xd" Jul 15 12:22:34.081571 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:34.081248 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fbfb49797-rt7xd" event={"ID":"84854dc7-c8c3-4750-8b06-186a1e49c138","Type":"ContainerDied","Data":"840338f67584b63151c386b2e5854b0ed9f3b00f2762745067d6d3e9f0222521"} Jul 15 12:22:34.081571 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:34.081296 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fbfb49797-rt7xd" event={"ID":"84854dc7-c8c3-4750-8b06-186a1e49c138","Type":"ContainerDied","Data":"9d1f8bde7792d76df9e4d6457745b7f64b8ac41630123ae5d2e61ddbcccc380b"} Jul 15 12:22:34.081571 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:34.081338 2560 scope.go:117] "RemoveContainer" containerID="840338f67584b63151c386b2e5854b0ed9f3b00f2762745067d6d3e9f0222521" Jul 15 12:22:34.090169 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:34.090152 2560 scope.go:117] "RemoveContainer" containerID="840338f67584b63151c386b2e5854b0ed9f3b00f2762745067d6d3e9f0222521" Jul 15 12:22:34.090425 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:22:34.090405 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"840338f67584b63151c386b2e5854b0ed9f3b00f2762745067d6d3e9f0222521\": container with ID starting with 840338f67584b63151c386b2e5854b0ed9f3b00f2762745067d6d3e9f0222521 not found: ID does not exist" containerID="840338f67584b63151c386b2e5854b0ed9f3b00f2762745067d6d3e9f0222521" Jul 15 12:22:34.090474 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:34.090434 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"840338f67584b63151c386b2e5854b0ed9f3b00f2762745067d6d3e9f0222521"} err="failed to get container status \"840338f67584b63151c386b2e5854b0ed9f3b00f2762745067d6d3e9f0222521\": rpc error: code = NotFound desc = could not find container \"840338f67584b63151c386b2e5854b0ed9f3b00f2762745067d6d3e9f0222521\": container with ID starting with 840338f67584b63151c386b2e5854b0ed9f3b00f2762745067d6d3e9f0222521 not found: ID does not exist" Jul 15 12:22:34.114455 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:34.114428 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-fbfb49797-rt7xd"] Jul 15 12:22:34.126739 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:34.126718 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-fbfb49797-rt7xd"] Jul 15 12:22:35.407153 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:35.407111 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84854dc7-c8c3-4750-8b06-186a1e49c138" path="/var/lib/kubelet/pods/84854dc7-c8c3-4750-8b06-186a1e49c138/volumes" Jul 15 12:22:40.681465 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:40.681436 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-788f455ccc-8tj2l" Jul 15 12:22:40.685179 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:22:40.685131 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-788f455ccc-8tj2l" Jul 15 12:23:18.507916 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.507880 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 12:23:18.508471 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.508323 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84854dc7-c8c3-4750-8b06-186a1e49c138" containerName="console" Jul 15 12:23:18.508471 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.508343 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="84854dc7-c8c3-4750-8b06-186a1e49c138" containerName="console" Jul 15 12:23:18.508471 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.508411 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="84854dc7-c8c3-4750-8b06-186a1e49c138" containerName="console" Jul 15 12:23:18.510834 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.510814 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.515363 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.515344 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 15 12:23:18.515474 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.515432 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 15 12:23:18.515474 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.515461 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 15 12:23:18.516260 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.516222 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 15 12:23:18.516354 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.516300 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 15 12:23:18.516414 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.516355 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 15 12:23:18.516414 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.516362 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 15 12:23:18.516514 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.516411 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 15 12:23:18.516514 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.516454 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-h44t4\"" Jul 15 12:23:18.520776 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.520760 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 15 12:23:18.539129 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.539105 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 12:23:18.578874 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.578843 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/792323f7-96a2-4633-91b8-94a6c9f42e30-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.578874 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.578875 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/792323f7-96a2-4633-91b8-94a6c9f42e30-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.579033 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.578895 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/792323f7-96a2-4633-91b8-94a6c9f42e30-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.579033 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.578926 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/792323f7-96a2-4633-91b8-94a6c9f42e30-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.579033 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.578954 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/792323f7-96a2-4633-91b8-94a6c9f42e30-tls-assets\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.579033 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.578973 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/792323f7-96a2-4633-91b8-94a6c9f42e30-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.579033 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.578995 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/792323f7-96a2-4633-91b8-94a6c9f42e30-config-out\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.579033 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.579020 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/792323f7-96a2-4633-91b8-94a6c9f42e30-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.579211 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.579067 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/792323f7-96a2-4633-91b8-94a6c9f42e30-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.579211 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.579090 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/792323f7-96a2-4633-91b8-94a6c9f42e30-config-volume\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.579211 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.579112 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qkwc\" (UniqueName: \"kubernetes.io/projected/792323f7-96a2-4633-91b8-94a6c9f42e30-kube-api-access-5qkwc\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.579211 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.579137 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/792323f7-96a2-4633-91b8-94a6c9f42e30-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.579211 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.579177 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/792323f7-96a2-4633-91b8-94a6c9f42e30-web-config\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.680451 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.680413 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/792323f7-96a2-4633-91b8-94a6c9f42e30-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.680451 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.680454 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/792323f7-96a2-4633-91b8-94a6c9f42e30-config-volume\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.680699 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.680474 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qkwc\" (UniqueName: \"kubernetes.io/projected/792323f7-96a2-4633-91b8-94a6c9f42e30-kube-api-access-5qkwc\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.680699 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.680492 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/792323f7-96a2-4633-91b8-94a6c9f42e30-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.680699 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.680510 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/792323f7-96a2-4633-91b8-94a6c9f42e30-web-config\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.680699 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.680659 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/792323f7-96a2-4633-91b8-94a6c9f42e30-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.680699 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.680694 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/792323f7-96a2-4633-91b8-94a6c9f42e30-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.680952 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.680718 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/792323f7-96a2-4633-91b8-94a6c9f42e30-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.680952 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.680761 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/792323f7-96a2-4633-91b8-94a6c9f42e30-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.680952 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.680798 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/792323f7-96a2-4633-91b8-94a6c9f42e30-tls-assets\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.680952 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.680827 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/792323f7-96a2-4633-91b8-94a6c9f42e30-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.680952 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.680879 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/792323f7-96a2-4633-91b8-94a6c9f42e30-config-out\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.680952 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.680913 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/792323f7-96a2-4633-91b8-94a6c9f42e30-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.681253 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.680971 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/792323f7-96a2-4633-91b8-94a6c9f42e30-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.681548 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.681527 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/792323f7-96a2-4633-91b8-94a6c9f42e30-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.682490 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.682166 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/792323f7-96a2-4633-91b8-94a6c9f42e30-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.683626 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.683602 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/792323f7-96a2-4633-91b8-94a6c9f42e30-web-config\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.683825 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.683801 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/792323f7-96a2-4633-91b8-94a6c9f42e30-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.683914 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.683884 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/792323f7-96a2-4633-91b8-94a6c9f42e30-config-volume\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.684205 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.684182 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/792323f7-96a2-4633-91b8-94a6c9f42e30-tls-assets\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.684285 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.684218 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/792323f7-96a2-4633-91b8-94a6c9f42e30-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.684336 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.684316 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/792323f7-96a2-4633-91b8-94a6c9f42e30-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.684711 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.684693 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/792323f7-96a2-4633-91b8-94a6c9f42e30-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.685086 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.685062 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/792323f7-96a2-4633-91b8-94a6c9f42e30-config-out\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.685156 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.685143 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/792323f7-96a2-4633-91b8-94a6c9f42e30-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.692905 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.692888 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qkwc\" (UniqueName: \"kubernetes.io/projected/792323f7-96a2-4633-91b8-94a6c9f42e30-kube-api-access-5qkwc\") pod \"alertmanager-main-0\" (UID: \"792323f7-96a2-4633-91b8-94a6c9f42e30\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.820568 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.820538 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:23:18.972606 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:18.972574 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 12:23:18.976294 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:23:18.976252 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod792323f7_96a2_4633_91b8_94a6c9f42e30.slice/crio-83b52cc30f5193cbfb8873f448f2499b76b5f484a590abec627a90546b583ef7 WatchSource:0}: Error finding container 83b52cc30f5193cbfb8873f448f2499b76b5f484a590abec627a90546b583ef7: Status 404 returned error can't find the container with id 83b52cc30f5193cbfb8873f448f2499b76b5f484a590abec627a90546b583ef7 Jul 15 12:23:19.238283 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:19.238152 2560 generic.go:356] "Generic (PLEG): container finished" podID="792323f7-96a2-4633-91b8-94a6c9f42e30" containerID="16bd05f4202e692a94a5cba746bdf7b6e0ea81e93704b18c736dd883ada08c4d" exitCode=0 Jul 15 12:23:19.238283 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:19.238237 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"792323f7-96a2-4633-91b8-94a6c9f42e30","Type":"ContainerDied","Data":"16bd05f4202e692a94a5cba746bdf7b6e0ea81e93704b18c736dd883ada08c4d"} Jul 15 12:23:19.238458 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:19.238312 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"792323f7-96a2-4633-91b8-94a6c9f42e30","Type":"ContainerStarted","Data":"83b52cc30f5193cbfb8873f448f2499b76b5f484a590abec627a90546b583ef7"} Jul 15 12:23:20.750341 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:20.750306 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5dc48cd76c-xmq4w"] Jul 15 12:23:20.753155 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:20.753131 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dc48cd76c-xmq4w" Jul 15 12:23:20.757927 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:20.757909 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 15 12:23:20.781502 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:20.781478 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 15 12:23:20.782244 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:20.782228 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-w4w5h\"" Jul 15 12:23:20.790834 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:20.790804 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 15 12:23:20.790966 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:20.790864 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 15 12:23:20.795983 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:20.794501 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 15 12:23:20.795983 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:20.795793 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 15 12:23:20.796331 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:20.796312 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 15 12:23:20.798969 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:20.798939 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1bb4941f-99e8-4744-9f8e-088413d7372c-console-serving-cert\") pod \"console-5dc48cd76c-xmq4w\" (UID: \"1bb4941f-99e8-4744-9f8e-088413d7372c\") " pod="openshift-console/console-5dc48cd76c-xmq4w" Jul 15 12:23:20.799074 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:20.798984 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vgr9\" (UniqueName: \"kubernetes.io/projected/1bb4941f-99e8-4744-9f8e-088413d7372c-kube-api-access-8vgr9\") pod \"console-5dc48cd76c-xmq4w\" (UID: \"1bb4941f-99e8-4744-9f8e-088413d7372c\") " pod="openshift-console/console-5dc48cd76c-xmq4w" Jul 15 12:23:20.799074 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:20.799040 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1bb4941f-99e8-4744-9f8e-088413d7372c-console-config\") pod \"console-5dc48cd76c-xmq4w\" (UID: \"1bb4941f-99e8-4744-9f8e-088413d7372c\") " pod="openshift-console/console-5dc48cd76c-xmq4w" Jul 15 12:23:20.799189 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:20.799093 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1bb4941f-99e8-4744-9f8e-088413d7372c-oauth-serving-cert\") pod \"console-5dc48cd76c-xmq4w\" (UID: \"1bb4941f-99e8-4744-9f8e-088413d7372c\") " pod="openshift-console/console-5dc48cd76c-xmq4w" Jul 15 12:23:20.799244 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:20.799217 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1bb4941f-99e8-4744-9f8e-088413d7372c-console-oauth-config\") pod \"console-5dc48cd76c-xmq4w\" (UID: \"1bb4941f-99e8-4744-9f8e-088413d7372c\") " pod="openshift-console/console-5dc48cd76c-xmq4w" Jul 15 12:23:20.799330 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:20.799310 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bb4941f-99e8-4744-9f8e-088413d7372c-trusted-ca-bundle\") pod \"console-5dc48cd76c-xmq4w\" (UID: \"1bb4941f-99e8-4744-9f8e-088413d7372c\") " pod="openshift-console/console-5dc48cd76c-xmq4w" Jul 15 12:23:20.799445 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:20.799346 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1bb4941f-99e8-4744-9f8e-088413d7372c-service-ca\") pod \"console-5dc48cd76c-xmq4w\" (UID: \"1bb4941f-99e8-4744-9f8e-088413d7372c\") " pod="openshift-console/console-5dc48cd76c-xmq4w" Jul 15 12:23:20.857355 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:20.857326 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 15 12:23:20.857465 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:20.857360 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5dc48cd76c-xmq4w"] Jul 15 12:23:20.900572 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:20.900548 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1bb4941f-99e8-4744-9f8e-088413d7372c-console-config\") pod \"console-5dc48cd76c-xmq4w\" (UID: \"1bb4941f-99e8-4744-9f8e-088413d7372c\") " pod="openshift-console/console-5dc48cd76c-xmq4w" Jul 15 12:23:20.900696 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:20.900594 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1bb4941f-99e8-4744-9f8e-088413d7372c-oauth-serving-cert\") pod \"console-5dc48cd76c-xmq4w\" (UID: \"1bb4941f-99e8-4744-9f8e-088413d7372c\") " pod="openshift-console/console-5dc48cd76c-xmq4w" Jul 15 12:23:20.900696 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:20.900618 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1bb4941f-99e8-4744-9f8e-088413d7372c-console-oauth-config\") pod \"console-5dc48cd76c-xmq4w\" (UID: \"1bb4941f-99e8-4744-9f8e-088413d7372c\") " pod="openshift-console/console-5dc48cd76c-xmq4w" Jul 15 12:23:20.900815 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:20.900702 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bb4941f-99e8-4744-9f8e-088413d7372c-trusted-ca-bundle\") pod \"console-5dc48cd76c-xmq4w\" (UID: \"1bb4941f-99e8-4744-9f8e-088413d7372c\") " pod="openshift-console/console-5dc48cd76c-xmq4w" Jul 15 12:23:20.900815 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:20.900728 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1bb4941f-99e8-4744-9f8e-088413d7372c-service-ca\") pod \"console-5dc48cd76c-xmq4w\" (UID: \"1bb4941f-99e8-4744-9f8e-088413d7372c\") " pod="openshift-console/console-5dc48cd76c-xmq4w" Jul 15 12:23:20.900815 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:20.900755 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1bb4941f-99e8-4744-9f8e-088413d7372c-console-serving-cert\") pod \"console-5dc48cd76c-xmq4w\" (UID: \"1bb4941f-99e8-4744-9f8e-088413d7372c\") " pod="openshift-console/console-5dc48cd76c-xmq4w" Jul 15 12:23:20.900815 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:20.900780 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vgr9\" (UniqueName: \"kubernetes.io/projected/1bb4941f-99e8-4744-9f8e-088413d7372c-kube-api-access-8vgr9\") pod \"console-5dc48cd76c-xmq4w\" (UID: \"1bb4941f-99e8-4744-9f8e-088413d7372c\") " pod="openshift-console/console-5dc48cd76c-xmq4w" Jul 15 12:23:20.901418 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:20.901393 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1bb4941f-99e8-4744-9f8e-088413d7372c-console-config\") pod \"console-5dc48cd76c-xmq4w\" (UID: \"1bb4941f-99e8-4744-9f8e-088413d7372c\") " pod="openshift-console/console-5dc48cd76c-xmq4w" Jul 15 12:23:20.901524 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:20.901438 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1bb4941f-99e8-4744-9f8e-088413d7372c-oauth-serving-cert\") pod \"console-5dc48cd76c-xmq4w\" (UID: \"1bb4941f-99e8-4744-9f8e-088413d7372c\") " pod="openshift-console/console-5dc48cd76c-xmq4w" Jul 15 12:23:20.901695 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:20.901657 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bb4941f-99e8-4744-9f8e-088413d7372c-trusted-ca-bundle\") pod \"console-5dc48cd76c-xmq4w\" (UID: \"1bb4941f-99e8-4744-9f8e-088413d7372c\") " pod="openshift-console/console-5dc48cd76c-xmq4w" Jul 15 12:23:20.901695 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:20.901657 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1bb4941f-99e8-4744-9f8e-088413d7372c-service-ca\") pod \"console-5dc48cd76c-xmq4w\" (UID: \"1bb4941f-99e8-4744-9f8e-088413d7372c\") " pod="openshift-console/console-5dc48cd76c-xmq4w" Jul 15 12:23:20.902997 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:20.902976 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1bb4941f-99e8-4744-9f8e-088413d7372c-console-oauth-config\") pod \"console-5dc48cd76c-xmq4w\" (UID: \"1bb4941f-99e8-4744-9f8e-088413d7372c\") " pod="openshift-console/console-5dc48cd76c-xmq4w" Jul 15 12:23:20.903223 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:20.903205 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1bb4941f-99e8-4744-9f8e-088413d7372c-console-serving-cert\") pod \"console-5dc48cd76c-xmq4w\" (UID: \"1bb4941f-99e8-4744-9f8e-088413d7372c\") " pod="openshift-console/console-5dc48cd76c-xmq4w" Jul 15 12:23:20.922412 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:20.922390 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vgr9\" (UniqueName: \"kubernetes.io/projected/1bb4941f-99e8-4744-9f8e-088413d7372c-kube-api-access-8vgr9\") pod \"console-5dc48cd76c-xmq4w\" (UID: \"1bb4941f-99e8-4744-9f8e-088413d7372c\") " pod="openshift-console/console-5dc48cd76c-xmq4w" Jul 15 12:23:21.063804 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:21.063780 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dc48cd76c-xmq4w" Jul 15 12:23:21.217799 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:21.217773 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5dc48cd76c-xmq4w"] Jul 15 12:23:21.224058 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:23:21.224033 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bb4941f_99e8_4744_9f8e_088413d7372c.slice/crio-f1c23203eb19f630ba3b8896ef27a29a5c5d60b540eb7cfb99bd2e048e8895a7 WatchSource:0}: Error finding container f1c23203eb19f630ba3b8896ef27a29a5c5d60b540eb7cfb99bd2e048e8895a7: Status 404 returned error can't find the container with id f1c23203eb19f630ba3b8896ef27a29a5c5d60b540eb7cfb99bd2e048e8895a7 Jul 15 12:23:21.249137 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:21.249111 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"792323f7-96a2-4633-91b8-94a6c9f42e30","Type":"ContainerStarted","Data":"e49ad87377ee0a40b123bc9eb38b3946c51c328a1bf44b2a638c975a7aacf4bb"} Jul 15 12:23:21.249255 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:21.249191 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"792323f7-96a2-4633-91b8-94a6c9f42e30","Type":"ContainerStarted","Data":"e9af3a529f34231922e747b9aa5e5c464499d0a202703a358e543d9548780778"} Jul 15 12:23:21.249255 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:21.249236 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"792323f7-96a2-4633-91b8-94a6c9f42e30","Type":"ContainerStarted","Data":"5202772661eb64156be7abd312d678e397d6071756cd01984f2b2a15902b5241"} Jul 15 12:23:21.249337 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:21.249275 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"792323f7-96a2-4633-91b8-94a6c9f42e30","Type":"ContainerStarted","Data":"d7aaa326046a933bd46aac3cd5592b86c0f1b6f184902fd28fd772b47076c749"} Jul 15 12:23:21.249337 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:21.249321 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"792323f7-96a2-4633-91b8-94a6c9f42e30","Type":"ContainerStarted","Data":"100f0876a191e1d8e14a2334dd32ec8245c79bb67f977d9ae93e11f94c2da243"} Jul 15 12:23:21.249427 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:21.249350 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"792323f7-96a2-4633-91b8-94a6c9f42e30","Type":"ContainerStarted","Data":"b71b46a08b29d4bb852e8f60c7e39eda39ccadd3e517264daef3639ccdf50e85"} Jul 15 12:23:21.250384 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:21.250353 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dc48cd76c-xmq4w" event={"ID":"1bb4941f-99e8-4744-9f8e-088413d7372c","Type":"ContainerStarted","Data":"f1c23203eb19f630ba3b8896ef27a29a5c5d60b540eb7cfb99bd2e048e8895a7"} Jul 15 12:23:22.255829 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:22.255795 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dc48cd76c-xmq4w" event={"ID":"1bb4941f-99e8-4744-9f8e-088413d7372c","Type":"ContainerStarted","Data":"b1d18fc25254050a11a3281e844e3f6e6546f509e6531422ba506c31f7e0744f"} Jul 15 12:23:22.283854 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:22.283802 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.661039593 podStartE2EDuration="4.283785586s" totalImagesPullingTime="1.607713615s" totalInitContainerRuntime="15.032378ms" isStatefulPod=false podCreationTimestamp="2026-07-15 12:23:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:23:21.319037103 +0000 UTC m=+174.459833231" watchObservedRunningTime="2026-07-15 12:23:22.283785586 +0000 UTC m=+175.424581710" Jul 15 12:23:22.284215 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:22.284178 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5dc48cd76c-xmq4w" podStartSLOduration=2.284167795 podStartE2EDuration="2.284167795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:23:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:23:22.282773905 +0000 UTC m=+175.423570041" watchObservedRunningTime="2026-07-15 12:23:22.284167795 +0000 UTC m=+175.424963907" Jul 15 12:23:31.064274 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:31.064240 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5dc48cd76c-xmq4w" Jul 15 12:23:31.064758 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:31.064329 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5dc48cd76c-xmq4w" Jul 15 12:23:31.068905 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:31.068885 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5dc48cd76c-xmq4w" Jul 15 12:23:31.292764 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:23:31.292740 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5dc48cd76c-xmq4w" Jul 15 12:24:27.984079 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:27.984046 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wlqkt"] Jul 15 12:24:27.987677 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:27.987660 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wlqkt" Jul 15 12:24:27.991196 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:27.991176 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 15 12:24:27.991438 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:27.991424 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kzj4d\"" Jul 15 12:24:27.991938 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:27.991917 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 15 12:24:28.002162 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:28.002111 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wlqkt"] Jul 15 12:24:28.054064 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:28.054032 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k25wc\" (UniqueName: \"kubernetes.io/projected/e710cc4c-a042-48dd-97da-33ec6086d0a6-kube-api-access-k25wc\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wlqkt\" (UID: \"e710cc4c-a042-48dd-97da-33ec6086d0a6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wlqkt" Jul 15 12:24:28.054064 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:28.054069 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e710cc4c-a042-48dd-97da-33ec6086d0a6-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wlqkt\" (UID: \"e710cc4c-a042-48dd-97da-33ec6086d0a6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wlqkt" Jul 15 12:24:28.054269 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:28.054170 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e710cc4c-a042-48dd-97da-33ec6086d0a6-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wlqkt\" (UID: \"e710cc4c-a042-48dd-97da-33ec6086d0a6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wlqkt" Jul 15 12:24:28.154877 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:28.154840 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e710cc4c-a042-48dd-97da-33ec6086d0a6-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wlqkt\" (UID: \"e710cc4c-a042-48dd-97da-33ec6086d0a6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wlqkt" Jul 15 12:24:28.154877 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:28.154894 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k25wc\" (UniqueName: \"kubernetes.io/projected/e710cc4c-a042-48dd-97da-33ec6086d0a6-kube-api-access-k25wc\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wlqkt\" (UID: \"e710cc4c-a042-48dd-97da-33ec6086d0a6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wlqkt" Jul 15 12:24:28.155139 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:28.154921 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e710cc4c-a042-48dd-97da-33ec6086d0a6-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wlqkt\" (UID: \"e710cc4c-a042-48dd-97da-33ec6086d0a6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wlqkt" Jul 15 12:24:28.155294 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:28.155275 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e710cc4c-a042-48dd-97da-33ec6086d0a6-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wlqkt\" (UID: \"e710cc4c-a042-48dd-97da-33ec6086d0a6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wlqkt" Jul 15 12:24:28.155364 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:28.155321 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e710cc4c-a042-48dd-97da-33ec6086d0a6-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wlqkt\" (UID: \"e710cc4c-a042-48dd-97da-33ec6086d0a6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wlqkt" Jul 15 12:24:28.163581 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:28.163555 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k25wc\" (UniqueName: \"kubernetes.io/projected/e710cc4c-a042-48dd-97da-33ec6086d0a6-kube-api-access-k25wc\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wlqkt\" (UID: \"e710cc4c-a042-48dd-97da-33ec6086d0a6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wlqkt" Jul 15 12:24:28.296272 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:28.296248 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wlqkt" Jul 15 12:24:28.426428 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:28.426394 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wlqkt"] Jul 15 12:24:28.429355 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:24:28.429330 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode710cc4c_a042_48dd_97da_33ec6086d0a6.slice/crio-f40e627494f9a687a573dce9d0ef277846cfefc258c3d576af8df3471ccf4705 WatchSource:0}: Error finding container f40e627494f9a687a573dce9d0ef277846cfefc258c3d576af8df3471ccf4705: Status 404 returned error can't find the container with id f40e627494f9a687a573dce9d0ef277846cfefc258c3d576af8df3471ccf4705 Jul 15 12:24:28.489902 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:28.489870 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wlqkt" event={"ID":"e710cc4c-a042-48dd-97da-33ec6086d0a6","Type":"ContainerStarted","Data":"f40e627494f9a687a573dce9d0ef277846cfefc258c3d576af8df3471ccf4705"} Jul 15 12:24:36.524554 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:36.524523 2560 generic.go:356] "Generic (PLEG): container finished" podID="e710cc4c-a042-48dd-97da-33ec6086d0a6" containerID="c61336c38e9e4268f2af4adb20ace128a27ccace0563f226f2ad50c7c70e625b" exitCode=0 Jul 15 12:24:36.524977 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:36.524570 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wlqkt" event={"ID":"e710cc4c-a042-48dd-97da-33ec6086d0a6","Type":"ContainerDied","Data":"c61336c38e9e4268f2af4adb20ace128a27ccace0563f226f2ad50c7c70e625b"} Jul 15 12:24:39.539521 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:39.539488 2560 generic.go:356] "Generic (PLEG): container finished" podID="e710cc4c-a042-48dd-97da-33ec6086d0a6" containerID="e68dc5d2c8300fea9ab84170cd13fd380b3a3687cf362e7596b8161946e0c365" exitCode=0 Jul 15 12:24:39.539995 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:39.539568 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wlqkt" event={"ID":"e710cc4c-a042-48dd-97da-33ec6086d0a6","Type":"ContainerDied","Data":"e68dc5d2c8300fea9ab84170cd13fd380b3a3687cf362e7596b8161946e0c365"} Jul 15 12:24:46.568925 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:46.568893 2560 generic.go:356] "Generic (PLEG): container finished" podID="e710cc4c-a042-48dd-97da-33ec6086d0a6" containerID="5616a082621861cf472d8938a3d4b7b36086bbf2569df1f69f8bd78a85589a9f" exitCode=0 Jul 15 12:24:46.569303 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:46.568978 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wlqkt" event={"ID":"e710cc4c-a042-48dd-97da-33ec6086d0a6","Type":"ContainerDied","Data":"5616a082621861cf472d8938a3d4b7b36086bbf2569df1f69f8bd78a85589a9f"} Jul 15 12:24:47.693786 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:47.693764 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wlqkt" Jul 15 12:24:47.835219 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:47.835129 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e710cc4c-a042-48dd-97da-33ec6086d0a6-util\") pod \"e710cc4c-a042-48dd-97da-33ec6086d0a6\" (UID: \"e710cc4c-a042-48dd-97da-33ec6086d0a6\") " Jul 15 12:24:47.835219 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:47.835187 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e710cc4c-a042-48dd-97da-33ec6086d0a6-bundle\") pod \"e710cc4c-a042-48dd-97da-33ec6086d0a6\" (UID: \"e710cc4c-a042-48dd-97da-33ec6086d0a6\") " Jul 15 12:24:47.835396 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:47.835267 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k25wc\" (UniqueName: \"kubernetes.io/projected/e710cc4c-a042-48dd-97da-33ec6086d0a6-kube-api-access-k25wc\") pod \"e710cc4c-a042-48dd-97da-33ec6086d0a6\" (UID: \"e710cc4c-a042-48dd-97da-33ec6086d0a6\") " Jul 15 12:24:47.835997 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:47.835963 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e710cc4c-a042-48dd-97da-33ec6086d0a6-bundle" (OuterVolumeSpecName: "bundle") pod "e710cc4c-a042-48dd-97da-33ec6086d0a6" (UID: "e710cc4c-a042-48dd-97da-33ec6086d0a6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:24:47.837378 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:47.837349 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e710cc4c-a042-48dd-97da-33ec6086d0a6-kube-api-access-k25wc" (OuterVolumeSpecName: "kube-api-access-k25wc") pod "e710cc4c-a042-48dd-97da-33ec6086d0a6" (UID: "e710cc4c-a042-48dd-97da-33ec6086d0a6"). InnerVolumeSpecName "kube-api-access-k25wc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:24:47.840289 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:47.840266 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e710cc4c-a042-48dd-97da-33ec6086d0a6-util" (OuterVolumeSpecName: "util") pod "e710cc4c-a042-48dd-97da-33ec6086d0a6" (UID: "e710cc4c-a042-48dd-97da-33ec6086d0a6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:24:47.936683 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:47.936620 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k25wc\" (UniqueName: \"kubernetes.io/projected/e710cc4c-a042-48dd-97da-33ec6086d0a6-kube-api-access-k25wc\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:24:47.936683 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:47.936679 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e710cc4c-a042-48dd-97da-33ec6086d0a6-util\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:24:47.936683 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:47.936694 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e710cc4c-a042-48dd-97da-33ec6086d0a6-bundle\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:24:48.577783 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:48.577754 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wlqkt" Jul 15 12:24:48.577783 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:48.577761 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wlqkt" event={"ID":"e710cc4c-a042-48dd-97da-33ec6086d0a6","Type":"ContainerDied","Data":"f40e627494f9a687a573dce9d0ef277846cfefc258c3d576af8df3471ccf4705"} Jul 15 12:24:48.577974 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:48.577799 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f40e627494f9a687a573dce9d0ef277846cfefc258c3d576af8df3471ccf4705" Jul 15 12:24:56.322918 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:56.322828 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-467qn"] Jul 15 12:24:56.323353 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:56.323269 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e710cc4c-a042-48dd-97da-33ec6086d0a6" containerName="pull" Jul 15 12:24:56.323353 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:56.323288 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e710cc4c-a042-48dd-97da-33ec6086d0a6" containerName="pull" Jul 15 12:24:56.323353 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:56.323304 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e710cc4c-a042-48dd-97da-33ec6086d0a6" containerName="util" Jul 15 12:24:56.323353 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:56.323313 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e710cc4c-a042-48dd-97da-33ec6086d0a6" containerName="util" Jul 15 12:24:56.323353 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:56.323343 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e710cc4c-a042-48dd-97da-33ec6086d0a6" containerName="extract" Jul 15 12:24:56.323353 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:56.323350 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e710cc4c-a042-48dd-97da-33ec6086d0a6" containerName="extract" Jul 15 12:24:56.323630 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:56.323427 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e710cc4c-a042-48dd-97da-33ec6086d0a6" containerName="extract" Jul 15 12:24:56.328096 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:56.328074 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-467qn" Jul 15 12:24:56.330928 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:56.330899 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 15 12:24:56.330928 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:56.330919 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 15 12:24:56.331098 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:56.330935 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-td6bs\"" Jul 15 12:24:56.343710 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:56.343681 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-467qn"] Jul 15 12:24:56.406763 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:56.406728 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mznvl\" (UniqueName: \"kubernetes.io/projected/eb34a420-76c5-42d0-bed0-b9e9056e43a2-kube-api-access-mznvl\") pod \"cert-manager-operator-controller-manager-55c444bf86-467qn\" (UID: \"eb34a420-76c5-42d0-bed0-b9e9056e43a2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-467qn" Jul 15 12:24:56.406920 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:56.406794 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb34a420-76c5-42d0-bed0-b9e9056e43a2-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-467qn\" (UID: \"eb34a420-76c5-42d0-bed0-b9e9056e43a2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-467qn" Jul 15 12:24:56.507807 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:56.507776 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mznvl\" (UniqueName: \"kubernetes.io/projected/eb34a420-76c5-42d0-bed0-b9e9056e43a2-kube-api-access-mznvl\") pod \"cert-manager-operator-controller-manager-55c444bf86-467qn\" (UID: \"eb34a420-76c5-42d0-bed0-b9e9056e43a2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-467qn" Jul 15 12:24:56.507934 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:56.507820 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb34a420-76c5-42d0-bed0-b9e9056e43a2-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-467qn\" (UID: \"eb34a420-76c5-42d0-bed0-b9e9056e43a2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-467qn" Jul 15 12:24:56.508158 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:56.508143 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb34a420-76c5-42d0-bed0-b9e9056e43a2-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-467qn\" (UID: \"eb34a420-76c5-42d0-bed0-b9e9056e43a2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-467qn" Jul 15 12:24:56.521413 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:56.521391 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mznvl\" (UniqueName: \"kubernetes.io/projected/eb34a420-76c5-42d0-bed0-b9e9056e43a2-kube-api-access-mznvl\") pod \"cert-manager-operator-controller-manager-55c444bf86-467qn\" (UID: \"eb34a420-76c5-42d0-bed0-b9e9056e43a2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-467qn" Jul 15 12:24:56.637867 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:56.637786 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-467qn" Jul 15 12:24:56.777625 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:56.777598 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-467qn"] Jul 15 12:24:56.780265 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:24:56.780239 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb34a420_76c5_42d0_bed0_b9e9056e43a2.slice/crio-5d2f85cd74d962acae9c68d554edaaf4f3d30a2c7da8163a3209ed2cbfb6e33a WatchSource:0}: Error finding container 5d2f85cd74d962acae9c68d554edaaf4f3d30a2c7da8163a3209ed2cbfb6e33a: Status 404 returned error can't find the container with id 5d2f85cd74d962acae9c68d554edaaf4f3d30a2c7da8163a3209ed2cbfb6e33a Jul 15 12:24:57.612077 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:57.612043 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-467qn" event={"ID":"eb34a420-76c5-42d0-bed0-b9e9056e43a2","Type":"ContainerStarted","Data":"5d2f85cd74d962acae9c68d554edaaf4f3d30a2c7da8163a3209ed2cbfb6e33a"} Jul 15 12:24:58.618533 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:24:58.618499 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-467qn" event={"ID":"eb34a420-76c5-42d0-bed0-b9e9056e43a2","Type":"ContainerStarted","Data":"73b0f18f422c1577279331e6e4ffbdfe04bba049448780cd294cab79f0879591"} Jul 15 12:25:00.074145 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:00.074090 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-467qn" podStartSLOduration=2.468091988 podStartE2EDuration="4.074072252s" totalImagesPullingTime="1.605980264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:24:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:24:58.665448319 +0000 UTC m=+271.806244430" watchObservedRunningTime="2026-07-15 12:25:00.074072252 +0000 UTC m=+273.214868363" Jul 15 12:25:00.075102 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:00.075079 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmtn2w"] Jul 15 12:25:00.077484 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:00.077466 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmtn2w" Jul 15 12:25:00.099481 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:00.099457 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 15 12:25:00.099576 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:00.099511 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 15 12:25:00.099576 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:00.099516 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kzj4d\"" Jul 15 12:25:00.120519 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:00.120497 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmtn2w"] Jul 15 12:25:00.137399 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:00.137376 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4570ecc4-3baf-4af5-baa2-641755c20f9d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmtn2w\" (UID: \"4570ecc4-3baf-4af5-baa2-641755c20f9d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmtn2w" Jul 15 12:25:00.137495 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:00.137431 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f86mh\" (UniqueName: \"kubernetes.io/projected/4570ecc4-3baf-4af5-baa2-641755c20f9d-kube-api-access-f86mh\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmtn2w\" (UID: \"4570ecc4-3baf-4af5-baa2-641755c20f9d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmtn2w" Jul 15 12:25:00.137551 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:00.137529 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4570ecc4-3baf-4af5-baa2-641755c20f9d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmtn2w\" (UID: \"4570ecc4-3baf-4af5-baa2-641755c20f9d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmtn2w" Jul 15 12:25:00.238052 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:00.238024 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4570ecc4-3baf-4af5-baa2-641755c20f9d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmtn2w\" (UID: \"4570ecc4-3baf-4af5-baa2-641755c20f9d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmtn2w" Jul 15 12:25:00.238245 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:00.238090 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f86mh\" (UniqueName: \"kubernetes.io/projected/4570ecc4-3baf-4af5-baa2-641755c20f9d-kube-api-access-f86mh\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmtn2w\" (UID: \"4570ecc4-3baf-4af5-baa2-641755c20f9d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmtn2w" Jul 15 12:25:00.238245 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:00.238151 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4570ecc4-3baf-4af5-baa2-641755c20f9d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmtn2w\" (UID: \"4570ecc4-3baf-4af5-baa2-641755c20f9d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmtn2w" Jul 15 12:25:00.238439 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:00.238420 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4570ecc4-3baf-4af5-baa2-641755c20f9d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmtn2w\" (UID: \"4570ecc4-3baf-4af5-baa2-641755c20f9d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmtn2w" Jul 15 12:25:00.238505 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:00.238459 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4570ecc4-3baf-4af5-baa2-641755c20f9d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmtn2w\" (UID: \"4570ecc4-3baf-4af5-baa2-641755c20f9d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmtn2w" Jul 15 12:25:00.250880 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:00.250852 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f86mh\" (UniqueName: \"kubernetes.io/projected/4570ecc4-3baf-4af5-baa2-641755c20f9d-kube-api-access-f86mh\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmtn2w\" (UID: \"4570ecc4-3baf-4af5-baa2-641755c20f9d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmtn2w" Jul 15 12:25:00.386450 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:00.386373 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmtn2w" Jul 15 12:25:00.424751 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:00.424717 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-qxps5"] Jul 15 12:25:00.427021 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:00.427001 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-qxps5" Jul 15 12:25:00.430767 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:00.430751 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 15 12:25:00.431000 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:00.430969 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-7czkk\"" Jul 15 12:25:00.431145 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:00.431126 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 15 12:25:00.440593 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:00.440568 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eb45d31f-14b6-4326-8d57-7272a7c72fd9-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-qxps5\" (UID: \"eb45d31f-14b6-4326-8d57-7272a7c72fd9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qxps5" Jul 15 12:25:00.440725 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:00.440601 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khhk5\" (UniqueName: \"kubernetes.io/projected/eb45d31f-14b6-4326-8d57-7272a7c72fd9-kube-api-access-khhk5\") pod \"cert-manager-webhook-6486ffd997-qxps5\" (UID: \"eb45d31f-14b6-4326-8d57-7272a7c72fd9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qxps5" Jul 15 12:25:00.456359 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:00.456332 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-qxps5"] Jul 15 12:25:00.538438 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:00.538415 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmtn2w"] Jul 15 12:25:00.540861 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:25:00.540809 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4570ecc4_3baf_4af5_baa2_641755c20f9d.slice/crio-0976eed93fc646534319c6b1d4396543c251e5897050c68f7e33856916417f90 WatchSource:0}: Error finding container 0976eed93fc646534319c6b1d4396543c251e5897050c68f7e33856916417f90: Status 404 returned error can't find the container with id 0976eed93fc646534319c6b1d4396543c251e5897050c68f7e33856916417f90 Jul 15 12:25:00.541105 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:00.541081 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eb45d31f-14b6-4326-8d57-7272a7c72fd9-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-qxps5\" (UID: \"eb45d31f-14b6-4326-8d57-7272a7c72fd9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qxps5" Jul 15 12:25:00.541167 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:00.541117 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khhk5\" (UniqueName: \"kubernetes.io/projected/eb45d31f-14b6-4326-8d57-7272a7c72fd9-kube-api-access-khhk5\") pod \"cert-manager-webhook-6486ffd997-qxps5\" (UID: \"eb45d31f-14b6-4326-8d57-7272a7c72fd9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qxps5" Jul 15 12:25:00.566310 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:00.566285 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eb45d31f-14b6-4326-8d57-7272a7c72fd9-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-qxps5\" (UID: \"eb45d31f-14b6-4326-8d57-7272a7c72fd9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qxps5" Jul 15 12:25:00.568087 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:00.568064 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khhk5\" (UniqueName: \"kubernetes.io/projected/eb45d31f-14b6-4326-8d57-7272a7c72fd9-kube-api-access-khhk5\") pod \"cert-manager-webhook-6486ffd997-qxps5\" (UID: \"eb45d31f-14b6-4326-8d57-7272a7c72fd9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qxps5" Jul 15 12:25:00.632485 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:00.632458 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmtn2w" event={"ID":"4570ecc4-3baf-4af5-baa2-641755c20f9d","Type":"ContainerStarted","Data":"30741e91508b4ec149c4982c6652c0051783d878f9d2d76c6701fcf718c2d19f"} Jul 15 12:25:00.632607 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:00.632502 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmtn2w" event={"ID":"4570ecc4-3baf-4af5-baa2-641755c20f9d","Type":"ContainerStarted","Data":"0976eed93fc646534319c6b1d4396543c251e5897050c68f7e33856916417f90"} Jul 15 12:25:00.747155 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:00.747071 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-qxps5" Jul 15 12:25:00.873878 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:00.873854 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-qxps5"] Jul 15 12:25:00.876067 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:25:00.876040 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeb45d31f_14b6_4326_8d57_7272a7c72fd9.slice/crio-a9fb3fe4cb568ca2ca74ab9c067c4ef254777c468a932af2b9f17e7fc714296e WatchSource:0}: Error finding container a9fb3fe4cb568ca2ca74ab9c067c4ef254777c468a932af2b9f17e7fc714296e: Status 404 returned error can't find the container with id a9fb3fe4cb568ca2ca74ab9c067c4ef254777c468a932af2b9f17e7fc714296e Jul 15 12:25:01.638292 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:01.638257 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-qxps5" event={"ID":"eb45d31f-14b6-4326-8d57-7272a7c72fd9","Type":"ContainerStarted","Data":"a9fb3fe4cb568ca2ca74ab9c067c4ef254777c468a932af2b9f17e7fc714296e"} Jul 15 12:25:01.639580 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:01.639557 2560 generic.go:356] "Generic (PLEG): container finished" podID="4570ecc4-3baf-4af5-baa2-641755c20f9d" containerID="30741e91508b4ec149c4982c6652c0051783d878f9d2d76c6701fcf718c2d19f" exitCode=0 Jul 15 12:25:01.639703 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:01.639593 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmtn2w" event={"ID":"4570ecc4-3baf-4af5-baa2-641755c20f9d","Type":"ContainerDied","Data":"30741e91508b4ec149c4982c6652c0051783d878f9d2d76c6701fcf718c2d19f"} Jul 15 12:25:04.654956 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:04.654909 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-qxps5" event={"ID":"eb45d31f-14b6-4326-8d57-7272a7c72fd9","Type":"ContainerStarted","Data":"636ec565968640e60010ba4f64b1d36a3b66e58183afdf61de70eea64166e01f"} Jul 15 12:25:04.655377 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:04.655045 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-qxps5" Jul 15 12:25:04.693904 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:04.693851 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-qxps5" podStartSLOduration=1.908120743 podStartE2EDuration="4.693831374s" totalImagesPullingTime="2.785710631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:25:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:25:04.693269978 +0000 UTC m=+277.834066089" watchObservedRunningTime="2026-07-15 12:25:04.693831374 +0000 UTC m=+277.834627486" Jul 15 12:25:05.660491 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:05.660455 2560 generic.go:356] "Generic (PLEG): container finished" podID="4570ecc4-3baf-4af5-baa2-641755c20f9d" containerID="0b9bcaccf5cc9bb81e5dc6e47d2a91a610a542be63f0bb7b62ccf8359b3b52eb" exitCode=0 Jul 15 12:25:05.660880 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:05.660545 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmtn2w" event={"ID":"4570ecc4-3baf-4af5-baa2-641755c20f9d","Type":"ContainerDied","Data":"0b9bcaccf5cc9bb81e5dc6e47d2a91a610a542be63f0bb7b62ccf8359b3b52eb"} Jul 15 12:25:06.666471 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:06.666373 2560 generic.go:356] "Generic (PLEG): container finished" podID="4570ecc4-3baf-4af5-baa2-641755c20f9d" containerID="8f3b9cbdc5d2245d301e95c9af25b3bf65822fa5429b07605681d8b9d1ebe698" exitCode=0 Jul 15 12:25:06.666471 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:06.666444 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmtn2w" event={"ID":"4570ecc4-3baf-4af5-baa2-641755c20f9d","Type":"ContainerDied","Data":"8f3b9cbdc5d2245d301e95c9af25b3bf65822fa5429b07605681d8b9d1ebe698"} Jul 15 12:25:07.801769 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:07.801746 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmtn2w" Jul 15 12:25:07.904410 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:07.904379 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4570ecc4-3baf-4af5-baa2-641755c20f9d-util\") pod \"4570ecc4-3baf-4af5-baa2-641755c20f9d\" (UID: \"4570ecc4-3baf-4af5-baa2-641755c20f9d\") " Jul 15 12:25:07.904539 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:07.904489 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f86mh\" (UniqueName: \"kubernetes.io/projected/4570ecc4-3baf-4af5-baa2-641755c20f9d-kube-api-access-f86mh\") pod \"4570ecc4-3baf-4af5-baa2-641755c20f9d\" (UID: \"4570ecc4-3baf-4af5-baa2-641755c20f9d\") " Jul 15 12:25:07.904539 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:07.904528 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4570ecc4-3baf-4af5-baa2-641755c20f9d-bundle\") pod \"4570ecc4-3baf-4af5-baa2-641755c20f9d\" (UID: \"4570ecc4-3baf-4af5-baa2-641755c20f9d\") " Jul 15 12:25:07.904926 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:07.904904 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4570ecc4-3baf-4af5-baa2-641755c20f9d-bundle" (OuterVolumeSpecName: "bundle") pod "4570ecc4-3baf-4af5-baa2-641755c20f9d" (UID: "4570ecc4-3baf-4af5-baa2-641755c20f9d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:25:07.906544 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:07.906522 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4570ecc4-3baf-4af5-baa2-641755c20f9d-kube-api-access-f86mh" (OuterVolumeSpecName: "kube-api-access-f86mh") pod "4570ecc4-3baf-4af5-baa2-641755c20f9d" (UID: "4570ecc4-3baf-4af5-baa2-641755c20f9d"). InnerVolumeSpecName "kube-api-access-f86mh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:25:07.908648 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:07.908616 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4570ecc4-3baf-4af5-baa2-641755c20f9d-util" (OuterVolumeSpecName: "util") pod "4570ecc4-3baf-4af5-baa2-641755c20f9d" (UID: "4570ecc4-3baf-4af5-baa2-641755c20f9d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:25:08.005879 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:08.005808 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f86mh\" (UniqueName: \"kubernetes.io/projected/4570ecc4-3baf-4af5-baa2-641755c20f9d-kube-api-access-f86mh\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:25:08.005879 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:08.005831 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4570ecc4-3baf-4af5-baa2-641755c20f9d-bundle\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:25:08.005879 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:08.005841 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4570ecc4-3baf-4af5-baa2-641755c20f9d-util\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:25:08.676688 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:08.676658 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmtn2w" Jul 15 12:25:08.676688 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:08.676665 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmtn2w" event={"ID":"4570ecc4-3baf-4af5-baa2-641755c20f9d","Type":"ContainerDied","Data":"0976eed93fc646534319c6b1d4396543c251e5897050c68f7e33856916417f90"} Jul 15 12:25:08.676927 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:08.676709 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0976eed93fc646534319c6b1d4396543c251e5897050c68f7e33856916417f90" Jul 15 12:25:10.663273 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:10.663243 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-qxps5" Jul 15 12:25:15.191376 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:15.191340 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n29vc"] Jul 15 12:25:15.192460 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:15.192431 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4570ecc4-3baf-4af5-baa2-641755c20f9d" containerName="pull" Jul 15 12:25:15.192613 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:15.192601 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4570ecc4-3baf-4af5-baa2-641755c20f9d" containerName="pull" Jul 15 12:25:15.192733 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:15.192721 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4570ecc4-3baf-4af5-baa2-641755c20f9d" containerName="util" Jul 15 12:25:15.192808 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:15.192799 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4570ecc4-3baf-4af5-baa2-641755c20f9d" containerName="util" Jul 15 12:25:15.192877 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:15.192868 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4570ecc4-3baf-4af5-baa2-641755c20f9d" containerName="extract" Jul 15 12:25:15.192965 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:15.192956 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4570ecc4-3baf-4af5-baa2-641755c20f9d" containerName="extract" Jul 15 12:25:15.193209 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:15.193195 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4570ecc4-3baf-4af5-baa2-641755c20f9d" containerName="extract" Jul 15 12:25:15.233475 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:15.233443 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n29vc"] Jul 15 12:25:15.233588 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:15.233531 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n29vc" Jul 15 12:25:15.236035 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:15.236012 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 15 12:25:15.236928 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:15.236912 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-g82sj\"" Jul 15 12:25:15.237015 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:15.236987 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 15 12:25:15.367882 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:15.367853 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzv84\" (UniqueName: \"kubernetes.io/projected/1d8f0d3c-1d10-4039-a30d-76892ed8839b-kube-api-access-xzv84\") pod \"openshift-lws-operator-fd8ccff4c-n29vc\" (UID: \"1d8f0d3c-1d10-4039-a30d-76892ed8839b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n29vc" Jul 15 12:25:15.368028 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:15.367892 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d8f0d3c-1d10-4039-a30d-76892ed8839b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-n29vc\" (UID: \"1d8f0d3c-1d10-4039-a30d-76892ed8839b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n29vc" Jul 15 12:25:15.468845 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:15.468756 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzv84\" (UniqueName: \"kubernetes.io/projected/1d8f0d3c-1d10-4039-a30d-76892ed8839b-kube-api-access-xzv84\") pod \"openshift-lws-operator-fd8ccff4c-n29vc\" (UID: \"1d8f0d3c-1d10-4039-a30d-76892ed8839b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n29vc" Jul 15 12:25:15.468845 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:15.468811 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d8f0d3c-1d10-4039-a30d-76892ed8839b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-n29vc\" (UID: \"1d8f0d3c-1d10-4039-a30d-76892ed8839b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n29vc" Jul 15 12:25:15.469194 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:15.469168 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d8f0d3c-1d10-4039-a30d-76892ed8839b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-n29vc\" (UID: \"1d8f0d3c-1d10-4039-a30d-76892ed8839b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n29vc" Jul 15 12:25:15.477970 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:15.477941 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzv84\" (UniqueName: \"kubernetes.io/projected/1d8f0d3c-1d10-4039-a30d-76892ed8839b-kube-api-access-xzv84\") pod \"openshift-lws-operator-fd8ccff4c-n29vc\" (UID: \"1d8f0d3c-1d10-4039-a30d-76892ed8839b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n29vc" Jul 15 12:25:15.542939 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:15.542908 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n29vc" Jul 15 12:25:15.691016 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:15.690984 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n29vc"] Jul 15 12:25:15.694168 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:25:15.694127 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1d8f0d3c_1d10_4039_a30d_76892ed8839b.slice/crio-05f5d41603084933e1781f9927d77da871b5caa0049a85df0f179b7d2d95d645 WatchSource:0}: Error finding container 05f5d41603084933e1781f9927d77da871b5caa0049a85df0f179b7d2d95d645: Status 404 returned error can't find the container with id 05f5d41603084933e1781f9927d77da871b5caa0049a85df0f179b7d2d95d645 Jul 15 12:25:15.711091 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:15.711057 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n29vc" event={"ID":"1d8f0d3c-1d10-4039-a30d-76892ed8839b","Type":"ContainerStarted","Data":"05f5d41603084933e1781f9927d77da871b5caa0049a85df0f179b7d2d95d645"} Jul 15 12:25:18.726340 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:18.726305 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n29vc" event={"ID":"1d8f0d3c-1d10-4039-a30d-76892ed8839b","Type":"ContainerStarted","Data":"15a70a3b48346a37204e054d3e6a1a825e447b76e9a6ab69db62a8e4c2fccee6"} Jul 15 12:25:18.768994 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:18.768938 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n29vc" podStartSLOduration=1.630117217 podStartE2EDuration="3.768917677s" totalImagesPullingTime="2.13880046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:25:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:25:18.766932427 +0000 UTC m=+291.907728550" watchObservedRunningTime="2026-07-15 12:25:18.768917677 +0000 UTC m=+291.909713791" Jul 15 12:25:21.214655 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:21.214616 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fhhj"] Jul 15 12:25:21.217042 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:21.217024 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fhhj" Jul 15 12:25:21.219765 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:21.219746 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 15 12:25:21.220685 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:21.220666 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kzj4d\"" Jul 15 12:25:21.220795 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:21.220685 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 15 12:25:21.228394 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:21.228371 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fhhj"] Jul 15 12:25:21.316727 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:21.316694 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67kvj\" (UniqueName: \"kubernetes.io/projected/38c8bd4e-fd7f-427e-8813-f1fe56a8f114-kube-api-access-67kvj\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fhhj\" (UID: \"38c8bd4e-fd7f-427e-8813-f1fe56a8f114\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fhhj" Jul 15 12:25:21.316884 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:21.316749 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/38c8bd4e-fd7f-427e-8813-f1fe56a8f114-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fhhj\" (UID: \"38c8bd4e-fd7f-427e-8813-f1fe56a8f114\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fhhj" Jul 15 12:25:21.316884 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:21.316809 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/38c8bd4e-fd7f-427e-8813-f1fe56a8f114-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fhhj\" (UID: \"38c8bd4e-fd7f-427e-8813-f1fe56a8f114\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fhhj" Jul 15 12:25:21.417436 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:21.417398 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67kvj\" (UniqueName: \"kubernetes.io/projected/38c8bd4e-fd7f-427e-8813-f1fe56a8f114-kube-api-access-67kvj\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fhhj\" (UID: \"38c8bd4e-fd7f-427e-8813-f1fe56a8f114\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fhhj" Jul 15 12:25:21.417601 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:21.417455 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/38c8bd4e-fd7f-427e-8813-f1fe56a8f114-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fhhj\" (UID: \"38c8bd4e-fd7f-427e-8813-f1fe56a8f114\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fhhj" Jul 15 12:25:21.417601 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:21.417488 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/38c8bd4e-fd7f-427e-8813-f1fe56a8f114-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fhhj\" (UID: \"38c8bd4e-fd7f-427e-8813-f1fe56a8f114\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fhhj" Jul 15 12:25:21.417837 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:21.417821 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/38c8bd4e-fd7f-427e-8813-f1fe56a8f114-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fhhj\" (UID: \"38c8bd4e-fd7f-427e-8813-f1fe56a8f114\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fhhj" Jul 15 12:25:21.417873 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:21.417851 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/38c8bd4e-fd7f-427e-8813-f1fe56a8f114-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fhhj\" (UID: \"38c8bd4e-fd7f-427e-8813-f1fe56a8f114\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fhhj" Jul 15 12:25:21.428237 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:21.428211 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67kvj\" (UniqueName: \"kubernetes.io/projected/38c8bd4e-fd7f-427e-8813-f1fe56a8f114-kube-api-access-67kvj\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fhhj\" (UID: \"38c8bd4e-fd7f-427e-8813-f1fe56a8f114\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fhhj" Jul 15 12:25:21.525798 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:21.525720 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fhhj" Jul 15 12:25:21.670790 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:21.670767 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fhhj"] Jul 15 12:25:21.673316 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:25:21.673290 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38c8bd4e_fd7f_427e_8813_f1fe56a8f114.slice/crio-5f13af8b8d43f773f6636de5b52080d4d88ba3b7abf3808bb7a01be044c1d445 WatchSource:0}: Error finding container 5f13af8b8d43f773f6636de5b52080d4d88ba3b7abf3808bb7a01be044c1d445: Status 404 returned error can't find the container with id 5f13af8b8d43f773f6636de5b52080d4d88ba3b7abf3808bb7a01be044c1d445 Jul 15 12:25:21.746425 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:21.746391 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fhhj" event={"ID":"38c8bd4e-fd7f-427e-8813-f1fe56a8f114","Type":"ContainerStarted","Data":"0a8785a831fe1aea4caf9d6d06c0e45b1845b15e4522709e26cfb0c60b201ed4"} Jul 15 12:25:21.746520 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:21.746458 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fhhj" event={"ID":"38c8bd4e-fd7f-427e-8813-f1fe56a8f114","Type":"ContainerStarted","Data":"5f13af8b8d43f773f6636de5b52080d4d88ba3b7abf3808bb7a01be044c1d445"} Jul 15 12:25:22.751584 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:22.751500 2560 generic.go:356] "Generic (PLEG): container finished" podID="38c8bd4e-fd7f-427e-8813-f1fe56a8f114" containerID="0a8785a831fe1aea4caf9d6d06c0e45b1845b15e4522709e26cfb0c60b201ed4" exitCode=0 Jul 15 12:25:22.751982 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:22.751586 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fhhj" event={"ID":"38c8bd4e-fd7f-427e-8813-f1fe56a8f114","Type":"ContainerDied","Data":"0a8785a831fe1aea4caf9d6d06c0e45b1845b15e4522709e26cfb0c60b201ed4"} Jul 15 12:25:23.757463 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:23.757426 2560 generic.go:356] "Generic (PLEG): container finished" podID="38c8bd4e-fd7f-427e-8813-f1fe56a8f114" containerID="161adefc3f39581edc8012866b80dca123b21430a72d7a415ae46b004364c733" exitCode=0 Jul 15 12:25:23.757843 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:23.757496 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fhhj" event={"ID":"38c8bd4e-fd7f-427e-8813-f1fe56a8f114","Type":"ContainerDied","Data":"161adefc3f39581edc8012866b80dca123b21430a72d7a415ae46b004364c733"} Jul 15 12:25:24.764750 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:24.764713 2560 generic.go:356] "Generic (PLEG): container finished" podID="38c8bd4e-fd7f-427e-8813-f1fe56a8f114" containerID="40002ecc673557aa510c44a47c19898614554c47c0ab4ff389ea0f028ce7d7fd" exitCode=0 Jul 15 12:25:24.765128 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:24.764794 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fhhj" event={"ID":"38c8bd4e-fd7f-427e-8813-f1fe56a8f114","Type":"ContainerDied","Data":"40002ecc673557aa510c44a47c19898614554c47c0ab4ff389ea0f028ce7d7fd"} Jul 15 12:25:25.887885 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:25.887857 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fhhj" Jul 15 12:25:25.956732 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:25.956705 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/38c8bd4e-fd7f-427e-8813-f1fe56a8f114-bundle\") pod \"38c8bd4e-fd7f-427e-8813-f1fe56a8f114\" (UID: \"38c8bd4e-fd7f-427e-8813-f1fe56a8f114\") " Jul 15 12:25:25.956732 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:25.956736 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/38c8bd4e-fd7f-427e-8813-f1fe56a8f114-util\") pod \"38c8bd4e-fd7f-427e-8813-f1fe56a8f114\" (UID: \"38c8bd4e-fd7f-427e-8813-f1fe56a8f114\") " Jul 15 12:25:25.956916 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:25.956810 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67kvj\" (UniqueName: \"kubernetes.io/projected/38c8bd4e-fd7f-427e-8813-f1fe56a8f114-kube-api-access-67kvj\") pod \"38c8bd4e-fd7f-427e-8813-f1fe56a8f114\" (UID: \"38c8bd4e-fd7f-427e-8813-f1fe56a8f114\") " Jul 15 12:25:25.957455 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:25.957432 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38c8bd4e-fd7f-427e-8813-f1fe56a8f114-bundle" (OuterVolumeSpecName: "bundle") pod "38c8bd4e-fd7f-427e-8813-f1fe56a8f114" (UID: "38c8bd4e-fd7f-427e-8813-f1fe56a8f114"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:25:25.958823 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:25.958798 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38c8bd4e-fd7f-427e-8813-f1fe56a8f114-kube-api-access-67kvj" (OuterVolumeSpecName: "kube-api-access-67kvj") pod "38c8bd4e-fd7f-427e-8813-f1fe56a8f114" (UID: "38c8bd4e-fd7f-427e-8813-f1fe56a8f114"). InnerVolumeSpecName "kube-api-access-67kvj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:25:25.962021 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:25.961985 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38c8bd4e-fd7f-427e-8813-f1fe56a8f114-util" (OuterVolumeSpecName: "util") pod "38c8bd4e-fd7f-427e-8813-f1fe56a8f114" (UID: "38c8bd4e-fd7f-427e-8813-f1fe56a8f114"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:25:26.057863 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:26.057831 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67kvj\" (UniqueName: \"kubernetes.io/projected/38c8bd4e-fd7f-427e-8813-f1fe56a8f114-kube-api-access-67kvj\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:25:26.057863 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:26.057861 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/38c8bd4e-fd7f-427e-8813-f1fe56a8f114-bundle\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:25:26.058028 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:26.057874 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/38c8bd4e-fd7f-427e-8813-f1fe56a8f114-util\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:25:26.775777 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:26.775748 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fhhj" Jul 15 12:25:26.775777 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:26.775759 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fhhj" event={"ID":"38c8bd4e-fd7f-427e-8813-f1fe56a8f114","Type":"ContainerDied","Data":"5f13af8b8d43f773f6636de5b52080d4d88ba3b7abf3808bb7a01be044c1d445"} Jul 15 12:25:26.775986 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:26.775810 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5f13af8b8d43f773f6636de5b52080d4d88ba3b7abf3808bb7a01be044c1d445" Jul 15 12:25:27.252582 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:27.252553 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bg9fm_808669ad-89bb-473a-86e9-7562fcd99736/ovn-acl-logging/0.log" Jul 15 12:25:27.256152 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:27.256133 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bg9fm_808669ad-89bb-473a-86e9-7562fcd99736/ovn-acl-logging/0.log" Jul 15 12:25:27.256967 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:27.256948 2560 kubelet.go:1648] "Image garbage collection succeeded" Jul 15 12:25:37.420658 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:37.420619 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f56ddd455-vzpvx"] Jul 15 12:25:37.422049 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:37.420913 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38c8bd4e-fd7f-427e-8813-f1fe56a8f114" containerName="pull" Jul 15 12:25:37.422049 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:37.420924 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="38c8bd4e-fd7f-427e-8813-f1fe56a8f114" containerName="pull" Jul 15 12:25:37.422049 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:37.420933 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38c8bd4e-fd7f-427e-8813-f1fe56a8f114" containerName="extract" Jul 15 12:25:37.422049 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:37.420938 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="38c8bd4e-fd7f-427e-8813-f1fe56a8f114" containerName="extract" Jul 15 12:25:37.422049 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:37.420957 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38c8bd4e-fd7f-427e-8813-f1fe56a8f114" containerName="util" Jul 15 12:25:37.422049 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:37.420962 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="38c8bd4e-fd7f-427e-8813-f1fe56a8f114" containerName="util" Jul 15 12:25:37.422049 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:37.421012 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="38c8bd4e-fd7f-427e-8813-f1fe56a8f114" containerName="extract" Jul 15 12:25:37.422822 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:37.422807 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-vzpvx" Jul 15 12:25:37.427194 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:37.427173 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 15 12:25:37.427318 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:37.427230 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 15 12:25:37.427318 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:37.427280 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-657ml\"" Jul 15 12:25:37.427595 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:37.427577 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 15 12:25:37.427707 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:37.427691 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 15 12:25:37.501745 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:37.501717 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f56ddd455-vzpvx"] Jul 15 12:25:37.557873 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:37.557848 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a5d0d0db-7435-4204-9255-12e3d3ed844d-tmp\") pod \"opendatahub-operator-controller-manager-5f56ddd455-vzpvx\" (UID: \"a5d0d0db-7435-4204-9255-12e3d3ed844d\") " pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-vzpvx" Jul 15 12:25:37.558002 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:37.557883 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a5d0d0db-7435-4204-9255-12e3d3ed844d-manifests\") pod \"opendatahub-operator-controller-manager-5f56ddd455-vzpvx\" (UID: \"a5d0d0db-7435-4204-9255-12e3d3ed844d\") " pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-vzpvx" Jul 15 12:25:37.558002 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:37.557901 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a5d0d0db-7435-4204-9255-12e3d3ed844d-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f56ddd455-vzpvx\" (UID: \"a5d0d0db-7435-4204-9255-12e3d3ed844d\") " pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-vzpvx" Jul 15 12:25:37.558002 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:37.557983 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a5d0d0db-7435-4204-9255-12e3d3ed844d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f56ddd455-vzpvx\" (UID: \"a5d0d0db-7435-4204-9255-12e3d3ed844d\") " pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-vzpvx" Jul 15 12:25:37.558112 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:37.558007 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqgd9\" (UniqueName: \"kubernetes.io/projected/a5d0d0db-7435-4204-9255-12e3d3ed844d-kube-api-access-lqgd9\") pod \"opendatahub-operator-controller-manager-5f56ddd455-vzpvx\" (UID: \"a5d0d0db-7435-4204-9255-12e3d3ed844d\") " pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-vzpvx" Jul 15 12:25:37.659168 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:37.659131 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a5d0d0db-7435-4204-9255-12e3d3ed844d-tmp\") pod \"opendatahub-operator-controller-manager-5f56ddd455-vzpvx\" (UID: \"a5d0d0db-7435-4204-9255-12e3d3ed844d\") " pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-vzpvx" Jul 15 12:25:37.659362 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:37.659176 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a5d0d0db-7435-4204-9255-12e3d3ed844d-manifests\") pod \"opendatahub-operator-controller-manager-5f56ddd455-vzpvx\" (UID: \"a5d0d0db-7435-4204-9255-12e3d3ed844d\") " pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-vzpvx" Jul 15 12:25:37.659362 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:37.659318 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a5d0d0db-7435-4204-9255-12e3d3ed844d-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f56ddd455-vzpvx\" (UID: \"a5d0d0db-7435-4204-9255-12e3d3ed844d\") " pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-vzpvx" Jul 15 12:25:37.659505 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:37.659487 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a5d0d0db-7435-4204-9255-12e3d3ed844d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f56ddd455-vzpvx\" (UID: \"a5d0d0db-7435-4204-9255-12e3d3ed844d\") " pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-vzpvx" Jul 15 12:25:37.659563 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:37.659527 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqgd9\" (UniqueName: \"kubernetes.io/projected/a5d0d0db-7435-4204-9255-12e3d3ed844d-kube-api-access-lqgd9\") pod \"opendatahub-operator-controller-manager-5f56ddd455-vzpvx\" (UID: \"a5d0d0db-7435-4204-9255-12e3d3ed844d\") " pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-vzpvx" Jul 15 12:25:37.659614 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:37.659565 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a5d0d0db-7435-4204-9255-12e3d3ed844d-manifests\") pod \"opendatahub-operator-controller-manager-5f56ddd455-vzpvx\" (UID: \"a5d0d0db-7435-4204-9255-12e3d3ed844d\") " pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-vzpvx" Jul 15 12:25:37.661526 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:37.661489 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a5d0d0db-7435-4204-9255-12e3d3ed844d-tmp\") pod \"opendatahub-operator-controller-manager-5f56ddd455-vzpvx\" (UID: \"a5d0d0db-7435-4204-9255-12e3d3ed844d\") " pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-vzpvx" Jul 15 12:25:37.661780 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:37.661759 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a5d0d0db-7435-4204-9255-12e3d3ed844d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f56ddd455-vzpvx\" (UID: \"a5d0d0db-7435-4204-9255-12e3d3ed844d\") " pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-vzpvx" Jul 15 12:25:37.661853 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:37.661786 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a5d0d0db-7435-4204-9255-12e3d3ed844d-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f56ddd455-vzpvx\" (UID: \"a5d0d0db-7435-4204-9255-12e3d3ed844d\") " pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-vzpvx" Jul 15 12:25:37.674325 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:37.674246 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqgd9\" (UniqueName: \"kubernetes.io/projected/a5d0d0db-7435-4204-9255-12e3d3ed844d-kube-api-access-lqgd9\") pod \"opendatahub-operator-controller-manager-5f56ddd455-vzpvx\" (UID: \"a5d0d0db-7435-4204-9255-12e3d3ed844d\") " pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-vzpvx" Jul 15 12:25:37.732713 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:37.732683 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-vzpvx" Jul 15 12:25:37.922126 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:37.922097 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f56ddd455-vzpvx"] Jul 15 12:25:37.924802 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:25:37.924741 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5d0d0db_7435_4204_9255_12e3d3ed844d.slice/crio-3a2bc0d752d1bf6aa723316d5031eaf0c3a83216319e3a077b6bb8c50ad49b97 WatchSource:0}: Error finding container 3a2bc0d752d1bf6aa723316d5031eaf0c3a83216319e3a077b6bb8c50ad49b97: Status 404 returned error can't find the container with id 3a2bc0d752d1bf6aa723316d5031eaf0c3a83216319e3a077b6bb8c50ad49b97 Jul 15 12:25:37.926576 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:37.926559 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 12:25:38.838419 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:38.838374 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-vzpvx" event={"ID":"a5d0d0db-7435-4204-9255-12e3d3ed844d","Type":"ContainerStarted","Data":"3a2bc0d752d1bf6aa723316d5031eaf0c3a83216319e3a077b6bb8c50ad49b97"} Jul 15 12:25:41.855048 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:41.855016 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-vzpvx" event={"ID":"a5d0d0db-7435-4204-9255-12e3d3ed844d","Type":"ContainerStarted","Data":"0c9fe6584a221435357a5df573d23e2137be99333897f9fabf8f25e8c47d5226"} Jul 15 12:25:42.863706 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:42.863671 2560 generic.go:356] "Generic (PLEG): container finished" podID="a5d0d0db-7435-4204-9255-12e3d3ed844d" containerID="0c9fe6584a221435357a5df573d23e2137be99333897f9fabf8f25e8c47d5226" exitCode=0 Jul 15 12:25:42.864108 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:42.863761 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-vzpvx" event={"ID":"a5d0d0db-7435-4204-9255-12e3d3ed844d","Type":"ContainerDied","Data":"0c9fe6584a221435357a5df573d23e2137be99333897f9fabf8f25e8c47d5226"} Jul 15 12:25:43.869932 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:43.869894 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-vzpvx" event={"ID":"a5d0d0db-7435-4204-9255-12e3d3ed844d","Type":"ContainerStarted","Data":"04da326fea3340446799e7a0c7041046480e44cc7a7f30530747edaa352d8b2c"} Jul 15 12:25:43.870324 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:43.869981 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-vzpvx" Jul 15 12:25:43.907588 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:43.907541 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-vzpvx" podStartSLOduration=2.387979002 podStartE2EDuration="6.907528405s" totalImagesPullingTime="3.462003874s" totalInitContainerRuntime="1.057545529s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:25:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:25:43.907384514 +0000 UTC m=+317.048180625" watchObservedRunningTime="2026-07-15 12:25:43.907528405 +0000 UTC m=+317.048324515" Jul 15 12:25:54.878207 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:54.878178 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-vzpvx" Jul 15 12:25:57.795061 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:57.795027 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6c484c84b4-qxxxh"] Jul 15 12:25:57.800132 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:57.800112 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6c484c84b4-qxxxh" Jul 15 12:25:57.803553 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:57.803514 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 15 12:25:57.803679 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:57.803621 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 15 12:25:57.803679 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:57.803651 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 15 12:25:57.808616 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:57.808595 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 15 12:25:57.842206 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:57.842178 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vctz\" (UniqueName: \"kubernetes.io/projected/0a6955c9-30b4-42be-a3a8-e1debd48a8b0-kube-api-access-4vctz\") pod \"kube-auth-proxy-6c484c84b4-qxxxh\" (UID: \"0a6955c9-30b4-42be-a3a8-e1debd48a8b0\") " pod="openshift-ingress/kube-auth-proxy-6c484c84b4-qxxxh" Jul 15 12:25:57.842362 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:57.842295 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a6955c9-30b4-42be-a3a8-e1debd48a8b0-tls-certs\") pod \"kube-auth-proxy-6c484c84b4-qxxxh\" (UID: \"0a6955c9-30b4-42be-a3a8-e1debd48a8b0\") " pod="openshift-ingress/kube-auth-proxy-6c484c84b4-qxxxh" Jul 15 12:25:57.842362 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:57.842334 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0a6955c9-30b4-42be-a3a8-e1debd48a8b0-tmp\") pod \"kube-auth-proxy-6c484c84b4-qxxxh\" (UID: \"0a6955c9-30b4-42be-a3a8-e1debd48a8b0\") " pod="openshift-ingress/kube-auth-proxy-6c484c84b4-qxxxh" Jul 15 12:25:57.849928 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:57.849903 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6c484c84b4-qxxxh"] Jul 15 12:25:57.942879 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:57.942846 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vctz\" (UniqueName: \"kubernetes.io/projected/0a6955c9-30b4-42be-a3a8-e1debd48a8b0-kube-api-access-4vctz\") pod \"kube-auth-proxy-6c484c84b4-qxxxh\" (UID: \"0a6955c9-30b4-42be-a3a8-e1debd48a8b0\") " pod="openshift-ingress/kube-auth-proxy-6c484c84b4-qxxxh" Jul 15 12:25:57.943080 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:57.942927 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a6955c9-30b4-42be-a3a8-e1debd48a8b0-tls-certs\") pod \"kube-auth-proxy-6c484c84b4-qxxxh\" (UID: \"0a6955c9-30b4-42be-a3a8-e1debd48a8b0\") " pod="openshift-ingress/kube-auth-proxy-6c484c84b4-qxxxh" Jul 15 12:25:57.943080 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:57.943045 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0a6955c9-30b4-42be-a3a8-e1debd48a8b0-tmp\") pod \"kube-auth-proxy-6c484c84b4-qxxxh\" (UID: \"0a6955c9-30b4-42be-a3a8-e1debd48a8b0\") " pod="openshift-ingress/kube-auth-proxy-6c484c84b4-qxxxh" Jul 15 12:25:57.945193 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:57.945165 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0a6955c9-30b4-42be-a3a8-e1debd48a8b0-tmp\") pod \"kube-auth-proxy-6c484c84b4-qxxxh\" (UID: \"0a6955c9-30b4-42be-a3a8-e1debd48a8b0\") " pod="openshift-ingress/kube-auth-proxy-6c484c84b4-qxxxh" Jul 15 12:25:57.945491 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:57.945471 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a6955c9-30b4-42be-a3a8-e1debd48a8b0-tls-certs\") pod \"kube-auth-proxy-6c484c84b4-qxxxh\" (UID: \"0a6955c9-30b4-42be-a3a8-e1debd48a8b0\") " pod="openshift-ingress/kube-auth-proxy-6c484c84b4-qxxxh" Jul 15 12:25:57.981498 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:57.981463 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vctz\" (UniqueName: \"kubernetes.io/projected/0a6955c9-30b4-42be-a3a8-e1debd48a8b0-kube-api-access-4vctz\") pod \"kube-auth-proxy-6c484c84b4-qxxxh\" (UID: \"0a6955c9-30b4-42be-a3a8-e1debd48a8b0\") " pod="openshift-ingress/kube-auth-proxy-6c484c84b4-qxxxh" Jul 15 12:25:58.000109 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:58.000078 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-5dc4q"] Jul 15 12:25:58.004011 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:58.003989 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-5dc4q" Jul 15 12:25:58.008121 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:58.008100 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 15 12:25:58.008290 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:58.008273 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-zcqjp\"" Jul 15 12:25:58.021825 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:58.021795 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-5dc4q"] Jul 15 12:25:58.109481 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:58.109404 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6c484c84b4-qxxxh" Jul 15 12:25:58.145618 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:58.145581 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a7189dee-11f0-4215-ac65-1b3979234a21-cert\") pod \"odh-model-controller-7998c67dc4-5dc4q\" (UID: \"a7189dee-11f0-4215-ac65-1b3979234a21\") " pod="opendatahub/odh-model-controller-7998c67dc4-5dc4q" Jul 15 12:25:58.145795 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:58.145620 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2rmq\" (UniqueName: \"kubernetes.io/projected/a7189dee-11f0-4215-ac65-1b3979234a21-kube-api-access-f2rmq\") pod \"odh-model-controller-7998c67dc4-5dc4q\" (UID: \"a7189dee-11f0-4215-ac65-1b3979234a21\") " pod="opendatahub/odh-model-controller-7998c67dc4-5dc4q" Jul 15 12:25:58.246458 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:58.246418 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a7189dee-11f0-4215-ac65-1b3979234a21-cert\") pod \"odh-model-controller-7998c67dc4-5dc4q\" (UID: \"a7189dee-11f0-4215-ac65-1b3979234a21\") " pod="opendatahub/odh-model-controller-7998c67dc4-5dc4q" Jul 15 12:25:58.246458 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:58.246457 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2rmq\" (UniqueName: \"kubernetes.io/projected/a7189dee-11f0-4215-ac65-1b3979234a21-kube-api-access-f2rmq\") pod \"odh-model-controller-7998c67dc4-5dc4q\" (UID: \"a7189dee-11f0-4215-ac65-1b3979234a21\") " pod="opendatahub/odh-model-controller-7998c67dc4-5dc4q" Jul 15 12:25:58.246671 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:25:58.246578 2560 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 15 12:25:58.246719 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:25:58.246676 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7189dee-11f0-4215-ac65-1b3979234a21-cert podName:a7189dee-11f0-4215-ac65-1b3979234a21 nodeName:}" failed. No retries permitted until 2026-07-15 12:25:58.746631375 +0000 UTC m=+331.887427465 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a7189dee-11f0-4215-ac65-1b3979234a21-cert") pod "odh-model-controller-7998c67dc4-5dc4q" (UID: "a7189dee-11f0-4215-ac65-1b3979234a21") : secret "odh-model-controller-webhook-cert" not found Jul 15 12:25:58.266359 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:58.266330 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2rmq\" (UniqueName: \"kubernetes.io/projected/a7189dee-11f0-4215-ac65-1b3979234a21-kube-api-access-f2rmq\") pod \"odh-model-controller-7998c67dc4-5dc4q\" (UID: \"a7189dee-11f0-4215-ac65-1b3979234a21\") " pod="opendatahub/odh-model-controller-7998c67dc4-5dc4q" Jul 15 12:25:58.277467 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:25:58.277444 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a6955c9_30b4_42be_a3a8_e1debd48a8b0.slice/crio-c43cd65bcc5108a6c12b722731994cc3fe4e6d31df198d503cc6bc6ece689693 WatchSource:0}: Error finding container c43cd65bcc5108a6c12b722731994cc3fe4e6d31df198d503cc6bc6ece689693: Status 404 returned error can't find the container with id c43cd65bcc5108a6c12b722731994cc3fe4e6d31df198d503cc6bc6ece689693 Jul 15 12:25:58.278297 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:58.278272 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6c484c84b4-qxxxh"] Jul 15 12:25:58.751821 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:58.751217 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a7189dee-11f0-4215-ac65-1b3979234a21-cert\") pod \"odh-model-controller-7998c67dc4-5dc4q\" (UID: \"a7189dee-11f0-4215-ac65-1b3979234a21\") " pod="opendatahub/odh-model-controller-7998c67dc4-5dc4q" Jul 15 12:25:58.755191 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:58.755164 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a7189dee-11f0-4215-ac65-1b3979234a21-cert\") pod \"odh-model-controller-7998c67dc4-5dc4q\" (UID: \"a7189dee-11f0-4215-ac65-1b3979234a21\") " pod="opendatahub/odh-model-controller-7998c67dc4-5dc4q" Jul 15 12:25:58.916122 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:58.916068 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-5dc4q" Jul 15 12:25:58.939298 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:58.939267 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6c484c84b4-qxxxh" event={"ID":"0a6955c9-30b4-42be-a3a8-e1debd48a8b0","Type":"ContainerStarted","Data":"c43cd65bcc5108a6c12b722731994cc3fe4e6d31df198d503cc6bc6ece689693"} Jul 15 12:25:59.073766 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:59.073679 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-5dc4q"] Jul 15 12:25:59.078267 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:25:59.078228 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7189dee_11f0_4215_ac65_1b3979234a21.slice/crio-d921e613b0bd047fee69c12a59b294edd468eb11d12fcfd1e692b0a430bd792c WatchSource:0}: Error finding container d921e613b0bd047fee69c12a59b294edd468eb11d12fcfd1e692b0a430bd792c: Status 404 returned error can't find the container with id d921e613b0bd047fee69c12a59b294edd468eb11d12fcfd1e692b0a430bd792c Jul 15 12:25:59.718415 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:59.718379 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gk6xz"] Jul 15 12:25:59.722122 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:59.722080 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-gk6xz" Jul 15 12:25:59.727847 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:59.727821 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 15 12:25:59.749285 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:59.749263 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gk6xz"] Jul 15 12:25:59.863262 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:59.863222 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8c8n2\" (UniqueName: \"kubernetes.io/projected/65e23de7-f6fd-4018-b78e-fa388c118295-kube-api-access-8c8n2\") pod \"model-serving-api-54d89bd5fc-gk6xz\" (UID: \"65e23de7-f6fd-4018-b78e-fa388c118295\") " pod="opendatahub/model-serving-api-54d89bd5fc-gk6xz" Jul 15 12:25:59.863446 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:59.863301 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65e23de7-f6fd-4018-b78e-fa388c118295-tls-certs\") pod \"model-serving-api-54d89bd5fc-gk6xz\" (UID: \"65e23de7-f6fd-4018-b78e-fa388c118295\") " pod="opendatahub/model-serving-api-54d89bd5fc-gk6xz" Jul 15 12:25:59.946433 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:59.946313 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-5dc4q" event={"ID":"a7189dee-11f0-4215-ac65-1b3979234a21","Type":"ContainerStarted","Data":"d921e613b0bd047fee69c12a59b294edd468eb11d12fcfd1e692b0a430bd792c"} Jul 15 12:25:59.964856 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:59.964822 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8c8n2\" (UniqueName: \"kubernetes.io/projected/65e23de7-f6fd-4018-b78e-fa388c118295-kube-api-access-8c8n2\") pod \"model-serving-api-54d89bd5fc-gk6xz\" (UID: \"65e23de7-f6fd-4018-b78e-fa388c118295\") " pod="opendatahub/model-serving-api-54d89bd5fc-gk6xz" Jul 15 12:25:59.965019 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:59.964983 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65e23de7-f6fd-4018-b78e-fa388c118295-tls-certs\") pod \"model-serving-api-54d89bd5fc-gk6xz\" (UID: \"65e23de7-f6fd-4018-b78e-fa388c118295\") " pod="opendatahub/model-serving-api-54d89bd5fc-gk6xz" Jul 15 12:25:59.965144 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:25:59.965126 2560 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 15 12:25:59.965201 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:25:59.965191 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65e23de7-f6fd-4018-b78e-fa388c118295-tls-certs podName:65e23de7-f6fd-4018-b78e-fa388c118295 nodeName:}" failed. No retries permitted until 2026-07-15 12:26:00.465173533 +0000 UTC m=+333.605969628 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/65e23de7-f6fd-4018-b78e-fa388c118295-tls-certs") pod "model-serving-api-54d89bd5fc-gk6xz" (UID: "65e23de7-f6fd-4018-b78e-fa388c118295") : secret "model-serving-api-tls" not found Jul 15 12:25:59.993938 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:25:59.993873 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8c8n2\" (UniqueName: \"kubernetes.io/projected/65e23de7-f6fd-4018-b78e-fa388c118295-kube-api-access-8c8n2\") pod \"model-serving-api-54d89bd5fc-gk6xz\" (UID: \"65e23de7-f6fd-4018-b78e-fa388c118295\") " pod="opendatahub/model-serving-api-54d89bd5fc-gk6xz" Jul 15 12:26:00.469567 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:00.469532 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65e23de7-f6fd-4018-b78e-fa388c118295-tls-certs\") pod \"model-serving-api-54d89bd5fc-gk6xz\" (UID: \"65e23de7-f6fd-4018-b78e-fa388c118295\") " pod="opendatahub/model-serving-api-54d89bd5fc-gk6xz" Jul 15 12:26:00.471889 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:00.471871 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65e23de7-f6fd-4018-b78e-fa388c118295-tls-certs\") pod \"model-serving-api-54d89bd5fc-gk6xz\" (UID: \"65e23de7-f6fd-4018-b78e-fa388c118295\") " pod="opendatahub/model-serving-api-54d89bd5fc-gk6xz" Jul 15 12:26:00.537451 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:00.537417 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7"] Jul 15 12:26:00.541170 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:00.541144 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7" Jul 15 12:26:00.545848 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:00.545234 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 15 12:26:00.545848 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:00.545508 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-qs5rl\"" Jul 15 12:26:00.545848 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:00.545515 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 15 12:26:00.545848 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:00.545734 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 15 12:26:00.545848 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:00.545748 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 15 12:26:00.575932 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:00.575900 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7"] Jul 15 12:26:00.631991 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:00.631961 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-gk6xz" Jul 15 12:26:00.672324 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:00.671919 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/05c095b7-d1e4-4949-a4dc-37d83003eb0a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-xjkq7\" (UID: \"05c095b7-d1e4-4949-a4dc-37d83003eb0a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7" Jul 15 12:26:00.672324 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:00.671975 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/05c095b7-d1e4-4949-a4dc-37d83003eb0a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-xjkq7\" (UID: \"05c095b7-d1e4-4949-a4dc-37d83003eb0a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7" Jul 15 12:26:00.672324 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:00.672032 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/05c095b7-d1e4-4949-a4dc-37d83003eb0a-cacerts\") pod \"istiod-openshift-gateway-7574459656-xjkq7\" (UID: \"05c095b7-d1e4-4949-a4dc-37d83003eb0a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7" Jul 15 12:26:00.672324 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:00.672083 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/05c095b7-d1e4-4949-a4dc-37d83003eb0a-local-certs\") pod \"istiod-openshift-gateway-7574459656-xjkq7\" (UID: \"05c095b7-d1e4-4949-a4dc-37d83003eb0a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7" Jul 15 12:26:00.672324 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:00.672106 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05c095b7-d1e4-4949-a4dc-37d83003eb0a-istio-token\") pod \"istiod-openshift-gateway-7574459656-xjkq7\" (UID: \"05c095b7-d1e4-4949-a4dc-37d83003eb0a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7" Jul 15 12:26:00.672324 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:00.672160 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/05c095b7-d1e4-4949-a4dc-37d83003eb0a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-xjkq7\" (UID: \"05c095b7-d1e4-4949-a4dc-37d83003eb0a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7" Jul 15 12:26:00.672324 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:00.672187 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8spr\" (UniqueName: \"kubernetes.io/projected/05c095b7-d1e4-4949-a4dc-37d83003eb0a-kube-api-access-j8spr\") pod \"istiod-openshift-gateway-7574459656-xjkq7\" (UID: \"05c095b7-d1e4-4949-a4dc-37d83003eb0a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7" Jul 15 12:26:00.773421 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:00.773385 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/05c095b7-d1e4-4949-a4dc-37d83003eb0a-local-certs\") pod \"istiod-openshift-gateway-7574459656-xjkq7\" (UID: \"05c095b7-d1e4-4949-a4dc-37d83003eb0a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7" Jul 15 12:26:00.773590 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:00.773434 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05c095b7-d1e4-4949-a4dc-37d83003eb0a-istio-token\") pod \"istiod-openshift-gateway-7574459656-xjkq7\" (UID: \"05c095b7-d1e4-4949-a4dc-37d83003eb0a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7" Jul 15 12:26:00.773590 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:00.773489 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/05c095b7-d1e4-4949-a4dc-37d83003eb0a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-xjkq7\" (UID: \"05c095b7-d1e4-4949-a4dc-37d83003eb0a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7" Jul 15 12:26:00.773590 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:00.773523 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8spr\" (UniqueName: \"kubernetes.io/projected/05c095b7-d1e4-4949-a4dc-37d83003eb0a-kube-api-access-j8spr\") pod \"istiod-openshift-gateway-7574459656-xjkq7\" (UID: \"05c095b7-d1e4-4949-a4dc-37d83003eb0a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7" Jul 15 12:26:00.773794 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:00.773623 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/05c095b7-d1e4-4949-a4dc-37d83003eb0a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-xjkq7\" (UID: \"05c095b7-d1e4-4949-a4dc-37d83003eb0a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7" Jul 15 12:26:00.773794 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:00.773688 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/05c095b7-d1e4-4949-a4dc-37d83003eb0a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-xjkq7\" (UID: \"05c095b7-d1e4-4949-a4dc-37d83003eb0a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7" Jul 15 12:26:00.773794 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:00.773756 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/05c095b7-d1e4-4949-a4dc-37d83003eb0a-cacerts\") pod \"istiod-openshift-gateway-7574459656-xjkq7\" (UID: \"05c095b7-d1e4-4949-a4dc-37d83003eb0a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7" Jul 15 12:26:00.774990 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:00.774959 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/05c095b7-d1e4-4949-a4dc-37d83003eb0a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-xjkq7\" (UID: \"05c095b7-d1e4-4949-a4dc-37d83003eb0a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7" Jul 15 12:26:00.776862 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:00.776837 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/05c095b7-d1e4-4949-a4dc-37d83003eb0a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-xjkq7\" (UID: \"05c095b7-d1e4-4949-a4dc-37d83003eb0a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7" Jul 15 12:26:00.777090 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:00.777067 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/05c095b7-d1e4-4949-a4dc-37d83003eb0a-local-certs\") pod \"istiod-openshift-gateway-7574459656-xjkq7\" (UID: \"05c095b7-d1e4-4949-a4dc-37d83003eb0a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7" Jul 15 12:26:00.778579 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:00.778448 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/05c095b7-d1e4-4949-a4dc-37d83003eb0a-cacerts\") pod \"istiod-openshift-gateway-7574459656-xjkq7\" (UID: \"05c095b7-d1e4-4949-a4dc-37d83003eb0a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7" Jul 15 12:26:00.778899 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:00.778877 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/05c095b7-d1e4-4949-a4dc-37d83003eb0a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-xjkq7\" (UID: \"05c095b7-d1e4-4949-a4dc-37d83003eb0a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7" Jul 15 12:26:00.785383 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:00.785302 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05c095b7-d1e4-4949-a4dc-37d83003eb0a-istio-token\") pod \"istiod-openshift-gateway-7574459656-xjkq7\" (UID: \"05c095b7-d1e4-4949-a4dc-37d83003eb0a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7" Jul 15 12:26:00.785493 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:00.785400 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8spr\" (UniqueName: \"kubernetes.io/projected/05c095b7-d1e4-4949-a4dc-37d83003eb0a-kube-api-access-j8spr\") pod \"istiod-openshift-gateway-7574459656-xjkq7\" (UID: \"05c095b7-d1e4-4949-a4dc-37d83003eb0a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7" Jul 15 12:26:00.826831 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:00.826799 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gk6xz"] Jul 15 12:26:00.855700 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:00.855628 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7" Jul 15 12:26:01.427344 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:26:01.427308 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65e23de7_f6fd_4018_b78e_fa388c118295.slice/crio-6d9aa574b64259aa0450de90395dde272d7d7e44674746884173e1db81c2cf1f WatchSource:0}: Error finding container 6d9aa574b64259aa0450de90395dde272d7d7e44674746884173e1db81c2cf1f: Status 404 returned error can't find the container with id 6d9aa574b64259aa0450de90395dde272d7d7e44674746884173e1db81c2cf1f Jul 15 12:26:01.959651 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:01.959595 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-gk6xz" event={"ID":"65e23de7-f6fd-4018-b78e-fa388c118295","Type":"ContainerStarted","Data":"6d9aa574b64259aa0450de90395dde272d7d7e44674746884173e1db81c2cf1f"} Jul 15 12:26:02.911363 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:02.911331 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7"] Jul 15 12:26:02.912387 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:26:02.912361 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05c095b7_d1e4_4949_a4dc_37d83003eb0a.slice/crio-4078ba6715c7ff2ce595091d5a1e370646f8484e9f2c69b336618ab0fa980f4f WatchSource:0}: Error finding container 4078ba6715c7ff2ce595091d5a1e370646f8484e9f2c69b336618ab0fa980f4f: Status 404 returned error can't find the container with id 4078ba6715c7ff2ce595091d5a1e370646f8484e9f2c69b336618ab0fa980f4f Jul 15 12:26:02.966398 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:02.966364 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-5dc4q" event={"ID":"a7189dee-11f0-4215-ac65-1b3979234a21","Type":"ContainerStarted","Data":"4dbb0cd1969d714346177602db4fcbe224880844b8d7d2e98b2fd7376d4ab0a2"} Jul 15 12:26:02.966568 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:02.966427 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-5dc4q" Jul 15 12:26:02.967774 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:02.967749 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6c484c84b4-qxxxh" event={"ID":"0a6955c9-30b4-42be-a3a8-e1debd48a8b0","Type":"ContainerStarted","Data":"3ba9780e8659009a5472ddd7dbf88d9a45a621d9bf9f3e53ccbe150b0e6706a9"} Jul 15 12:26:02.968892 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:02.968870 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7" event={"ID":"05c095b7-d1e4-4949-a4dc-37d83003eb0a","Type":"ContainerStarted","Data":"4078ba6715c7ff2ce595091d5a1e370646f8484e9f2c69b336618ab0fa980f4f"} Jul 15 12:26:02.999980 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:02.999882 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-5dc4q" podStartSLOduration=2.296132563 podStartE2EDuration="5.999864849s" totalImagesPullingTime="3.703732286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:25:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:26:02.998380965 +0000 UTC m=+336.139177076" watchObservedRunningTime="2026-07-15 12:26:02.999864849 +0000 UTC m=+336.140660959" Jul 15 12:26:03.055148 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:03.055098 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6c484c84b4-qxxxh" podStartSLOduration=1.595831285 podStartE2EDuration="6.055079944s" totalImagesPullingTime="4.459248659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:25:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:26:03.050003848 +0000 UTC m=+336.190799959" watchObservedRunningTime="2026-07-15 12:26:03.055079944 +0000 UTC m=+336.195876055" Jul 15 12:26:03.980767 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:03.980688 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-gk6xz" event={"ID":"65e23de7-f6fd-4018-b78e-fa388c118295","Type":"ContainerStarted","Data":"40e2e078dc697c839125ad52e317a0aa0c8bc5a8e0a4dda9bb05da7b4ab11d08"} Jul 15 12:26:03.981470 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:03.981442 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-gk6xz" Jul 15 12:26:04.020662 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:04.020577 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-gk6xz" podStartSLOduration=2.892832395 podStartE2EDuration="5.020554978s" totalImagesPullingTime="2.127722583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:25:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:26:04.019797733 +0000 UTC m=+337.160593868" watchObservedRunningTime="2026-07-15 12:26:04.020554978 +0000 UTC m=+337.161351090" Jul 15 12:26:05.367191 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:05.367152 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:26:05.367537 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:05.367230 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:26:05.994145 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:05.994102 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7" event={"ID":"05c095b7-d1e4-4949-a4dc-37d83003eb0a","Type":"ContainerStarted","Data":"bb595b4eaf3dfc343bbb011412f6859e1eae1bdb8cf38b4833ef5eb88b70e836"} Jul 15 12:26:05.994334 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:05.994316 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7" Jul 15 12:26:06.093521 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:06.093467 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7" podStartSLOduration=3.64095314 podStartE2EDuration="6.093448126s" totalImagesPullingTime="2.452494986s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:26:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:26:06.086890009 +0000 UTC m=+339.227686119" watchObservedRunningTime="2026-07-15 12:26:06.093448126 +0000 UTC m=+339.234244237" Jul 15 12:26:07.000868 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:07.000835 2560 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-xjkq7 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 12:26:07.001235 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:07.000886 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7" podUID="05c095b7-d1e4-4949-a4dc-37d83003eb0a" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 12:26:08.421352 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.421320 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr"] Jul 15 12:26:08.424407 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.424389 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" Jul 15 12:26:08.427711 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.427685 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-vlh2v\"" Jul 15 12:26:08.449405 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.449383 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr"] Jul 15 12:26:08.499967 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.499936 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bzkz"] Jul 15 12:26:08.503257 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.503236 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bzkz" Jul 15 12:26:08.535411 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.535390 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bzkz"] Jul 15 12:26:08.551793 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.551761 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6fa3be8-3559-4e2f-882c-a4bc7b84c55a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l2zsr\" (UID: \"c6fa3be8-3559-4e2f-882c-a4bc7b84c55a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" Jul 15 12:26:08.551976 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.551944 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6fa3be8-3559-4e2f-882c-a4bc7b84c55a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l2zsr\" (UID: \"c6fa3be8-3559-4e2f-882c-a4bc7b84c55a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" Jul 15 12:26:08.552180 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.552164 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6fa3be8-3559-4e2f-882c-a4bc7b84c55a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l2zsr\" (UID: \"c6fa3be8-3559-4e2f-882c-a4bc7b84c55a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" Jul 15 12:26:08.552512 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.552294 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6fa3be8-3559-4e2f-882c-a4bc7b84c55a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l2zsr\" (UID: \"c6fa3be8-3559-4e2f-882c-a4bc7b84c55a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" Jul 15 12:26:08.552512 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.552324 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t42xt\" (UniqueName: \"kubernetes.io/projected/c6fa3be8-3559-4e2f-882c-a4bc7b84c55a-kube-api-access-t42xt\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l2zsr\" (UID: \"c6fa3be8-3559-4e2f-882c-a4bc7b84c55a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" Jul 15 12:26:08.552512 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.552469 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6fa3be8-3559-4e2f-882c-a4bc7b84c55a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l2zsr\" (UID: \"c6fa3be8-3559-4e2f-882c-a4bc7b84c55a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" Jul 15 12:26:08.552512 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.552512 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6fa3be8-3559-4e2f-882c-a4bc7b84c55a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l2zsr\" (UID: \"c6fa3be8-3559-4e2f-882c-a4bc7b84c55a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" Jul 15 12:26:08.552809 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.552563 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6fa3be8-3559-4e2f-882c-a4bc7b84c55a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l2zsr\" (UID: \"c6fa3be8-3559-4e2f-882c-a4bc7b84c55a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" Jul 15 12:26:08.552809 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.552677 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6fa3be8-3559-4e2f-882c-a4bc7b84c55a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l2zsr\" (UID: \"c6fa3be8-3559-4e2f-882c-a4bc7b84c55a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" Jul 15 12:26:08.653605 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.653576 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6fa3be8-3559-4e2f-882c-a4bc7b84c55a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l2zsr\" (UID: \"c6fa3be8-3559-4e2f-882c-a4bc7b84c55a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" Jul 15 12:26:08.653805 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.653620 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63ea41ea-c57e-4adc-a87f-6ac5436abfc1-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-7bzkz\" (UID: \"63ea41ea-c57e-4adc-a87f-6ac5436abfc1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bzkz" Jul 15 12:26:08.653805 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.653673 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6fa3be8-3559-4e2f-882c-a4bc7b84c55a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l2zsr\" (UID: \"c6fa3be8-3559-4e2f-882c-a4bc7b84c55a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" Jul 15 12:26:08.653805 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.653705 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6fa3be8-3559-4e2f-882c-a4bc7b84c55a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l2zsr\" (UID: \"c6fa3be8-3559-4e2f-882c-a4bc7b84c55a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" Jul 15 12:26:08.653805 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.653730 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/63ea41ea-c57e-4adc-a87f-6ac5436abfc1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-7bzkz\" (UID: \"63ea41ea-c57e-4adc-a87f-6ac5436abfc1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bzkz" Jul 15 12:26:08.653805 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.653776 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6fa3be8-3559-4e2f-882c-a4bc7b84c55a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l2zsr\" (UID: \"c6fa3be8-3559-4e2f-882c-a4bc7b84c55a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" Jul 15 12:26:08.653805 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.653796 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t42xt\" (UniqueName: \"kubernetes.io/projected/c6fa3be8-3559-4e2f-882c-a4bc7b84c55a-kube-api-access-t42xt\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l2zsr\" (UID: \"c6fa3be8-3559-4e2f-882c-a4bc7b84c55a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" Jul 15 12:26:08.654145 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.653813 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/63ea41ea-c57e-4adc-a87f-6ac5436abfc1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-7bzkz\" (UID: \"63ea41ea-c57e-4adc-a87f-6ac5436abfc1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bzkz" Jul 15 12:26:08.654145 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.653847 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/63ea41ea-c57e-4adc-a87f-6ac5436abfc1-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-7bzkz\" (UID: \"63ea41ea-c57e-4adc-a87f-6ac5436abfc1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bzkz" Jul 15 12:26:08.654145 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.653875 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6fa3be8-3559-4e2f-882c-a4bc7b84c55a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l2zsr\" (UID: \"c6fa3be8-3559-4e2f-882c-a4bc7b84c55a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" Jul 15 12:26:08.654145 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.653903 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6fa3be8-3559-4e2f-882c-a4bc7b84c55a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l2zsr\" (UID: \"c6fa3be8-3559-4e2f-882c-a4bc7b84c55a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" Jul 15 12:26:08.654145 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.653959 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6fa3be8-3559-4e2f-882c-a4bc7b84c55a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l2zsr\" (UID: \"c6fa3be8-3559-4e2f-882c-a4bc7b84c55a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" Jul 15 12:26:08.654145 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.654001 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c65n9\" (UniqueName: \"kubernetes.io/projected/63ea41ea-c57e-4adc-a87f-6ac5436abfc1-kube-api-access-c65n9\") pod \"istiod-openshift-gateway-94bb8fbfd-7bzkz\" (UID: \"63ea41ea-c57e-4adc-a87f-6ac5436abfc1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bzkz" Jul 15 12:26:08.654145 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.654030 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/63ea41ea-c57e-4adc-a87f-6ac5436abfc1-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-7bzkz\" (UID: \"63ea41ea-c57e-4adc-a87f-6ac5436abfc1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bzkz" Jul 15 12:26:08.654145 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.654084 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6fa3be8-3559-4e2f-882c-a4bc7b84c55a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l2zsr\" (UID: \"c6fa3be8-3559-4e2f-882c-a4bc7b84c55a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" Jul 15 12:26:08.654527 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.654165 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/63ea41ea-c57e-4adc-a87f-6ac5436abfc1-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-7bzkz\" (UID: \"63ea41ea-c57e-4adc-a87f-6ac5436abfc1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bzkz" Jul 15 12:26:08.654527 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.654285 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6fa3be8-3559-4e2f-882c-a4bc7b84c55a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l2zsr\" (UID: \"c6fa3be8-3559-4e2f-882c-a4bc7b84c55a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" Jul 15 12:26:08.654527 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.654333 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6fa3be8-3559-4e2f-882c-a4bc7b84c55a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l2zsr\" (UID: \"c6fa3be8-3559-4e2f-882c-a4bc7b84c55a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" Jul 15 12:26:08.654527 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.654373 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6fa3be8-3559-4e2f-882c-a4bc7b84c55a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l2zsr\" (UID: \"c6fa3be8-3559-4e2f-882c-a4bc7b84c55a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" Jul 15 12:26:08.654527 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.654444 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6fa3be8-3559-4e2f-882c-a4bc7b84c55a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l2zsr\" (UID: \"c6fa3be8-3559-4e2f-882c-a4bc7b84c55a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" Jul 15 12:26:08.654527 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.654460 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6fa3be8-3559-4e2f-882c-a4bc7b84c55a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l2zsr\" (UID: \"c6fa3be8-3559-4e2f-882c-a4bc7b84c55a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" Jul 15 12:26:08.656108 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.656084 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6fa3be8-3559-4e2f-882c-a4bc7b84c55a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l2zsr\" (UID: \"c6fa3be8-3559-4e2f-882c-a4bc7b84c55a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" Jul 15 12:26:08.656224 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.656179 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6fa3be8-3559-4e2f-882c-a4bc7b84c55a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l2zsr\" (UID: \"c6fa3be8-3559-4e2f-882c-a4bc7b84c55a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" Jul 15 12:26:08.665224 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.665195 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t42xt\" (UniqueName: \"kubernetes.io/projected/c6fa3be8-3559-4e2f-882c-a4bc7b84c55a-kube-api-access-t42xt\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l2zsr\" (UID: \"c6fa3be8-3559-4e2f-882c-a4bc7b84c55a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" Jul 15 12:26:08.667988 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.667969 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6fa3be8-3559-4e2f-882c-a4bc7b84c55a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l2zsr\" (UID: \"c6fa3be8-3559-4e2f-882c-a4bc7b84c55a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" Jul 15 12:26:08.734364 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.734298 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" Jul 15 12:26:08.754867 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.754844 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/63ea41ea-c57e-4adc-a87f-6ac5436abfc1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-7bzkz\" (UID: \"63ea41ea-c57e-4adc-a87f-6ac5436abfc1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bzkz" Jul 15 12:26:08.754980 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.754886 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/63ea41ea-c57e-4adc-a87f-6ac5436abfc1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-7bzkz\" (UID: \"63ea41ea-c57e-4adc-a87f-6ac5436abfc1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bzkz" Jul 15 12:26:08.754980 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.754917 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/63ea41ea-c57e-4adc-a87f-6ac5436abfc1-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-7bzkz\" (UID: \"63ea41ea-c57e-4adc-a87f-6ac5436abfc1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bzkz" Jul 15 12:26:08.754980 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.754953 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c65n9\" (UniqueName: \"kubernetes.io/projected/63ea41ea-c57e-4adc-a87f-6ac5436abfc1-kube-api-access-c65n9\") pod \"istiod-openshift-gateway-94bb8fbfd-7bzkz\" (UID: \"63ea41ea-c57e-4adc-a87f-6ac5436abfc1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bzkz" Jul 15 12:26:08.754980 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.754976 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/63ea41ea-c57e-4adc-a87f-6ac5436abfc1-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-7bzkz\" (UID: \"63ea41ea-c57e-4adc-a87f-6ac5436abfc1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bzkz" Jul 15 12:26:08.755183 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.755018 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/63ea41ea-c57e-4adc-a87f-6ac5436abfc1-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-7bzkz\" (UID: \"63ea41ea-c57e-4adc-a87f-6ac5436abfc1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bzkz" Jul 15 12:26:08.755183 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.755084 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63ea41ea-c57e-4adc-a87f-6ac5436abfc1-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-7bzkz\" (UID: \"63ea41ea-c57e-4adc-a87f-6ac5436abfc1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bzkz" Jul 15 12:26:08.755737 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.755713 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/63ea41ea-c57e-4adc-a87f-6ac5436abfc1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-7bzkz\" (UID: \"63ea41ea-c57e-4adc-a87f-6ac5436abfc1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bzkz" Jul 15 12:26:08.757917 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.757892 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/63ea41ea-c57e-4adc-a87f-6ac5436abfc1-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-7bzkz\" (UID: \"63ea41ea-c57e-4adc-a87f-6ac5436abfc1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bzkz" Jul 15 12:26:08.758023 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.757944 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/63ea41ea-c57e-4adc-a87f-6ac5436abfc1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-7bzkz\" (UID: \"63ea41ea-c57e-4adc-a87f-6ac5436abfc1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bzkz" Jul 15 12:26:08.758292 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.758268 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/63ea41ea-c57e-4adc-a87f-6ac5436abfc1-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-7bzkz\" (UID: \"63ea41ea-c57e-4adc-a87f-6ac5436abfc1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bzkz" Jul 15 12:26:08.758390 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.758304 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/63ea41ea-c57e-4adc-a87f-6ac5436abfc1-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-7bzkz\" (UID: \"63ea41ea-c57e-4adc-a87f-6ac5436abfc1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bzkz" Jul 15 12:26:08.771531 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.771501 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63ea41ea-c57e-4adc-a87f-6ac5436abfc1-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-7bzkz\" (UID: \"63ea41ea-c57e-4adc-a87f-6ac5436abfc1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bzkz" Jul 15 12:26:08.782181 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.782155 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c65n9\" (UniqueName: \"kubernetes.io/projected/63ea41ea-c57e-4adc-a87f-6ac5436abfc1-kube-api-access-c65n9\") pod \"istiod-openshift-gateway-94bb8fbfd-7bzkz\" (UID: \"63ea41ea-c57e-4adc-a87f-6ac5436abfc1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bzkz" Jul 15 12:26:08.811920 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.811884 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bzkz" Jul 15 12:26:08.904751 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:26:08.904716 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6fa3be8_3559_4e2f_882c_a4bc7b84c55a.slice/crio-20f642541dfd4dbd4689d2fe3c00b380394b5fa7aa44f7aab9dbb3c2d37cb7d9 WatchSource:0}: Error finding container 20f642541dfd4dbd4689d2fe3c00b380394b5fa7aa44f7aab9dbb3c2d37cb7d9: Status 404 returned error can't find the container with id 20f642541dfd4dbd4689d2fe3c00b380394b5fa7aa44f7aab9dbb3c2d37cb7d9 Jul 15 12:26:08.915667 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.915629 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr"] Jul 15 12:26:08.982838 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.982813 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bzkz"] Jul 15 12:26:08.985181 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:26:08.985121 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63ea41ea_c57e_4adc_a87f_6ac5436abfc1.slice/crio-5335f5b82280e217f90d7da3ed35fdf302f1c3f802526f9139a427252ca9d169 WatchSource:0}: Error finding container 5335f5b82280e217f90d7da3ed35fdf302f1c3f802526f9139a427252ca9d169: Status 404 returned error can't find the container with id 5335f5b82280e217f90d7da3ed35fdf302f1c3f802526f9139a427252ca9d169 Jul 15 12:26:08.988354 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.988325 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:26:08.988440 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:08.988380 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:26:09.011064 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:09.011039 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bzkz" event={"ID":"63ea41ea-c57e-4adc-a87f-6ac5436abfc1","Type":"ContainerStarted","Data":"5335f5b82280e217f90d7da3ed35fdf302f1c3f802526f9139a427252ca9d169"} Jul 15 12:26:09.012205 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:09.012185 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" event={"ID":"c6fa3be8-3559-4e2f-882c-a4bc7b84c55a","Type":"ContainerStarted","Data":"20f642541dfd4dbd4689d2fe3c00b380394b5fa7aa44f7aab9dbb3c2d37cb7d9"} Jul 15 12:26:10.000313 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.000284 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7" Jul 15 12:26:10.020021 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.019991 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bzkz" event={"ID":"63ea41ea-c57e-4adc-a87f-6ac5436abfc1","Type":"ContainerStarted","Data":"4759a9e2fc0007bc238c0bd5e5b19acf57b4272c424ce96e088e3e5f9d2b2424"} Jul 15 12:26:10.020195 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.020176 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bzkz" Jul 15 12:26:10.022173 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.022147 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bzkz" Jul 15 12:26:10.080975 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.080927 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7bzkz" podStartSLOduration=2.080911763 podStartE2EDuration="2.080911763s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:26:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:26:10.078439418 +0000 UTC m=+343.219235558" watchObservedRunningTime="2026-07-15 12:26:10.080911763 +0000 UTC m=+343.221707874" Jul 15 12:26:10.137978 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.137941 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7"] Jul 15 12:26:10.138202 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.138172 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7" podUID="05c095b7-d1e4-4949-a4dc-37d83003eb0a" containerName="discovery" containerID="cri-o://bb595b4eaf3dfc343bbb011412f6859e1eae1bdb8cf38b4833ef5eb88b70e836" gracePeriod=30 Jul 15 12:26:10.410114 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.410087 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7" Jul 15 12:26:10.577176 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.577146 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/05c095b7-d1e4-4949-a4dc-37d83003eb0a-local-certs\") pod \"05c095b7-d1e4-4949-a4dc-37d83003eb0a\" (UID: \"05c095b7-d1e4-4949-a4dc-37d83003eb0a\") " Jul 15 12:26:10.577349 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.577197 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/05c095b7-d1e4-4949-a4dc-37d83003eb0a-istio-csr-dns-cert\") pod \"05c095b7-d1e4-4949-a4dc-37d83003eb0a\" (UID: \"05c095b7-d1e4-4949-a4dc-37d83003eb0a\") " Jul 15 12:26:10.577349 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.577264 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j8spr\" (UniqueName: \"kubernetes.io/projected/05c095b7-d1e4-4949-a4dc-37d83003eb0a-kube-api-access-j8spr\") pod \"05c095b7-d1e4-4949-a4dc-37d83003eb0a\" (UID: \"05c095b7-d1e4-4949-a4dc-37d83003eb0a\") " Jul 15 12:26:10.577349 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.577312 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/05c095b7-d1e4-4949-a4dc-37d83003eb0a-istio-kubeconfig\") pod \"05c095b7-d1e4-4949-a4dc-37d83003eb0a\" (UID: \"05c095b7-d1e4-4949-a4dc-37d83003eb0a\") " Jul 15 12:26:10.577510 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.577354 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05c095b7-d1e4-4949-a4dc-37d83003eb0a-istio-token\") pod \"05c095b7-d1e4-4949-a4dc-37d83003eb0a\" (UID: \"05c095b7-d1e4-4949-a4dc-37d83003eb0a\") " Jul 15 12:26:10.577510 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.577389 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/05c095b7-d1e4-4949-a4dc-37d83003eb0a-cacerts\") pod \"05c095b7-d1e4-4949-a4dc-37d83003eb0a\" (UID: \"05c095b7-d1e4-4949-a4dc-37d83003eb0a\") " Jul 15 12:26:10.577510 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.577413 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/05c095b7-d1e4-4949-a4dc-37d83003eb0a-istio-csr-ca-configmap\") pod \"05c095b7-d1e4-4949-a4dc-37d83003eb0a\" (UID: \"05c095b7-d1e4-4949-a4dc-37d83003eb0a\") " Jul 15 12:26:10.578141 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.578095 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05c095b7-d1e4-4949-a4dc-37d83003eb0a-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "05c095b7-d1e4-4949-a4dc-37d83003eb0a" (UID: "05c095b7-d1e4-4949-a4dc-37d83003eb0a"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:26:10.580098 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.580070 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05c095b7-d1e4-4949-a4dc-37d83003eb0a-istio-token" (OuterVolumeSpecName: "istio-token") pod "05c095b7-d1e4-4949-a4dc-37d83003eb0a" (UID: "05c095b7-d1e4-4949-a4dc-37d83003eb0a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:26:10.580201 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.580118 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05c095b7-d1e4-4949-a4dc-37d83003eb0a-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "05c095b7-d1e4-4949-a4dc-37d83003eb0a" (UID: "05c095b7-d1e4-4949-a4dc-37d83003eb0a"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:26:10.580409 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.580338 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05c095b7-d1e4-4949-a4dc-37d83003eb0a-local-certs" (OuterVolumeSpecName: "local-certs") pod "05c095b7-d1e4-4949-a4dc-37d83003eb0a" (UID: "05c095b7-d1e4-4949-a4dc-37d83003eb0a"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:26:10.580758 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.580737 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05c095b7-d1e4-4949-a4dc-37d83003eb0a-cacerts" (OuterVolumeSpecName: "cacerts") pod "05c095b7-d1e4-4949-a4dc-37d83003eb0a" (UID: "05c095b7-d1e4-4949-a4dc-37d83003eb0a"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:26:10.580833 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.580782 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05c095b7-d1e4-4949-a4dc-37d83003eb0a-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "05c095b7-d1e4-4949-a4dc-37d83003eb0a" (UID: "05c095b7-d1e4-4949-a4dc-37d83003eb0a"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:26:10.580833 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.580794 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05c095b7-d1e4-4949-a4dc-37d83003eb0a-kube-api-access-j8spr" (OuterVolumeSpecName: "kube-api-access-j8spr") pod "05c095b7-d1e4-4949-a4dc-37d83003eb0a" (UID: "05c095b7-d1e4-4949-a4dc-37d83003eb0a"). InnerVolumeSpecName "kube-api-access-j8spr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:26:10.678944 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.678826 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8spr\" (UniqueName: \"kubernetes.io/projected/05c095b7-d1e4-4949-a4dc-37d83003eb0a-kube-api-access-j8spr\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:26:10.678944 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.678858 2560 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/05c095b7-d1e4-4949-a4dc-37d83003eb0a-istio-kubeconfig\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:26:10.678944 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.678874 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05c095b7-d1e4-4949-a4dc-37d83003eb0a-istio-token\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:26:10.678944 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.678889 2560 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/05c095b7-d1e4-4949-a4dc-37d83003eb0a-cacerts\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:26:10.678944 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.678903 2560 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/05c095b7-d1e4-4949-a4dc-37d83003eb0a-istio-csr-ca-configmap\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:26:10.678944 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.678917 2560 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/05c095b7-d1e4-4949-a4dc-37d83003eb0a-local-certs\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:26:10.678944 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.678931 2560 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/05c095b7-d1e4-4949-a4dc-37d83003eb0a-istio-csr-dns-cert\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:26:10.867758 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.867665 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-vrxrj"] Jul 15 12:26:10.868232 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.868209 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05c095b7-d1e4-4949-a4dc-37d83003eb0a" containerName="discovery" Jul 15 12:26:10.868232 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.868234 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="05c095b7-d1e4-4949-a4dc-37d83003eb0a" containerName="discovery" Jul 15 12:26:10.868403 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.868356 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="05c095b7-d1e4-4949-a4dc-37d83003eb0a" containerName="discovery" Jul 15 12:26:10.875197 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.875170 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vrxrj" Jul 15 12:26:10.878084 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.878061 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 15 12:26:10.878199 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.878135 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-m4mlf\"" Jul 15 12:26:10.894072 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.894050 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-vrxrj"] Jul 15 12:26:10.989301 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.989099 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lmr6\" (UniqueName: \"kubernetes.io/projected/baf577c8-f3f4-4d3a-b525-dfa341a0a771-kube-api-access-5lmr6\") pod \"kserve-controller-manager-d6f4b65b8-vrxrj\" (UID: \"baf577c8-f3f4-4d3a-b525-dfa341a0a771\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vrxrj" Jul 15 12:26:10.989301 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:10.989189 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/baf577c8-f3f4-4d3a-b525-dfa341a0a771-cert\") pod \"kserve-controller-manager-d6f4b65b8-vrxrj\" (UID: \"baf577c8-f3f4-4d3a-b525-dfa341a0a771\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vrxrj" Jul 15 12:26:11.028269 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:11.028235 2560 generic.go:356] "Generic (PLEG): container finished" podID="05c095b7-d1e4-4949-a4dc-37d83003eb0a" containerID="bb595b4eaf3dfc343bbb011412f6859e1eae1bdb8cf38b4833ef5eb88b70e836" exitCode=0 Jul 15 12:26:11.028744 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:11.028290 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7" Jul 15 12:26:11.028744 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:11.028326 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7" event={"ID":"05c095b7-d1e4-4949-a4dc-37d83003eb0a","Type":"ContainerDied","Data":"bb595b4eaf3dfc343bbb011412f6859e1eae1bdb8cf38b4833ef5eb88b70e836"} Jul 15 12:26:11.028744 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:11.028377 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7" event={"ID":"05c095b7-d1e4-4949-a4dc-37d83003eb0a","Type":"ContainerDied","Data":"4078ba6715c7ff2ce595091d5a1e370646f8484e9f2c69b336618ab0fa980f4f"} Jul 15 12:26:11.028744 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:11.028407 2560 scope.go:117] "RemoveContainer" containerID="bb595b4eaf3dfc343bbb011412f6859e1eae1bdb8cf38b4833ef5eb88b70e836" Jul 15 12:26:11.072032 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:11.072006 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7"] Jul 15 12:26:11.090285 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:11.090252 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/baf577c8-f3f4-4d3a-b525-dfa341a0a771-cert\") pod \"kserve-controller-manager-d6f4b65b8-vrxrj\" (UID: \"baf577c8-f3f4-4d3a-b525-dfa341a0a771\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vrxrj" Jul 15 12:26:11.090409 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:11.090384 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lmr6\" (UniqueName: \"kubernetes.io/projected/baf577c8-f3f4-4d3a-b525-dfa341a0a771-kube-api-access-5lmr6\") pod \"kserve-controller-manager-d6f4b65b8-vrxrj\" (UID: \"baf577c8-f3f4-4d3a-b525-dfa341a0a771\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vrxrj" Jul 15 12:26:11.093023 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:11.092999 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/baf577c8-f3f4-4d3a-b525-dfa341a0a771-cert\") pod \"kserve-controller-manager-d6f4b65b8-vrxrj\" (UID: \"baf577c8-f3f4-4d3a-b525-dfa341a0a771\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vrxrj" Jul 15 12:26:11.093174 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:11.093158 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-xjkq7"] Jul 15 12:26:11.112003 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:11.111975 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lmr6\" (UniqueName: \"kubernetes.io/projected/baf577c8-f3f4-4d3a-b525-dfa341a0a771-kube-api-access-5lmr6\") pod \"kserve-controller-manager-d6f4b65b8-vrxrj\" (UID: \"baf577c8-f3f4-4d3a-b525-dfa341a0a771\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vrxrj" Jul 15 12:26:11.189881 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:11.189813 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vrxrj" Jul 15 12:26:11.409425 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:11.409391 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05c095b7-d1e4-4949-a4dc-37d83003eb0a" path="/var/lib/kubelet/pods/05c095b7-d1e4-4949-a4dc-37d83003eb0a/volumes" Jul 15 12:26:11.638711 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:11.638686 2560 scope.go:117] "RemoveContainer" containerID="bb595b4eaf3dfc343bbb011412f6859e1eae1bdb8cf38b4833ef5eb88b70e836" Jul 15 12:26:11.639029 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:26:11.639008 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb595b4eaf3dfc343bbb011412f6859e1eae1bdb8cf38b4833ef5eb88b70e836\": container with ID starting with bb595b4eaf3dfc343bbb011412f6859e1eae1bdb8cf38b4833ef5eb88b70e836 not found: ID does not exist" containerID="bb595b4eaf3dfc343bbb011412f6859e1eae1bdb8cf38b4833ef5eb88b70e836" Jul 15 12:26:11.639110 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:11.639037 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb595b4eaf3dfc343bbb011412f6859e1eae1bdb8cf38b4833ef5eb88b70e836"} err="failed to get container status \"bb595b4eaf3dfc343bbb011412f6859e1eae1bdb8cf38b4833ef5eb88b70e836\": rpc error: code = NotFound desc = could not find container \"bb595b4eaf3dfc343bbb011412f6859e1eae1bdb8cf38b4833ef5eb88b70e836\": container with ID starting with bb595b4eaf3dfc343bbb011412f6859e1eae1bdb8cf38b4833ef5eb88b70e836 not found: ID does not exist" Jul 15 12:26:11.793801 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:11.793765 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-vrxrj"] Jul 15 12:26:11.975471 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:26:11.975394 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaf577c8_f3f4_4d3a_b525_dfa341a0a771.slice/crio-607fb713aa9f819fb3b1f6d20c2c9d9c989f9cf0a001425ad69d7f14667697d6 WatchSource:0}: Error finding container 607fb713aa9f819fb3b1f6d20c2c9d9c989f9cf0a001425ad69d7f14667697d6: Status 404 returned error can't find the container with id 607fb713aa9f819fb3b1f6d20c2c9d9c989f9cf0a001425ad69d7f14667697d6 Jul 15 12:26:12.009038 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:12.009003 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:26:12.009130 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:12.009075 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:26:12.009130 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:12.009104 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:26:12.034844 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:12.034822 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vrxrj" event={"ID":"baf577c8-f3f4-4d3a-b525-dfa341a0a771","Type":"ContainerStarted","Data":"607fb713aa9f819fb3b1f6d20c2c9d9c989f9cf0a001425ad69d7f14667697d6"} Jul 15 12:26:13.042599 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:13.042564 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" event={"ID":"c6fa3be8-3559-4e2f-882c-a4bc7b84c55a","Type":"ContainerStarted","Data":"c2c668129ef88657f694cad9c51f938118d80fe3289effd4c7b86df991a53ebf"} Jul 15 12:26:13.094514 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:13.094453 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" podStartSLOduration=1.99288113 podStartE2EDuration="5.09442916s" totalImagesPullingTime="3.10154803s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:26:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:26:13.089566833 +0000 UTC m=+346.230362969" watchObservedRunningTime="2026-07-15 12:26:13.09442916 +0000 UTC m=+346.235225272" Jul 15 12:26:13.735224 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:13.735188 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" Jul 15 12:26:13.740775 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:13.740749 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" Jul 15 12:26:13.983591 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:13.983559 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-5dc4q" Jul 15 12:26:14.048443 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:14.048370 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" Jul 15 12:26:14.049593 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:14.049569 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l2zsr" Jul 15 12:26:15.054365 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:15.054335 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vrxrj" event={"ID":"baf577c8-f3f4-4d3a-b525-dfa341a0a771","Type":"ContainerStarted","Data":"cc35dfc78aeb3b78b761271db9b0ed7d2b5a6bb46b2602dfea59b7aa4fcd7b82"} Jul 15 12:26:15.054764 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:15.054406 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vrxrj" Jul 15 12:26:15.089595 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:15.089528 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vrxrj" podStartSLOduration=2.669693479 podStartE2EDuration="5.089507335s" totalImagesPullingTime="2.419813856s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:26:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:26:15.085033121 +0000 UTC m=+348.225829249" watchObservedRunningTime="2026-07-15 12:26:15.089507335 +0000 UTC m=+348.230303449" Jul 15 12:26:15.998220 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:15.998191 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-gk6xz" Jul 15 12:26:46.065187 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:46.065157 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vrxrj" Jul 15 12:26:58.920554 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:58.920515 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n592n"] Jul 15 12:26:58.924017 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:58.923992 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-n592n" Jul 15 12:26:58.942667 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:58.942625 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 15 12:26:58.942770 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:58.942742 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-pkwk4\"" Jul 15 12:26:58.943954 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:58.943939 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 15 12:26:58.955245 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:58.955220 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n592n"] Jul 15 12:26:59.011336 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:59.011301 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4sv4\" (UniqueName: \"kubernetes.io/projected/83469563-6c7d-4a2d-9df2-fd96e4e1dcd1-kube-api-access-t4sv4\") pod \"kuadrant-operator-catalog-n592n\" (UID: \"83469563-6c7d-4a2d-9df2-fd96e4e1dcd1\") " pod="kuadrant-system/kuadrant-operator-catalog-n592n" Jul 15 12:26:59.112432 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:59.112391 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4sv4\" (UniqueName: \"kubernetes.io/projected/83469563-6c7d-4a2d-9df2-fd96e4e1dcd1-kube-api-access-t4sv4\") pod \"kuadrant-operator-catalog-n592n\" (UID: \"83469563-6c7d-4a2d-9df2-fd96e4e1dcd1\") " pod="kuadrant-system/kuadrant-operator-catalog-n592n" Jul 15 12:26:59.133796 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:59.133762 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4sv4\" (UniqueName: \"kubernetes.io/projected/83469563-6c7d-4a2d-9df2-fd96e4e1dcd1-kube-api-access-t4sv4\") pod \"kuadrant-operator-catalog-n592n\" (UID: \"83469563-6c7d-4a2d-9df2-fd96e4e1dcd1\") " pod="kuadrant-system/kuadrant-operator-catalog-n592n" Jul 15 12:26:59.152530 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:59.152498 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n592n"] Jul 15 12:26:59.152875 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:59.152862 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-n592n" Jul 15 12:26:59.330251 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:59.330226 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n592n"] Jul 15 12:26:59.332793 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:26:59.332764 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83469563_6c7d_4a2d_9df2_fd96e4e1dcd1.slice/crio-d53ea4eb426072956e0ab1dc456ab60c0332edce62c28f84e49eeb022083ef82 WatchSource:0}: Error finding container d53ea4eb426072956e0ab1dc456ab60c0332edce62c28f84e49eeb022083ef82: Status 404 returned error can't find the container with id d53ea4eb426072956e0ab1dc456ab60c0332edce62c28f84e49eeb022083ef82 Jul 15 12:26:59.396442 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:59.396405 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r95g2"] Jul 15 12:26:59.400990 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:59.400973 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r95g2" Jul 15 12:26:59.423301 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:59.423273 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r95g2"] Jul 15 12:26:59.516530 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:59.516450 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqld7\" (UniqueName: \"kubernetes.io/projected/51609fa2-4c5e-4031-8df6-11319aad0f4c-kube-api-access-lqld7\") pod \"kuadrant-operator-catalog-r95g2\" (UID: \"51609fa2-4c5e-4031-8df6-11319aad0f4c\") " pod="kuadrant-system/kuadrant-operator-catalog-r95g2" Jul 15 12:26:59.617618 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:59.617588 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqld7\" (UniqueName: \"kubernetes.io/projected/51609fa2-4c5e-4031-8df6-11319aad0f4c-kube-api-access-lqld7\") pod \"kuadrant-operator-catalog-r95g2\" (UID: \"51609fa2-4c5e-4031-8df6-11319aad0f4c\") " pod="kuadrant-system/kuadrant-operator-catalog-r95g2" Jul 15 12:26:59.630315 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:59.630290 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqld7\" (UniqueName: \"kubernetes.io/projected/51609fa2-4c5e-4031-8df6-11319aad0f4c-kube-api-access-lqld7\") pod \"kuadrant-operator-catalog-r95g2\" (UID: \"51609fa2-4c5e-4031-8df6-11319aad0f4c\") " pod="kuadrant-system/kuadrant-operator-catalog-r95g2" Jul 15 12:26:59.711190 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:59.711152 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r95g2" Jul 15 12:26:59.853572 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:26:59.853487 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r95g2"] Jul 15 12:26:59.855741 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:26:59.855715 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51609fa2_4c5e_4031_8df6_11319aad0f4c.slice/crio-de4dbf00186e1fb76f1d80fd487c90c5c7fb844585dce4b6b50f3a8ddff7e951 WatchSource:0}: Error finding container de4dbf00186e1fb76f1d80fd487c90c5c7fb844585dce4b6b50f3a8ddff7e951: Status 404 returned error can't find the container with id de4dbf00186e1fb76f1d80fd487c90c5c7fb844585dce4b6b50f3a8ddff7e951 Jul 15 12:27:00.277383 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:00.277283 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-n592n" event={"ID":"83469563-6c7d-4a2d-9df2-fd96e4e1dcd1","Type":"ContainerStarted","Data":"d53ea4eb426072956e0ab1dc456ab60c0332edce62c28f84e49eeb022083ef82"} Jul 15 12:27:00.278611 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:00.278586 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r95g2" event={"ID":"51609fa2-4c5e-4031-8df6-11319aad0f4c","Type":"ContainerStarted","Data":"de4dbf00186e1fb76f1d80fd487c90c5c7fb844585dce4b6b50f3a8ddff7e951"} Jul 15 12:27:02.290858 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:02.290815 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r95g2" event={"ID":"51609fa2-4c5e-4031-8df6-11319aad0f4c","Type":"ContainerStarted","Data":"c08369a3468bf57a9e544256401b54a70c7a9582b21d53a35c0d317dd87cc129"} Jul 15 12:27:02.292204 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:02.292179 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-n592n" event={"ID":"83469563-6c7d-4a2d-9df2-fd96e4e1dcd1","Type":"ContainerStarted","Data":"ab51dcf07d5efede65d8f5bd252f0df859cf66c7ef61980e41ba523b763bc603"} Jul 15 12:27:02.292301 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:02.292279 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-n592n" podUID="83469563-6c7d-4a2d-9df2-fd96e4e1dcd1" containerName="registry-server" containerID="cri-o://ab51dcf07d5efede65d8f5bd252f0df859cf66c7ef61980e41ba523b763bc603" gracePeriod=2 Jul 15 12:27:02.331680 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:02.331606 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-r95g2" podStartSLOduration=1.609694177 podStartE2EDuration="3.331588716s" totalImagesPullingTime="1.721894539s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:26:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:27:02.330330637 +0000 UTC m=+395.471126749" watchObservedRunningTime="2026-07-15 12:27:02.331588716 +0000 UTC m=+395.472384826" Jul 15 12:27:02.373807 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:02.373755 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-n592n" podStartSLOduration=2.131571793 podStartE2EDuration="4.373738234s" totalImagesPullingTime="2.242166441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:26:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:27:02.371159623 +0000 UTC m=+395.511955752" watchObservedRunningTime="2026-07-15 12:27:02.373738234 +0000 UTC m=+395.514534345" Jul 15 12:27:02.538391 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:02.538366 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-n592n" Jul 15 12:27:02.643588 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:02.643506 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t4sv4\" (UniqueName: \"kubernetes.io/projected/83469563-6c7d-4a2d-9df2-fd96e4e1dcd1-kube-api-access-t4sv4\") pod \"83469563-6c7d-4a2d-9df2-fd96e4e1dcd1\" (UID: \"83469563-6c7d-4a2d-9df2-fd96e4e1dcd1\") " Jul 15 12:27:02.645618 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:02.645590 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83469563-6c7d-4a2d-9df2-fd96e4e1dcd1-kube-api-access-t4sv4" (OuterVolumeSpecName: "kube-api-access-t4sv4") pod "83469563-6c7d-4a2d-9df2-fd96e4e1dcd1" (UID: "83469563-6c7d-4a2d-9df2-fd96e4e1dcd1"). InnerVolumeSpecName "kube-api-access-t4sv4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:27:02.744612 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:02.744575 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4sv4\" (UniqueName: \"kubernetes.io/projected/83469563-6c7d-4a2d-9df2-fd96e4e1dcd1-kube-api-access-t4sv4\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:27:03.299099 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:03.299065 2560 generic.go:356] "Generic (PLEG): container finished" podID="83469563-6c7d-4a2d-9df2-fd96e4e1dcd1" containerID="ab51dcf07d5efede65d8f5bd252f0df859cf66c7ef61980e41ba523b763bc603" exitCode=0 Jul 15 12:27:03.299513 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:03.299107 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-n592n" Jul 15 12:27:03.299513 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:03.299153 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-n592n" event={"ID":"83469563-6c7d-4a2d-9df2-fd96e4e1dcd1","Type":"ContainerDied","Data":"ab51dcf07d5efede65d8f5bd252f0df859cf66c7ef61980e41ba523b763bc603"} Jul 15 12:27:03.299513 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:03.299201 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-n592n" event={"ID":"83469563-6c7d-4a2d-9df2-fd96e4e1dcd1","Type":"ContainerDied","Data":"d53ea4eb426072956e0ab1dc456ab60c0332edce62c28f84e49eeb022083ef82"} Jul 15 12:27:03.299513 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:03.299226 2560 scope.go:117] "RemoveContainer" containerID="ab51dcf07d5efede65d8f5bd252f0df859cf66c7ef61980e41ba523b763bc603" Jul 15 12:27:03.311535 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:03.310175 2560 scope.go:117] "RemoveContainer" containerID="ab51dcf07d5efede65d8f5bd252f0df859cf66c7ef61980e41ba523b763bc603" Jul 15 12:27:03.311535 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:27:03.310540 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab51dcf07d5efede65d8f5bd252f0df859cf66c7ef61980e41ba523b763bc603\": container with ID starting with ab51dcf07d5efede65d8f5bd252f0df859cf66c7ef61980e41ba523b763bc603 not found: ID does not exist" containerID="ab51dcf07d5efede65d8f5bd252f0df859cf66c7ef61980e41ba523b763bc603" Jul 15 12:27:03.311535 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:03.310577 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab51dcf07d5efede65d8f5bd252f0df859cf66c7ef61980e41ba523b763bc603"} err="failed to get container status \"ab51dcf07d5efede65d8f5bd252f0df859cf66c7ef61980e41ba523b763bc603\": rpc error: code = NotFound desc = could not find container \"ab51dcf07d5efede65d8f5bd252f0df859cf66c7ef61980e41ba523b763bc603\": container with ID starting with ab51dcf07d5efede65d8f5bd252f0df859cf66c7ef61980e41ba523b763bc603 not found: ID does not exist" Jul 15 12:27:03.330061 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:03.330030 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n592n"] Jul 15 12:27:03.335308 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:03.335285 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n592n"] Jul 15 12:27:03.408525 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:03.408493 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83469563-6c7d-4a2d-9df2-fd96e4e1dcd1" path="/var/lib/kubelet/pods/83469563-6c7d-4a2d-9df2-fd96e4e1dcd1/volumes" Jul 15 12:27:09.711511 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:09.711476 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-r95g2" Jul 15 12:27:09.711990 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:09.711549 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-r95g2" Jul 15 12:27:09.734343 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:09.734295 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-r95g2" Jul 15 12:27:10.358066 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:10.358042 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-r95g2" Jul 15 12:27:14.651240 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:14.651205 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8"] Jul 15 12:27:14.651618 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:14.651557 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83469563-6c7d-4a2d-9df2-fd96e4e1dcd1" containerName="registry-server" Jul 15 12:27:14.651618 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:14.651567 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="83469563-6c7d-4a2d-9df2-fd96e4e1dcd1" containerName="registry-server" Jul 15 12:27:14.651714 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:14.651654 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="83469563-6c7d-4a2d-9df2-fd96e4e1dcd1" containerName="registry-server" Jul 15 12:27:14.656192 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:14.656175 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8" Jul 15 12:27:14.660129 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:14.660107 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9hp94\"" Jul 15 12:27:14.677634 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:14.677604 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8"] Jul 15 12:27:14.748055 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:14.748021 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twnvm\" (UniqueName: \"kubernetes.io/projected/77f52d1d-5d55-4e20-90fd-e79abe8c1734-kube-api-access-twnvm\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8\" (UID: \"77f52d1d-5d55-4e20-90fd-e79abe8c1734\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8" Jul 15 12:27:14.748055 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:14.748056 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77f52d1d-5d55-4e20-90fd-e79abe8c1734-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8\" (UID: \"77f52d1d-5d55-4e20-90fd-e79abe8c1734\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8" Jul 15 12:27:14.748252 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:14.748090 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77f52d1d-5d55-4e20-90fd-e79abe8c1734-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8\" (UID: \"77f52d1d-5d55-4e20-90fd-e79abe8c1734\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8" Jul 15 12:27:14.849032 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:14.848997 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twnvm\" (UniqueName: \"kubernetes.io/projected/77f52d1d-5d55-4e20-90fd-e79abe8c1734-kube-api-access-twnvm\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8\" (UID: \"77f52d1d-5d55-4e20-90fd-e79abe8c1734\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8" Jul 15 12:27:14.849032 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:14.849034 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77f52d1d-5d55-4e20-90fd-e79abe8c1734-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8\" (UID: \"77f52d1d-5d55-4e20-90fd-e79abe8c1734\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8" Jul 15 12:27:14.849233 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:14.849065 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77f52d1d-5d55-4e20-90fd-e79abe8c1734-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8\" (UID: \"77f52d1d-5d55-4e20-90fd-e79abe8c1734\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8" Jul 15 12:27:14.849471 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:14.849449 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77f52d1d-5d55-4e20-90fd-e79abe8c1734-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8\" (UID: \"77f52d1d-5d55-4e20-90fd-e79abe8c1734\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8" Jul 15 12:27:14.849508 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:14.849459 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77f52d1d-5d55-4e20-90fd-e79abe8c1734-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8\" (UID: \"77f52d1d-5d55-4e20-90fd-e79abe8c1734\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8" Jul 15 12:27:14.857757 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:14.857728 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twnvm\" (UniqueName: \"kubernetes.io/projected/77f52d1d-5d55-4e20-90fd-e79abe8c1734-kube-api-access-twnvm\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8\" (UID: \"77f52d1d-5d55-4e20-90fd-e79abe8c1734\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8" Jul 15 12:27:14.965458 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:14.965381 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8" Jul 15 12:27:15.101757 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:15.101732 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8"] Jul 15 12:27:15.104056 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:27:15.104032 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77f52d1d_5d55_4e20_90fd_e79abe8c1734.slice/crio-301fd62fda087a4c44a799811c80d81ff798c96c1f3ec5b8aab4a6edec4b1942 WatchSource:0}: Error finding container 301fd62fda087a4c44a799811c80d81ff798c96c1f3ec5b8aab4a6edec4b1942: Status 404 returned error can't find the container with id 301fd62fda087a4c44a799811c80d81ff798c96c1f3ec5b8aab4a6edec4b1942 Jul 15 12:27:15.198627 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:15.198596 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87"] Jul 15 12:27:15.202257 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:15.202238 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87" Jul 15 12:27:15.211840 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:15.211818 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87"] Jul 15 12:27:15.251279 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:15.251205 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fed32aad-0e99-44a1-86cf-43839588c5e2-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87\" (UID: \"fed32aad-0e99-44a1-86cf-43839588c5e2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87" Jul 15 12:27:15.251279 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:15.251244 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqlt7\" (UniqueName: \"kubernetes.io/projected/fed32aad-0e99-44a1-86cf-43839588c5e2-kube-api-access-vqlt7\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87\" (UID: \"fed32aad-0e99-44a1-86cf-43839588c5e2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87" Jul 15 12:27:15.251458 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:15.251349 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fed32aad-0e99-44a1-86cf-43839588c5e2-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87\" (UID: \"fed32aad-0e99-44a1-86cf-43839588c5e2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87" Jul 15 12:27:15.351953 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:15.351927 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fed32aad-0e99-44a1-86cf-43839588c5e2-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87\" (UID: \"fed32aad-0e99-44a1-86cf-43839588c5e2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87" Jul 15 12:27:15.352113 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:15.351964 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqlt7\" (UniqueName: \"kubernetes.io/projected/fed32aad-0e99-44a1-86cf-43839588c5e2-kube-api-access-vqlt7\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87\" (UID: \"fed32aad-0e99-44a1-86cf-43839588c5e2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87" Jul 15 12:27:15.352113 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:15.352031 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fed32aad-0e99-44a1-86cf-43839588c5e2-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87\" (UID: \"fed32aad-0e99-44a1-86cf-43839588c5e2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87" Jul 15 12:27:15.352322 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:15.352304 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fed32aad-0e99-44a1-86cf-43839588c5e2-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87\" (UID: \"fed32aad-0e99-44a1-86cf-43839588c5e2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87" Jul 15 12:27:15.352390 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:15.352351 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fed32aad-0e99-44a1-86cf-43839588c5e2-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87\" (UID: \"fed32aad-0e99-44a1-86cf-43839588c5e2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87" Jul 15 12:27:15.360568 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:15.360544 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqlt7\" (UniqueName: \"kubernetes.io/projected/fed32aad-0e99-44a1-86cf-43839588c5e2-kube-api-access-vqlt7\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87\" (UID: \"fed32aad-0e99-44a1-86cf-43839588c5e2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87" Jul 15 12:27:15.362100 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:15.362071 2560 generic.go:356] "Generic (PLEG): container finished" podID="77f52d1d-5d55-4e20-90fd-e79abe8c1734" containerID="10137232170da0aa5e3ff0edcf9f3bb08a8c08260a064ea3b9d851a446189cf4" exitCode=0 Jul 15 12:27:15.362190 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:15.362116 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8" event={"ID":"77f52d1d-5d55-4e20-90fd-e79abe8c1734","Type":"ContainerDied","Data":"10137232170da0aa5e3ff0edcf9f3bb08a8c08260a064ea3b9d851a446189cf4"} Jul 15 12:27:15.362190 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:15.362168 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8" event={"ID":"77f52d1d-5d55-4e20-90fd-e79abe8c1734","Type":"ContainerStarted","Data":"301fd62fda087a4c44a799811c80d81ff798c96c1f3ec5b8aab4a6edec4b1942"} Jul 15 12:27:15.558217 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:15.558183 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87" Jul 15 12:27:15.617197 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:15.617166 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq"] Jul 15 12:27:15.622761 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:15.622741 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq" Jul 15 12:27:15.631650 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:15.631617 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq"] Jul 15 12:27:15.654490 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:15.654464 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c2ec5a7-8c26-4c79-9118-f8a8af840042-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq\" (UID: \"5c2ec5a7-8c26-4c79-9118-f8a8af840042\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq" Jul 15 12:27:15.654848 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:15.654504 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c2ec5a7-8c26-4c79-9118-f8a8af840042-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq\" (UID: \"5c2ec5a7-8c26-4c79-9118-f8a8af840042\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq" Jul 15 12:27:15.654848 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:15.654536 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bmvm\" (UniqueName: \"kubernetes.io/projected/5c2ec5a7-8c26-4c79-9118-f8a8af840042-kube-api-access-2bmvm\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq\" (UID: \"5c2ec5a7-8c26-4c79-9118-f8a8af840042\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq" Jul 15 12:27:15.695426 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:15.695392 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87"] Jul 15 12:27:15.696048 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:27:15.696020 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfed32aad_0e99_44a1_86cf_43839588c5e2.slice/crio-a1bd7f3d833eac84229d91eac88c6e4b54ee63095c71208136d8683601ec2e5c WatchSource:0}: Error finding container a1bd7f3d833eac84229d91eac88c6e4b54ee63095c71208136d8683601ec2e5c: Status 404 returned error can't find the container with id a1bd7f3d833eac84229d91eac88c6e4b54ee63095c71208136d8683601ec2e5c Jul 15 12:27:15.755156 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:15.755134 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c2ec5a7-8c26-4c79-9118-f8a8af840042-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq\" (UID: \"5c2ec5a7-8c26-4c79-9118-f8a8af840042\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq" Jul 15 12:27:15.755220 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:15.755174 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c2ec5a7-8c26-4c79-9118-f8a8af840042-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq\" (UID: \"5c2ec5a7-8c26-4c79-9118-f8a8af840042\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq" Jul 15 12:27:15.755220 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:15.755199 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bmvm\" (UniqueName: \"kubernetes.io/projected/5c2ec5a7-8c26-4c79-9118-f8a8af840042-kube-api-access-2bmvm\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq\" (UID: \"5c2ec5a7-8c26-4c79-9118-f8a8af840042\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq" Jul 15 12:27:15.755504 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:15.755484 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c2ec5a7-8c26-4c79-9118-f8a8af840042-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq\" (UID: \"5c2ec5a7-8c26-4c79-9118-f8a8af840042\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq" Jul 15 12:27:15.755567 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:15.755520 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c2ec5a7-8c26-4c79-9118-f8a8af840042-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq\" (UID: \"5c2ec5a7-8c26-4c79-9118-f8a8af840042\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq" Jul 15 12:27:15.764581 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:15.764558 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bmvm\" (UniqueName: \"kubernetes.io/projected/5c2ec5a7-8c26-4c79-9118-f8a8af840042-kube-api-access-2bmvm\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq\" (UID: \"5c2ec5a7-8c26-4c79-9118-f8a8af840042\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq" Jul 15 12:27:15.932847 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:15.932816 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq" Jul 15 12:27:16.004042 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:16.004014 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49"] Jul 15 12:27:16.008452 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:16.008427 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49" Jul 15 12:27:16.021175 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:16.020808 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49"] Jul 15 12:27:16.057759 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:16.057731 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/050ecdb1-3d3e-454d-abb1-723157f29c5a-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49\" (UID: \"050ecdb1-3d3e-454d-abb1-723157f29c5a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49" Jul 15 12:27:16.057933 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:16.057862 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/050ecdb1-3d3e-454d-abb1-723157f29c5a-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49\" (UID: \"050ecdb1-3d3e-454d-abb1-723157f29c5a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49" Jul 15 12:27:16.058011 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:16.057937 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hs7mc\" (UniqueName: \"kubernetes.io/projected/050ecdb1-3d3e-454d-abb1-723157f29c5a-kube-api-access-hs7mc\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49\" (UID: \"050ecdb1-3d3e-454d-abb1-723157f29c5a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49" Jul 15 12:27:16.081700 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:16.081670 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq"] Jul 15 12:27:16.111269 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:27:16.111241 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c2ec5a7_8c26_4c79_9118_f8a8af840042.slice/crio-946097d1d1e875b1c799dab991f249dd3e968cedfaa25d3c47ca0fc53e0d5933 WatchSource:0}: Error finding container 946097d1d1e875b1c799dab991f249dd3e968cedfaa25d3c47ca0fc53e0d5933: Status 404 returned error can't find the container with id 946097d1d1e875b1c799dab991f249dd3e968cedfaa25d3c47ca0fc53e0d5933 Jul 15 12:27:16.158351 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:16.158329 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/050ecdb1-3d3e-454d-abb1-723157f29c5a-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49\" (UID: \"050ecdb1-3d3e-454d-abb1-723157f29c5a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49" Jul 15 12:27:16.158485 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:16.158388 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs7mc\" (UniqueName: \"kubernetes.io/projected/050ecdb1-3d3e-454d-abb1-723157f29c5a-kube-api-access-hs7mc\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49\" (UID: \"050ecdb1-3d3e-454d-abb1-723157f29c5a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49" Jul 15 12:27:16.158485 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:16.158447 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/050ecdb1-3d3e-454d-abb1-723157f29c5a-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49\" (UID: \"050ecdb1-3d3e-454d-abb1-723157f29c5a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49" Jul 15 12:27:16.158760 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:16.158741 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/050ecdb1-3d3e-454d-abb1-723157f29c5a-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49\" (UID: \"050ecdb1-3d3e-454d-abb1-723157f29c5a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49" Jul 15 12:27:16.158838 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:16.158773 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/050ecdb1-3d3e-454d-abb1-723157f29c5a-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49\" (UID: \"050ecdb1-3d3e-454d-abb1-723157f29c5a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49" Jul 15 12:27:16.172442 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:16.172419 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hs7mc\" (UniqueName: \"kubernetes.io/projected/050ecdb1-3d3e-454d-abb1-723157f29c5a-kube-api-access-hs7mc\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49\" (UID: \"050ecdb1-3d3e-454d-abb1-723157f29c5a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49" Jul 15 12:27:16.321806 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:16.321780 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49" Jul 15 12:27:16.369785 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:16.369751 2560 generic.go:356] "Generic (PLEG): container finished" podID="fed32aad-0e99-44a1-86cf-43839588c5e2" containerID="0e20e77e07d17b7add75751b7b9f36d545a8ec1980f8744b2fd4a70b9fddba30" exitCode=0 Jul 15 12:27:16.369963 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:16.369789 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87" event={"ID":"fed32aad-0e99-44a1-86cf-43839588c5e2","Type":"ContainerDied","Data":"0e20e77e07d17b7add75751b7b9f36d545a8ec1980f8744b2fd4a70b9fddba30"} Jul 15 12:27:16.369963 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:16.369857 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87" event={"ID":"fed32aad-0e99-44a1-86cf-43839588c5e2","Type":"ContainerStarted","Data":"a1bd7f3d833eac84229d91eac88c6e4b54ee63095c71208136d8683601ec2e5c"} Jul 15 12:27:16.373106 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:16.372089 2560 generic.go:356] "Generic (PLEG): container finished" podID="5c2ec5a7-8c26-4c79-9118-f8a8af840042" containerID="f22f673e99d914a4ea36c815a031a904fd099d4922605e38db04b4819fc25e72" exitCode=0 Jul 15 12:27:16.373106 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:16.372231 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq" event={"ID":"5c2ec5a7-8c26-4c79-9118-f8a8af840042","Type":"ContainerDied","Data":"f22f673e99d914a4ea36c815a031a904fd099d4922605e38db04b4819fc25e72"} Jul 15 12:27:16.373106 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:16.372272 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq" event={"ID":"5c2ec5a7-8c26-4c79-9118-f8a8af840042","Type":"ContainerStarted","Data":"946097d1d1e875b1c799dab991f249dd3e968cedfaa25d3c47ca0fc53e0d5933"} Jul 15 12:27:16.375523 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:16.375493 2560 generic.go:356] "Generic (PLEG): container finished" podID="77f52d1d-5d55-4e20-90fd-e79abe8c1734" containerID="70c353dc4f08704adf44bea8aa503ee46204f3a5ffbfdb50a16f9d76f86d9765" exitCode=0 Jul 15 12:27:16.375595 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:16.375542 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8" event={"ID":"77f52d1d-5d55-4e20-90fd-e79abe8c1734","Type":"ContainerDied","Data":"70c353dc4f08704adf44bea8aa503ee46204f3a5ffbfdb50a16f9d76f86d9765"} Jul 15 12:27:16.468472 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:16.468448 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49"] Jul 15 12:27:16.471149 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:27:16.470621 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod050ecdb1_3d3e_454d_abb1_723157f29c5a.slice/crio-7441ad47e1089e34644bdd6d95b79d737e9ba7debdb60efe37434bf84e28c32d WatchSource:0}: Error finding container 7441ad47e1089e34644bdd6d95b79d737e9ba7debdb60efe37434bf84e28c32d: Status 404 returned error can't find the container with id 7441ad47e1089e34644bdd6d95b79d737e9ba7debdb60efe37434bf84e28c32d Jul 15 12:27:17.382127 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:17.382103 2560 generic.go:356] "Generic (PLEG): container finished" podID="77f52d1d-5d55-4e20-90fd-e79abe8c1734" containerID="04e2b8064e3de912d6f4eb548d85d09f15677b7962491a4e34ddcce1a4541f68" exitCode=0 Jul 15 12:27:17.382448 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:17.382188 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8" event={"ID":"77f52d1d-5d55-4e20-90fd-e79abe8c1734","Type":"ContainerDied","Data":"04e2b8064e3de912d6f4eb548d85d09f15677b7962491a4e34ddcce1a4541f68"} Jul 15 12:27:17.383716 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:17.383694 2560 generic.go:356] "Generic (PLEG): container finished" podID="050ecdb1-3d3e-454d-abb1-723157f29c5a" containerID="cc7eed321f577ae2f86f2e83d15382377fdbf5fb87385a5e628bf8594c94b3f8" exitCode=0 Jul 15 12:27:17.383805 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:17.383772 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49" event={"ID":"050ecdb1-3d3e-454d-abb1-723157f29c5a","Type":"ContainerDied","Data":"cc7eed321f577ae2f86f2e83d15382377fdbf5fb87385a5e628bf8594c94b3f8"} Jul 15 12:27:17.383852 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:17.383812 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49" event={"ID":"050ecdb1-3d3e-454d-abb1-723157f29c5a","Type":"ContainerStarted","Data":"7441ad47e1089e34644bdd6d95b79d737e9ba7debdb60efe37434bf84e28c32d"} Jul 15 12:27:17.385895 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:17.385870 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87" event={"ID":"fed32aad-0e99-44a1-86cf-43839588c5e2","Type":"ContainerStarted","Data":"8d89680fc7aa077d2fd5dc2370d491dc1000b74bdbb1ff8e0c0d65036d7f8080"} Jul 15 12:27:17.387900 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:17.387875 2560 generic.go:356] "Generic (PLEG): container finished" podID="5c2ec5a7-8c26-4c79-9118-f8a8af840042" containerID="d3c336fd90fe2a7ed953fcbaafbdfa2bcdd2474905996c83edce0f0b78e20a3d" exitCode=0 Jul 15 12:27:17.387987 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:17.387906 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq" event={"ID":"5c2ec5a7-8c26-4c79-9118-f8a8af840042","Type":"ContainerDied","Data":"d3c336fd90fe2a7ed953fcbaafbdfa2bcdd2474905996c83edce0f0b78e20a3d"} Jul 15 12:27:18.397718 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:18.397694 2560 generic.go:356] "Generic (PLEG): container finished" podID="050ecdb1-3d3e-454d-abb1-723157f29c5a" containerID="c17a12a92c3e4e9f214d92c47dab881db7f4e114e9b94d8a77467f85d010e441" exitCode=0 Jul 15 12:27:18.398077 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:18.397780 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49" event={"ID":"050ecdb1-3d3e-454d-abb1-723157f29c5a","Type":"ContainerDied","Data":"c17a12a92c3e4e9f214d92c47dab881db7f4e114e9b94d8a77467f85d010e441"} Jul 15 12:27:18.399796 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:18.399775 2560 generic.go:356] "Generic (PLEG): container finished" podID="fed32aad-0e99-44a1-86cf-43839588c5e2" containerID="8d89680fc7aa077d2fd5dc2370d491dc1000b74bdbb1ff8e0c0d65036d7f8080" exitCode=0 Jul 15 12:27:18.399882 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:18.399850 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87" event={"ID":"fed32aad-0e99-44a1-86cf-43839588c5e2","Type":"ContainerDied","Data":"8d89680fc7aa077d2fd5dc2370d491dc1000b74bdbb1ff8e0c0d65036d7f8080"} Jul 15 12:27:18.401972 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:18.401948 2560 generic.go:356] "Generic (PLEG): container finished" podID="5c2ec5a7-8c26-4c79-9118-f8a8af840042" containerID="f0c63a1f085481ddecf5ecb703ab56341d7965104ab96e7a3702b6439642cc32" exitCode=0 Jul 15 12:27:18.402051 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:18.402025 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq" event={"ID":"5c2ec5a7-8c26-4c79-9118-f8a8af840042","Type":"ContainerDied","Data":"f0c63a1f085481ddecf5ecb703ab56341d7965104ab96e7a3702b6439642cc32"} Jul 15 12:27:18.537325 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:18.537298 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8" Jul 15 12:27:18.578699 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:18.578669 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twnvm\" (UniqueName: \"kubernetes.io/projected/77f52d1d-5d55-4e20-90fd-e79abe8c1734-kube-api-access-twnvm\") pod \"77f52d1d-5d55-4e20-90fd-e79abe8c1734\" (UID: \"77f52d1d-5d55-4e20-90fd-e79abe8c1734\") " Jul 15 12:27:18.578843 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:18.578740 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77f52d1d-5d55-4e20-90fd-e79abe8c1734-util\") pod \"77f52d1d-5d55-4e20-90fd-e79abe8c1734\" (UID: \"77f52d1d-5d55-4e20-90fd-e79abe8c1734\") " Jul 15 12:27:18.578843 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:18.578779 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77f52d1d-5d55-4e20-90fd-e79abe8c1734-bundle\") pod \"77f52d1d-5d55-4e20-90fd-e79abe8c1734\" (UID: \"77f52d1d-5d55-4e20-90fd-e79abe8c1734\") " Jul 15 12:27:18.579338 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:18.579304 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77f52d1d-5d55-4e20-90fd-e79abe8c1734-bundle" (OuterVolumeSpecName: "bundle") pod "77f52d1d-5d55-4e20-90fd-e79abe8c1734" (UID: "77f52d1d-5d55-4e20-90fd-e79abe8c1734"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:27:18.580578 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:18.580554 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77f52d1d-5d55-4e20-90fd-e79abe8c1734-kube-api-access-twnvm" (OuterVolumeSpecName: "kube-api-access-twnvm") pod "77f52d1d-5d55-4e20-90fd-e79abe8c1734" (UID: "77f52d1d-5d55-4e20-90fd-e79abe8c1734"). InnerVolumeSpecName "kube-api-access-twnvm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:27:18.584538 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:18.584511 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77f52d1d-5d55-4e20-90fd-e79abe8c1734-util" (OuterVolumeSpecName: "util") pod "77f52d1d-5d55-4e20-90fd-e79abe8c1734" (UID: "77f52d1d-5d55-4e20-90fd-e79abe8c1734"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:27:18.679458 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:18.679425 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twnvm\" (UniqueName: \"kubernetes.io/projected/77f52d1d-5d55-4e20-90fd-e79abe8c1734-kube-api-access-twnvm\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:27:18.679458 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:18.679451 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77f52d1d-5d55-4e20-90fd-e79abe8c1734-util\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:27:18.679458 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:18.679461 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77f52d1d-5d55-4e20-90fd-e79abe8c1734-bundle\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:27:19.410342 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:19.410314 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8" Jul 15 12:27:19.410935 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:19.410312 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8" event={"ID":"77f52d1d-5d55-4e20-90fd-e79abe8c1734","Type":"ContainerDied","Data":"301fd62fda087a4c44a799811c80d81ff798c96c1f3ec5b8aab4a6edec4b1942"} Jul 15 12:27:19.410935 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:19.410438 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="301fd62fda087a4c44a799811c80d81ff798c96c1f3ec5b8aab4a6edec4b1942" Jul 15 12:27:19.412474 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:19.412446 2560 generic.go:356] "Generic (PLEG): container finished" podID="050ecdb1-3d3e-454d-abb1-723157f29c5a" containerID="8055b01098db2ad8957ec2190271c94e466719f1d14f6622a3466c135915a199" exitCode=0 Jul 15 12:27:19.412598 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:19.412529 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49" event={"ID":"050ecdb1-3d3e-454d-abb1-723157f29c5a","Type":"ContainerDied","Data":"8055b01098db2ad8957ec2190271c94e466719f1d14f6622a3466c135915a199"} Jul 15 12:27:19.414655 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:19.414621 2560 generic.go:356] "Generic (PLEG): container finished" podID="fed32aad-0e99-44a1-86cf-43839588c5e2" containerID="8fbbdb9e1d8e476f0b4a87223b5cec03a1f4abe10a29045329576289f33ea0c4" exitCode=0 Jul 15 12:27:19.414751 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:19.414685 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87" event={"ID":"fed32aad-0e99-44a1-86cf-43839588c5e2","Type":"ContainerDied","Data":"8fbbdb9e1d8e476f0b4a87223b5cec03a1f4abe10a29045329576289f33ea0c4"} Jul 15 12:27:19.540348 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:19.540325 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq" Jul 15 12:27:19.585385 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:19.585339 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c2ec5a7-8c26-4c79-9118-f8a8af840042-bundle\") pod \"5c2ec5a7-8c26-4c79-9118-f8a8af840042\" (UID: \"5c2ec5a7-8c26-4c79-9118-f8a8af840042\") " Jul 15 12:27:19.585537 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:19.585434 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c2ec5a7-8c26-4c79-9118-f8a8af840042-util\") pod \"5c2ec5a7-8c26-4c79-9118-f8a8af840042\" (UID: \"5c2ec5a7-8c26-4c79-9118-f8a8af840042\") " Jul 15 12:27:19.585537 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:19.585464 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2bmvm\" (UniqueName: \"kubernetes.io/projected/5c2ec5a7-8c26-4c79-9118-f8a8af840042-kube-api-access-2bmvm\") pod \"5c2ec5a7-8c26-4c79-9118-f8a8af840042\" (UID: \"5c2ec5a7-8c26-4c79-9118-f8a8af840042\") " Jul 15 12:27:19.585927 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:19.585900 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c2ec5a7-8c26-4c79-9118-f8a8af840042-bundle" (OuterVolumeSpecName: "bundle") pod "5c2ec5a7-8c26-4c79-9118-f8a8af840042" (UID: "5c2ec5a7-8c26-4c79-9118-f8a8af840042"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:27:19.587597 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:19.587567 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c2ec5a7-8c26-4c79-9118-f8a8af840042-kube-api-access-2bmvm" (OuterVolumeSpecName: "kube-api-access-2bmvm") pod "5c2ec5a7-8c26-4c79-9118-f8a8af840042" (UID: "5c2ec5a7-8c26-4c79-9118-f8a8af840042"). InnerVolumeSpecName "kube-api-access-2bmvm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:27:19.590326 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:19.590293 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c2ec5a7-8c26-4c79-9118-f8a8af840042-util" (OuterVolumeSpecName: "util") pod "5c2ec5a7-8c26-4c79-9118-f8a8af840042" (UID: "5c2ec5a7-8c26-4c79-9118-f8a8af840042"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:27:19.686944 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:19.686876 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c2ec5a7-8c26-4c79-9118-f8a8af840042-util\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:27:19.686944 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:19.686898 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bmvm\" (UniqueName: \"kubernetes.io/projected/5c2ec5a7-8c26-4c79-9118-f8a8af840042-kube-api-access-2bmvm\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:27:19.686944 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:19.686906 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c2ec5a7-8c26-4c79-9118-f8a8af840042-bundle\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:27:20.422602 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:20.422572 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq" Jul 15 12:27:20.423003 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:20.422607 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq" event={"ID":"5c2ec5a7-8c26-4c79-9118-f8a8af840042","Type":"ContainerDied","Data":"946097d1d1e875b1c799dab991f249dd3e968cedfaa25d3c47ca0fc53e0d5933"} Jul 15 12:27:20.423003 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:20.422675 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="946097d1d1e875b1c799dab991f249dd3e968cedfaa25d3c47ca0fc53e0d5933" Jul 15 12:27:20.584651 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:20.584620 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87" Jul 15 12:27:20.587522 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:20.587496 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49" Jul 15 12:27:20.595556 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:20.595537 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fed32aad-0e99-44a1-86cf-43839588c5e2-util\") pod \"fed32aad-0e99-44a1-86cf-43839588c5e2\" (UID: \"fed32aad-0e99-44a1-86cf-43839588c5e2\") " Jul 15 12:27:20.595710 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:20.595570 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vqlt7\" (UniqueName: \"kubernetes.io/projected/fed32aad-0e99-44a1-86cf-43839588c5e2-kube-api-access-vqlt7\") pod \"fed32aad-0e99-44a1-86cf-43839588c5e2\" (UID: \"fed32aad-0e99-44a1-86cf-43839588c5e2\") " Jul 15 12:27:20.595710 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:20.595603 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/050ecdb1-3d3e-454d-abb1-723157f29c5a-util\") pod \"050ecdb1-3d3e-454d-abb1-723157f29c5a\" (UID: \"050ecdb1-3d3e-454d-abb1-723157f29c5a\") " Jul 15 12:27:20.595710 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:20.595697 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/050ecdb1-3d3e-454d-abb1-723157f29c5a-bundle\") pod \"050ecdb1-3d3e-454d-abb1-723157f29c5a\" (UID: \"050ecdb1-3d3e-454d-abb1-723157f29c5a\") " Jul 15 12:27:20.595875 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:20.595737 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fed32aad-0e99-44a1-86cf-43839588c5e2-bundle\") pod \"fed32aad-0e99-44a1-86cf-43839588c5e2\" (UID: \"fed32aad-0e99-44a1-86cf-43839588c5e2\") " Jul 15 12:27:20.595875 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:20.595808 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hs7mc\" (UniqueName: \"kubernetes.io/projected/050ecdb1-3d3e-454d-abb1-723157f29c5a-kube-api-access-hs7mc\") pod \"050ecdb1-3d3e-454d-abb1-723157f29c5a\" (UID: \"050ecdb1-3d3e-454d-abb1-723157f29c5a\") " Jul 15 12:27:20.596328 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:20.596302 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fed32aad-0e99-44a1-86cf-43839588c5e2-bundle" (OuterVolumeSpecName: "bundle") pod "fed32aad-0e99-44a1-86cf-43839588c5e2" (UID: "fed32aad-0e99-44a1-86cf-43839588c5e2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:27:20.596414 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:20.596360 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/050ecdb1-3d3e-454d-abb1-723157f29c5a-bundle" (OuterVolumeSpecName: "bundle") pod "050ecdb1-3d3e-454d-abb1-723157f29c5a" (UID: "050ecdb1-3d3e-454d-abb1-723157f29c5a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:27:20.597932 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:20.597902 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/050ecdb1-3d3e-454d-abb1-723157f29c5a-kube-api-access-hs7mc" (OuterVolumeSpecName: "kube-api-access-hs7mc") pod "050ecdb1-3d3e-454d-abb1-723157f29c5a" (UID: "050ecdb1-3d3e-454d-abb1-723157f29c5a"). InnerVolumeSpecName "kube-api-access-hs7mc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:27:20.598032 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:20.598009 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fed32aad-0e99-44a1-86cf-43839588c5e2-kube-api-access-vqlt7" (OuterVolumeSpecName: "kube-api-access-vqlt7") pod "fed32aad-0e99-44a1-86cf-43839588c5e2" (UID: "fed32aad-0e99-44a1-86cf-43839588c5e2"). InnerVolumeSpecName "kube-api-access-vqlt7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:27:20.601223 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:20.601185 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fed32aad-0e99-44a1-86cf-43839588c5e2-util" (OuterVolumeSpecName: "util") pod "fed32aad-0e99-44a1-86cf-43839588c5e2" (UID: "fed32aad-0e99-44a1-86cf-43839588c5e2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:27:20.601685 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:20.601665 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/050ecdb1-3d3e-454d-abb1-723157f29c5a-util" (OuterVolumeSpecName: "util") pod "050ecdb1-3d3e-454d-abb1-723157f29c5a" (UID: "050ecdb1-3d3e-454d-abb1-723157f29c5a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:27:20.696684 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:20.696595 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fed32aad-0e99-44a1-86cf-43839588c5e2-util\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:27:20.696684 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:20.696619 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vqlt7\" (UniqueName: \"kubernetes.io/projected/fed32aad-0e99-44a1-86cf-43839588c5e2-kube-api-access-vqlt7\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:27:20.696684 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:20.696629 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/050ecdb1-3d3e-454d-abb1-723157f29c5a-util\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:27:20.696684 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:20.696661 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/050ecdb1-3d3e-454d-abb1-723157f29c5a-bundle\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:27:20.696684 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:20.696678 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fed32aad-0e99-44a1-86cf-43839588c5e2-bundle\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:27:20.696932 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:20.696689 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hs7mc\" (UniqueName: \"kubernetes.io/projected/050ecdb1-3d3e-454d-abb1-723157f29c5a-kube-api-access-hs7mc\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:27:21.429603 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:21.429579 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49" Jul 15 12:27:21.429972 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:21.429582 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49" event={"ID":"050ecdb1-3d3e-454d-abb1-723157f29c5a","Type":"ContainerDied","Data":"7441ad47e1089e34644bdd6d95b79d737e9ba7debdb60efe37434bf84e28c32d"} Jul 15 12:27:21.429972 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:21.429682 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7441ad47e1089e34644bdd6d95b79d737e9ba7debdb60efe37434bf84e28c32d" Jul 15 12:27:21.431698 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:21.431671 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87" event={"ID":"fed32aad-0e99-44a1-86cf-43839588c5e2","Type":"ContainerDied","Data":"a1bd7f3d833eac84229d91eac88c6e4b54ee63095c71208136d8683601ec2e5c"} Jul 15 12:27:21.431797 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:21.431717 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a1bd7f3d833eac84229d91eac88c6e4b54ee63095c71208136d8683601ec2e5c" Jul 15 12:27:21.431860 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:21.431842 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87" Jul 15 12:27:30.907962 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:30.907933 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fndcc"] Jul 15 12:27:30.908354 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:30.908283 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c2ec5a7-8c26-4c79-9118-f8a8af840042" containerName="pull" Jul 15 12:27:30.908354 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:30.908293 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c2ec5a7-8c26-4c79-9118-f8a8af840042" containerName="pull" Jul 15 12:27:30.908354 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:30.908303 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77f52d1d-5d55-4e20-90fd-e79abe8c1734" containerName="extract" Jul 15 12:27:30.908354 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:30.908309 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="77f52d1d-5d55-4e20-90fd-e79abe8c1734" containerName="extract" Jul 15 12:27:30.908354 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:30.908318 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c2ec5a7-8c26-4c79-9118-f8a8af840042" containerName="extract" Jul 15 12:27:30.908354 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:30.908324 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c2ec5a7-8c26-4c79-9118-f8a8af840042" containerName="extract" Jul 15 12:27:30.908354 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:30.908336 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c2ec5a7-8c26-4c79-9118-f8a8af840042" containerName="util" Jul 15 12:27:30.908354 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:30.908340 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c2ec5a7-8c26-4c79-9118-f8a8af840042" containerName="util" Jul 15 12:27:30.908354 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:30.908347 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="050ecdb1-3d3e-454d-abb1-723157f29c5a" containerName="extract" Jul 15 12:27:30.908354 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:30.908353 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="050ecdb1-3d3e-454d-abb1-723157f29c5a" containerName="extract" Jul 15 12:27:30.908704 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:30.908361 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77f52d1d-5d55-4e20-90fd-e79abe8c1734" containerName="pull" Jul 15 12:27:30.908704 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:30.908367 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="77f52d1d-5d55-4e20-90fd-e79abe8c1734" containerName="pull" Jul 15 12:27:30.908704 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:30.908374 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77f52d1d-5d55-4e20-90fd-e79abe8c1734" containerName="util" Jul 15 12:27:30.908704 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:30.908380 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="77f52d1d-5d55-4e20-90fd-e79abe8c1734" containerName="util" Jul 15 12:27:30.908704 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:30.908385 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fed32aad-0e99-44a1-86cf-43839588c5e2" containerName="extract" Jul 15 12:27:30.908704 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:30.908390 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="fed32aad-0e99-44a1-86cf-43839588c5e2" containerName="extract" Jul 15 12:27:30.908704 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:30.908400 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="050ecdb1-3d3e-454d-abb1-723157f29c5a" containerName="util" Jul 15 12:27:30.908704 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:30.908404 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="050ecdb1-3d3e-454d-abb1-723157f29c5a" containerName="util" Jul 15 12:27:30.908704 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:30.908412 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fed32aad-0e99-44a1-86cf-43839588c5e2" containerName="pull" Jul 15 12:27:30.908704 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:30.908416 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="fed32aad-0e99-44a1-86cf-43839588c5e2" containerName="pull" Jul 15 12:27:30.908704 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:30.908422 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fed32aad-0e99-44a1-86cf-43839588c5e2" containerName="util" Jul 15 12:27:30.908704 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:30.908426 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="fed32aad-0e99-44a1-86cf-43839588c5e2" containerName="util" Jul 15 12:27:30.908704 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:30.908432 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="050ecdb1-3d3e-454d-abb1-723157f29c5a" containerName="pull" Jul 15 12:27:30.908704 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:30.908437 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="050ecdb1-3d3e-454d-abb1-723157f29c5a" containerName="pull" Jul 15 12:27:30.908704 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:30.908489 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="050ecdb1-3d3e-454d-abb1-723157f29c5a" containerName="extract" Jul 15 12:27:30.908704 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:30.908497 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c2ec5a7-8c26-4c79-9118-f8a8af840042" containerName="extract" Jul 15 12:27:30.908704 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:30.908503 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="fed32aad-0e99-44a1-86cf-43839588c5e2" containerName="extract" Jul 15 12:27:30.908704 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:30.908509 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="77f52d1d-5d55-4e20-90fd-e79abe8c1734" containerName="extract" Jul 15 12:27:30.912845 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:30.912824 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fndcc" Jul 15 12:27:30.915385 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:30.915363 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-fktlm\"" Jul 15 12:27:30.922581 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:30.922552 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fndcc"] Jul 15 12:27:30.984242 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:30.984211 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dpwq\" (UniqueName: \"kubernetes.io/projected/d34877b9-67a2-451a-9730-71957ab87c0c-kube-api-access-6dpwq\") pod \"limitador-operator-controller-manager-859d7b78bd-fndcc\" (UID: \"d34877b9-67a2-451a-9730-71957ab87c0c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fndcc" Jul 15 12:27:31.085150 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:31.085122 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dpwq\" (UniqueName: \"kubernetes.io/projected/d34877b9-67a2-451a-9730-71957ab87c0c-kube-api-access-6dpwq\") pod \"limitador-operator-controller-manager-859d7b78bd-fndcc\" (UID: \"d34877b9-67a2-451a-9730-71957ab87c0c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fndcc" Jul 15 12:27:31.098414 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:31.098391 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dpwq\" (UniqueName: \"kubernetes.io/projected/d34877b9-67a2-451a-9730-71957ab87c0c-kube-api-access-6dpwq\") pod \"limitador-operator-controller-manager-859d7b78bd-fndcc\" (UID: \"d34877b9-67a2-451a-9730-71957ab87c0c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fndcc" Jul 15 12:27:31.223973 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:31.223893 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fndcc" Jul 15 12:27:31.376346 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:31.376317 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fndcc"] Jul 15 12:27:31.377168 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:27:31.377143 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd34877b9_67a2_451a_9730_71957ab87c0c.slice/crio-c441dfad72bcf33f0f70e9cf964062823e944ac487026bed805aca2a90363419 WatchSource:0}: Error finding container c441dfad72bcf33f0f70e9cf964062823e944ac487026bed805aca2a90363419: Status 404 returned error can't find the container with id c441dfad72bcf33f0f70e9cf964062823e944ac487026bed805aca2a90363419 Jul 15 12:27:31.486388 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:31.486306 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fndcc" event={"ID":"d34877b9-67a2-451a-9730-71957ab87c0c","Type":"ContainerStarted","Data":"c441dfad72bcf33f0f70e9cf964062823e944ac487026bed805aca2a90363419"} Jul 15 12:27:34.509858 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:34.509818 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fndcc" event={"ID":"d34877b9-67a2-451a-9730-71957ab87c0c","Type":"ContainerStarted","Data":"92768906e4b0ec4cdba56c60e915129cc525041dbc0d6d6d07690f518b8250a1"} Jul 15 12:27:34.510219 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:34.509888 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fndcc" Jul 15 12:27:34.546719 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:34.546673 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fndcc" podStartSLOduration=2.481205685 podStartE2EDuration="4.546659002s" totalImagesPullingTime="2.065453317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:27:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:27:34.546450073 +0000 UTC m=+427.687246184" watchObservedRunningTime="2026-07-15 12:27:34.546659002 +0000 UTC m=+427.687455103" Jul 15 12:27:42.533366 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:42.533328 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5dc48cd76c-xmq4w"] Jul 15 12:27:45.517490 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:45.517459 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fndcc" Jul 15 12:27:48.577103 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:48.577061 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-9fplt"] Jul 15 12:27:48.583682 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:48.583662 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9fplt" Jul 15 12:27:48.586174 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:48.586139 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 15 12:27:48.586276 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:48.586184 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9hp94\"" Jul 15 12:27:48.586276 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:48.586193 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 15 12:27:48.590121 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:48.590096 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-9fplt"] Jul 15 12:27:48.748634 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:48.748604 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1dde5304-3dde-4ae0-bb1e-1b9422a3a447-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-9fplt\" (UID: \"1dde5304-3dde-4ae0-bb1e-1b9422a3a447\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9fplt" Jul 15 12:27:48.748837 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:48.748662 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1dde5304-3dde-4ae0-bb1e-1b9422a3a447-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-9fplt\" (UID: \"1dde5304-3dde-4ae0-bb1e-1b9422a3a447\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9fplt" Jul 15 12:27:48.748837 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:48.748742 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tbhf\" (UniqueName: \"kubernetes.io/projected/1dde5304-3dde-4ae0-bb1e-1b9422a3a447-kube-api-access-7tbhf\") pod \"kuadrant-console-plugin-66b5774985-9fplt\" (UID: \"1dde5304-3dde-4ae0-bb1e-1b9422a3a447\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9fplt" Jul 15 12:27:48.850256 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:48.850157 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1dde5304-3dde-4ae0-bb1e-1b9422a3a447-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-9fplt\" (UID: \"1dde5304-3dde-4ae0-bb1e-1b9422a3a447\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9fplt" Jul 15 12:27:48.850256 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:48.850202 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1dde5304-3dde-4ae0-bb1e-1b9422a3a447-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-9fplt\" (UID: \"1dde5304-3dde-4ae0-bb1e-1b9422a3a447\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9fplt" Jul 15 12:27:48.850480 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:27:48.850302 2560 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 15 12:27:48.850480 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:48.850320 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tbhf\" (UniqueName: \"kubernetes.io/projected/1dde5304-3dde-4ae0-bb1e-1b9422a3a447-kube-api-access-7tbhf\") pod \"kuadrant-console-plugin-66b5774985-9fplt\" (UID: \"1dde5304-3dde-4ae0-bb1e-1b9422a3a447\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9fplt" Jul 15 12:27:48.850480 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:27:48.850380 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dde5304-3dde-4ae0-bb1e-1b9422a3a447-plugin-serving-cert podName:1dde5304-3dde-4ae0-bb1e-1b9422a3a447 nodeName:}" failed. No retries permitted until 2026-07-15 12:27:49.350361832 +0000 UTC m=+442.491157921 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/1dde5304-3dde-4ae0-bb1e-1b9422a3a447-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-9fplt" (UID: "1dde5304-3dde-4ae0-bb1e-1b9422a3a447") : secret "plugin-serving-cert" not found Jul 15 12:27:48.850934 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:48.850913 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1dde5304-3dde-4ae0-bb1e-1b9422a3a447-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-9fplt\" (UID: \"1dde5304-3dde-4ae0-bb1e-1b9422a3a447\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9fplt" Jul 15 12:27:48.864184 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:48.864158 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tbhf\" (UniqueName: \"kubernetes.io/projected/1dde5304-3dde-4ae0-bb1e-1b9422a3a447-kube-api-access-7tbhf\") pod \"kuadrant-console-plugin-66b5774985-9fplt\" (UID: \"1dde5304-3dde-4ae0-bb1e-1b9422a3a447\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9fplt" Jul 15 12:27:49.355671 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:49.355628 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1dde5304-3dde-4ae0-bb1e-1b9422a3a447-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-9fplt\" (UID: \"1dde5304-3dde-4ae0-bb1e-1b9422a3a447\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9fplt" Jul 15 12:27:49.357934 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:49.357907 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1dde5304-3dde-4ae0-bb1e-1b9422a3a447-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-9fplt\" (UID: \"1dde5304-3dde-4ae0-bb1e-1b9422a3a447\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9fplt" Jul 15 12:27:49.493745 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:49.493715 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9fplt" Jul 15 12:27:49.629934 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:49.629898 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-9fplt"] Jul 15 12:27:49.631480 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:27:49.631457 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1dde5304_3dde_4ae0_bb1e_1b9422a3a447.slice/crio-6ec8e6577eb5edf5b2474ed1ad21dcf52bbcae25426afdfcda9172a7c0f360c0 WatchSource:0}: Error finding container 6ec8e6577eb5edf5b2474ed1ad21dcf52bbcae25426afdfcda9172a7c0f360c0: Status 404 returned error can't find the container with id 6ec8e6577eb5edf5b2474ed1ad21dcf52bbcae25426afdfcda9172a7c0f360c0 Jul 15 12:27:50.597421 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:50.597386 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9fplt" event={"ID":"1dde5304-3dde-4ae0-bb1e-1b9422a3a447","Type":"ContainerStarted","Data":"6ec8e6577eb5edf5b2474ed1ad21dcf52bbcae25426afdfcda9172a7c0f360c0"} Jul 15 12:27:59.720011 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:59.719978 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fndcc"] Jul 15 12:27:59.720466 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:59.720273 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fndcc" podUID="d34877b9-67a2-451a-9730-71957ab87c0c" containerName="manager" containerID="cri-o://92768906e4b0ec4cdba56c60e915129cc525041dbc0d6d6d07690f518b8250a1" gracePeriod=2 Jul 15 12:27:59.743089 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:59.742631 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fndcc"] Jul 15 12:27:59.772356 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:27:59.772312 2560 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"d34877b9-67a2-451a-9730-71957ab87c0c\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-fndcc\": pods \"limitador-operator-controller-manager-859d7b78bd-fndcc\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fndcc" Jul 15 12:27:59.774427 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:59.774370 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8ht4"] Jul 15 12:27:59.775148 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:59.775125 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d34877b9-67a2-451a-9730-71957ab87c0c" containerName="manager" Jul 15 12:27:59.775148 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:59.775148 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d34877b9-67a2-451a-9730-71957ab87c0c" containerName="manager" Jul 15 12:27:59.775329 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:59.775236 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d34877b9-67a2-451a-9730-71957ab87c0c" containerName="manager" Jul 15 12:27:59.779714 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:59.779686 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8ht4" Jul 15 12:27:59.783182 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:27:59.783142 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-fndcc\" is forbidden: User \"system:node:ip-10-0-139-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-188.ec2.internal' and this object" podUID="d34877b9-67a2-451a-9730-71957ab87c0c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fndcc" Jul 15 12:27:59.800515 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:59.800488 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8ht4"] Jul 15 12:27:59.860548 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:59.860514 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94wt8\" (UniqueName: \"kubernetes.io/projected/4f16448a-6aa2-425b-9449-138af341ded4-kube-api-access-94wt8\") pod \"limitador-operator-controller-manager-859d7b78bd-m8ht4\" (UID: \"4f16448a-6aa2-425b-9449-138af341ded4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8ht4" Jul 15 12:27:59.961104 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:59.961071 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94wt8\" (UniqueName: \"kubernetes.io/projected/4f16448a-6aa2-425b-9449-138af341ded4-kube-api-access-94wt8\") pod \"limitador-operator-controller-manager-859d7b78bd-m8ht4\" (UID: \"4f16448a-6aa2-425b-9449-138af341ded4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8ht4" Jul 15 12:27:59.994950 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:27:59.994874 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94wt8\" (UniqueName: \"kubernetes.io/projected/4f16448a-6aa2-425b-9449-138af341ded4-kube-api-access-94wt8\") pod \"limitador-operator-controller-manager-859d7b78bd-m8ht4\" (UID: \"4f16448a-6aa2-425b-9449-138af341ded4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8ht4" Jul 15 12:28:00.093733 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:00.093696 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8ht4" Jul 15 12:28:01.853210 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:01.853178 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjqjw"] Jul 15 12:28:01.877908 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:01.877875 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjqjw" Jul 15 12:28:01.882018 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:01.881989 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-htfqg\"" Jul 15 12:28:01.912932 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:01.912903 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjqjw"] Jul 15 12:28:01.979119 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:01.979079 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-km8qs\" (UniqueName: \"kubernetes.io/projected/5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce-kube-api-access-km8qs\") pod \"kuadrant-operator-controller-manager-86878c769f-bjqjw\" (UID: \"5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjqjw" Jul 15 12:28:01.979304 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:01.979189 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bjqjw\" (UID: \"5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjqjw" Jul 15 12:28:02.080003 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:02.079972 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-km8qs\" (UniqueName: \"kubernetes.io/projected/5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce-kube-api-access-km8qs\") pod \"kuadrant-operator-controller-manager-86878c769f-bjqjw\" (UID: \"5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjqjw" Jul 15 12:28:02.080183 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:02.080030 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bjqjw\" (UID: \"5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjqjw" Jul 15 12:28:02.080409 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:02.080377 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bjqjw\" (UID: \"5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjqjw" Jul 15 12:28:02.117274 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:02.117195 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-km8qs\" (UniqueName: \"kubernetes.io/projected/5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce-kube-api-access-km8qs\") pod \"kuadrant-operator-controller-manager-86878c769f-bjqjw\" (UID: \"5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjqjw" Jul 15 12:28:02.190973 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:02.190933 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjqjw" Jul 15 12:28:07.555560 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:07.555515 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5dc48cd76c-xmq4w" podUID="1bb4941f-99e8-4744-9f8e-088413d7372c" containerName="console" containerID="cri-o://b1d18fc25254050a11a3281e844e3f6e6546f509e6531422ba506c31f7e0744f" gracePeriod=15 Jul 15 12:28:11.289991 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:11.289953 2560 patch_prober.go:28] interesting pod/console-5dc48cd76c-xmq4w container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.133.0.17:8443/health\": dial tcp 10.133.0.17:8443: connect: connection refused" start-of-body= Jul 15 12:28:11.290419 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:11.290015 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-5dc48cd76c-xmq4w" podUID="1bb4941f-99e8-4744-9f8e-088413d7372c" containerName="console" probeResult="failure" output="Get \"https://10.133.0.17:8443/health\": dial tcp 10.133.0.17:8443: connect: connection refused" Jul 15 12:28:13.247522 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.247493 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fndcc" Jul 15 12:28:13.250398 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:28:13.250285 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-fndcc\" is forbidden: User \"system:node:ip-10-0-139-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-188.ec2.internal' and this object" podUID="d34877b9-67a2-451a-9730-71957ab87c0c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fndcc" Jul 15 12:28:13.250826 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.250812 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dc48cd76c-xmq4w" Jul 15 12:28:13.258829 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:28:13.258805 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-fndcc\" is forbidden: User \"system:node:ip-10-0-139-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-188.ec2.internal' and this object" podUID="d34877b9-67a2-451a-9730-71957ab87c0c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fndcc" Jul 15 12:28:13.284804 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.284783 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8ht4"] Jul 15 12:28:13.286426 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:28:13.286402 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f16448a_6aa2_425b_9449_138af341ded4.slice/crio-54dff325f3a6909cdff9ce6eb36de8e5ce57bc477385dd10fdd66a0702610b79 WatchSource:0}: Error finding container 54dff325f3a6909cdff9ce6eb36de8e5ce57bc477385dd10fdd66a0702610b79: Status 404 returned error can't find the container with id 54dff325f3a6909cdff9ce6eb36de8e5ce57bc477385dd10fdd66a0702610b79 Jul 15 12:28:13.293269 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.293247 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dpwq\" (UniqueName: \"kubernetes.io/projected/d34877b9-67a2-451a-9730-71957ab87c0c-kube-api-access-6dpwq\") pod \"d34877b9-67a2-451a-9730-71957ab87c0c\" (UID: \"d34877b9-67a2-451a-9730-71957ab87c0c\") " Jul 15 12:28:13.293446 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.293293 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1bb4941f-99e8-4744-9f8e-088413d7372c-console-serving-cert\") pod \"1bb4941f-99e8-4744-9f8e-088413d7372c\" (UID: \"1bb4941f-99e8-4744-9f8e-088413d7372c\") " Jul 15 12:28:13.293446 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.293321 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1bb4941f-99e8-4744-9f8e-088413d7372c-console-config\") pod \"1bb4941f-99e8-4744-9f8e-088413d7372c\" (UID: \"1bb4941f-99e8-4744-9f8e-088413d7372c\") " Jul 15 12:28:13.293446 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.293363 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1bb4941f-99e8-4744-9f8e-088413d7372c-oauth-serving-cert\") pod \"1bb4941f-99e8-4744-9f8e-088413d7372c\" (UID: \"1bb4941f-99e8-4744-9f8e-088413d7372c\") " Jul 15 12:28:13.293446 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.293386 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bb4941f-99e8-4744-9f8e-088413d7372c-trusted-ca-bundle\") pod \"1bb4941f-99e8-4744-9f8e-088413d7372c\" (UID: \"1bb4941f-99e8-4744-9f8e-088413d7372c\") " Jul 15 12:28:13.293660 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.293456 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1bb4941f-99e8-4744-9f8e-088413d7372c-console-oauth-config\") pod \"1bb4941f-99e8-4744-9f8e-088413d7372c\" (UID: \"1bb4941f-99e8-4744-9f8e-088413d7372c\") " Jul 15 12:28:13.293660 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.293494 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1bb4941f-99e8-4744-9f8e-088413d7372c-service-ca\") pod \"1bb4941f-99e8-4744-9f8e-088413d7372c\" (UID: \"1bb4941f-99e8-4744-9f8e-088413d7372c\") " Jul 15 12:28:13.293660 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.293565 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vgr9\" (UniqueName: \"kubernetes.io/projected/1bb4941f-99e8-4744-9f8e-088413d7372c-kube-api-access-8vgr9\") pod \"1bb4941f-99e8-4744-9f8e-088413d7372c\" (UID: \"1bb4941f-99e8-4744-9f8e-088413d7372c\") " Jul 15 12:28:13.293861 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.293759 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bb4941f-99e8-4744-9f8e-088413d7372c-console-config" (OuterVolumeSpecName: "console-config") pod "1bb4941f-99e8-4744-9f8e-088413d7372c" (UID: "1bb4941f-99e8-4744-9f8e-088413d7372c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:28:13.293922 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.293867 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bb4941f-99e8-4744-9f8e-088413d7372c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1bb4941f-99e8-4744-9f8e-088413d7372c" (UID: "1bb4941f-99e8-4744-9f8e-088413d7372c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:28:13.293922 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.293899 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1bb4941f-99e8-4744-9f8e-088413d7372c-console-config\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:28:13.294170 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.294139 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bb4941f-99e8-4744-9f8e-088413d7372c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1bb4941f-99e8-4744-9f8e-088413d7372c" (UID: "1bb4941f-99e8-4744-9f8e-088413d7372c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:28:13.294294 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.294243 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bb4941f-99e8-4744-9f8e-088413d7372c-service-ca" (OuterVolumeSpecName: "service-ca") pod "1bb4941f-99e8-4744-9f8e-088413d7372c" (UID: "1bb4941f-99e8-4744-9f8e-088413d7372c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:28:13.295691 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.295632 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d34877b9-67a2-451a-9730-71957ab87c0c-kube-api-access-6dpwq" (OuterVolumeSpecName: "kube-api-access-6dpwq") pod "d34877b9-67a2-451a-9730-71957ab87c0c" (UID: "d34877b9-67a2-451a-9730-71957ab87c0c"). InnerVolumeSpecName "kube-api-access-6dpwq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:28:13.296033 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.295982 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1bb4941f-99e8-4744-9f8e-088413d7372c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1bb4941f-99e8-4744-9f8e-088413d7372c" (UID: "1bb4941f-99e8-4744-9f8e-088413d7372c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:28:13.296687 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.296622 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1bb4941f-99e8-4744-9f8e-088413d7372c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1bb4941f-99e8-4744-9f8e-088413d7372c" (UID: "1bb4941f-99e8-4744-9f8e-088413d7372c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:28:13.296914 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.296884 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bb4941f-99e8-4744-9f8e-088413d7372c-kube-api-access-8vgr9" (OuterVolumeSpecName: "kube-api-access-8vgr9") pod "1bb4941f-99e8-4744-9f8e-088413d7372c" (UID: "1bb4941f-99e8-4744-9f8e-088413d7372c"). InnerVolumeSpecName "kube-api-access-8vgr9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:28:13.304410 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.304387 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjqjw"] Jul 15 12:28:13.307722 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:28:13.307700 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b6d05b1_b2f9_4bb7_9615_7bca4b23c7ce.slice/crio-6566b4e5a6b1cd5d5b096ef3293e6b8c49773022fd3643dd114c7ebb4f1e2e39 WatchSource:0}: Error finding container 6566b4e5a6b1cd5d5b096ef3293e6b8c49773022fd3643dd114c7ebb4f1e2e39: Status 404 returned error can't find the container with id 6566b4e5a6b1cd5d5b096ef3293e6b8c49773022fd3643dd114c7ebb4f1e2e39 Jul 15 12:28:13.394977 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.394942 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1bb4941f-99e8-4744-9f8e-088413d7372c-console-oauth-config\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:28:13.394977 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.394969 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1bb4941f-99e8-4744-9f8e-088413d7372c-service-ca\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:28:13.394977 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.394979 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vgr9\" (UniqueName: \"kubernetes.io/projected/1bb4941f-99e8-4744-9f8e-088413d7372c-kube-api-access-8vgr9\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:28:13.395190 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.394987 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dpwq\" (UniqueName: \"kubernetes.io/projected/d34877b9-67a2-451a-9730-71957ab87c0c-kube-api-access-6dpwq\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:28:13.395190 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.394997 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1bb4941f-99e8-4744-9f8e-088413d7372c-console-serving-cert\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:28:13.395190 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.395006 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1bb4941f-99e8-4744-9f8e-088413d7372c-oauth-serving-cert\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:28:13.395190 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.395013 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bb4941f-99e8-4744-9f8e-088413d7372c-trusted-ca-bundle\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:28:13.409756 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.409728 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d34877b9-67a2-451a-9730-71957ab87c0c" path="/var/lib/kubelet/pods/d34877b9-67a2-451a-9730-71957ab87c0c/volumes" Jul 15 12:28:13.736562 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.736484 2560 generic.go:356] "Generic (PLEG): container finished" podID="1bb4941f-99e8-4744-9f8e-088413d7372c" containerID="b1d18fc25254050a11a3281e844e3f6e6546f509e6531422ba506c31f7e0744f" exitCode=0 Jul 15 12:28:13.736562 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.736523 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dc48cd76c-xmq4w" Jul 15 12:28:13.736877 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.736562 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dc48cd76c-xmq4w" event={"ID":"1bb4941f-99e8-4744-9f8e-088413d7372c","Type":"ContainerDied","Data":"b1d18fc25254050a11a3281e844e3f6e6546f509e6531422ba506c31f7e0744f"} Jul 15 12:28:13.736877 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.736610 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dc48cd76c-xmq4w" event={"ID":"1bb4941f-99e8-4744-9f8e-088413d7372c","Type":"ContainerDied","Data":"f1c23203eb19f630ba3b8896ef27a29a5c5d60b540eb7cfb99bd2e048e8895a7"} Jul 15 12:28:13.736877 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.736662 2560 scope.go:117] "RemoveContainer" containerID="b1d18fc25254050a11a3281e844e3f6e6546f509e6531422ba506c31f7e0744f" Jul 15 12:28:13.738414 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.738387 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9fplt" event={"ID":"1dde5304-3dde-4ae0-bb1e-1b9422a3a447","Type":"ContainerStarted","Data":"b2a5c671c72170fd636f12f2112f385765e9a1b222cbe81653ce26942d69e975"} Jul 15 12:28:13.740245 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.740218 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8ht4" event={"ID":"4f16448a-6aa2-425b-9449-138af341ded4","Type":"ContainerStarted","Data":"9fe405f7d338960f3540b73bd03ad0c5ccfdfd418babac35e638f233f4a4d3a7"} Jul 15 12:28:13.740364 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.740265 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8ht4" Jul 15 12:28:13.740364 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.740289 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8ht4" event={"ID":"4f16448a-6aa2-425b-9449-138af341ded4","Type":"ContainerStarted","Data":"54dff325f3a6909cdff9ce6eb36de8e5ce57bc477385dd10fdd66a0702610b79"} Jul 15 12:28:13.741423 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.741399 2560 generic.go:356] "Generic (PLEG): container finished" podID="d34877b9-67a2-451a-9730-71957ab87c0c" containerID="92768906e4b0ec4cdba56c60e915129cc525041dbc0d6d6d07690f518b8250a1" exitCode=0 Jul 15 12:28:13.741528 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.741432 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fndcc" Jul 15 12:28:13.742869 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.742844 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjqjw" event={"ID":"5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce","Type":"ContainerStarted","Data":"6566b4e5a6b1cd5d5b096ef3293e6b8c49773022fd3643dd114c7ebb4f1e2e39"} Jul 15 12:28:13.744117 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:28:13.744093 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-fndcc\" is forbidden: User \"system:node:ip-10-0-139-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-188.ec2.internal' and this object" podUID="d34877b9-67a2-451a-9730-71957ab87c0c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fndcc" Jul 15 12:28:13.746255 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.746238 2560 scope.go:117] "RemoveContainer" containerID="b1d18fc25254050a11a3281e844e3f6e6546f509e6531422ba506c31f7e0744f" Jul 15 12:28:13.746497 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:28:13.746478 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1d18fc25254050a11a3281e844e3f6e6546f509e6531422ba506c31f7e0744f\": container with ID starting with b1d18fc25254050a11a3281e844e3f6e6546f509e6531422ba506c31f7e0744f not found: ID does not exist" containerID="b1d18fc25254050a11a3281e844e3f6e6546f509e6531422ba506c31f7e0744f" Jul 15 12:28:13.746542 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.746504 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1d18fc25254050a11a3281e844e3f6e6546f509e6531422ba506c31f7e0744f"} err="failed to get container status \"b1d18fc25254050a11a3281e844e3f6e6546f509e6531422ba506c31f7e0744f\": rpc error: code = NotFound desc = could not find container \"b1d18fc25254050a11a3281e844e3f6e6546f509e6531422ba506c31f7e0744f\": container with ID starting with b1d18fc25254050a11a3281e844e3f6e6546f509e6531422ba506c31f7e0744f not found: ID does not exist" Jul 15 12:28:13.746542 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.746520 2560 scope.go:117] "RemoveContainer" containerID="92768906e4b0ec4cdba56c60e915129cc525041dbc0d6d6d07690f518b8250a1" Jul 15 12:28:13.753909 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.753893 2560 scope.go:117] "RemoveContainer" containerID="92768906e4b0ec4cdba56c60e915129cc525041dbc0d6d6d07690f518b8250a1" Jul 15 12:28:13.754148 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:28:13.754129 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92768906e4b0ec4cdba56c60e915129cc525041dbc0d6d6d07690f518b8250a1\": container with ID starting with 92768906e4b0ec4cdba56c60e915129cc525041dbc0d6d6d07690f518b8250a1 not found: ID does not exist" containerID="92768906e4b0ec4cdba56c60e915129cc525041dbc0d6d6d07690f518b8250a1" Jul 15 12:28:13.754192 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.754155 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92768906e4b0ec4cdba56c60e915129cc525041dbc0d6d6d07690f518b8250a1"} err="failed to get container status \"92768906e4b0ec4cdba56c60e915129cc525041dbc0d6d6d07690f518b8250a1\": rpc error: code = NotFound desc = could not find container \"92768906e4b0ec4cdba56c60e915129cc525041dbc0d6d6d07690f518b8250a1\": container with ID starting with 92768906e4b0ec4cdba56c60e915129cc525041dbc0d6d6d07690f518b8250a1 not found: ID does not exist" Jul 15 12:28:13.771862 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.771821 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8ht4" podStartSLOduration=14.771805265 podStartE2EDuration="14.771805265s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:27:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:28:13.770475185 +0000 UTC m=+466.911271295" watchObservedRunningTime="2026-07-15 12:28:13.771805265 +0000 UTC m=+466.912601375" Jul 15 12:28:13.808013 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.807981 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5dc48cd76c-xmq4w"] Jul 15 12:28:13.816994 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.816970 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5dc48cd76c-xmq4w"] Jul 15 12:28:13.837500 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:13.837450 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9fplt" podStartSLOduration=2.292951317 podStartE2EDuration="25.837434928s" totalImagesPullingTime="23.544483611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:27:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:28:13.836682746 +0000 UTC m=+466.977478857" watchObservedRunningTime="2026-07-15 12:28:13.837434928 +0000 UTC m=+466.978231039" Jul 15 12:28:13.839974 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:28:13.839942 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-fndcc\" is forbidden: User \"system:node:ip-10-0-139-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-188.ec2.internal' and this object" podUID="d34877b9-67a2-451a-9730-71957ab87c0c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fndcc" Jul 15 12:28:15.411406 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:15.411368 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1bb4941f-99e8-4744-9f8e-088413d7372c" path="/var/lib/kubelet/pods/1bb4941f-99e8-4744-9f8e-088413d7372c/volumes" Jul 15 12:28:18.778223 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:18.778184 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjqjw" event={"ID":"5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce","Type":"ContainerStarted","Data":"d6741e1d9b0565057cab8215d29936d1d5a5cbc1ec60fb7ac547466dd3314984"} Jul 15 12:28:18.778606 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:18.778277 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjqjw" Jul 15 12:28:18.820432 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:18.820379 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjqjw" podStartSLOduration=13.107862249 podStartE2EDuration="17.820364038s" totalImagesPullingTime="4.712501789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:28:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:28:18.816398684 +0000 UTC m=+471.957194795" watchObservedRunningTime="2026-07-15 12:28:18.820364038 +0000 UTC m=+471.961160149" Jul 15 12:28:24.753613 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:24.753578 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8ht4" Jul 15 12:28:29.785628 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:28:29.785600 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjqjw" Jul 15 12:30:12.249314 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:12.249280 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-fbfx6"] Jul 15 12:30:12.249752 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:12.249702 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bb4941f-99e8-4744-9f8e-088413d7372c" containerName="console" Jul 15 12:30:12.249752 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:12.249715 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bb4941f-99e8-4744-9f8e-088413d7372c" containerName="console" Jul 15 12:30:12.249838 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:12.249824 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bb4941f-99e8-4744-9f8e-088413d7372c" containerName="console" Jul 15 12:30:12.252792 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:12.252776 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-fbfx6" Jul 15 12:30:12.255342 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:12.255322 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-bfqhx\"" Jul 15 12:30:12.255463 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:12.255438 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 15 12:30:12.267789 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:12.267762 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-fbfx6"] Jul 15 12:30:12.349041 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:12.349006 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ffbbb5d-6dd5-426e-ab11-2678758860b8-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-fbfx6\" (UID: \"9ffbbb5d-6dd5-426e-ab11-2678758860b8\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-fbfx6" Jul 15 12:30:12.349193 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:12.349073 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvj59\" (UniqueName: \"kubernetes.io/projected/9ffbbb5d-6dd5-426e-ab11-2678758860b8-kube-api-access-vvj59\") pod \"payload-processing-f9b8d9b9b-fbfx6\" (UID: \"9ffbbb5d-6dd5-426e-ab11-2678758860b8\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-fbfx6" Jul 15 12:30:12.450491 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:12.450455 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ffbbb5d-6dd5-426e-ab11-2678758860b8-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-fbfx6\" (UID: \"9ffbbb5d-6dd5-426e-ab11-2678758860b8\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-fbfx6" Jul 15 12:30:12.450724 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:12.450518 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvj59\" (UniqueName: \"kubernetes.io/projected/9ffbbb5d-6dd5-426e-ab11-2678758860b8-kube-api-access-vvj59\") pod \"payload-processing-f9b8d9b9b-fbfx6\" (UID: \"9ffbbb5d-6dd5-426e-ab11-2678758860b8\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-fbfx6" Jul 15 12:30:12.451261 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:12.451235 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ffbbb5d-6dd5-426e-ab11-2678758860b8-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-fbfx6\" (UID: \"9ffbbb5d-6dd5-426e-ab11-2678758860b8\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-fbfx6" Jul 15 12:30:12.459526 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:12.459505 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvj59\" (UniqueName: \"kubernetes.io/projected/9ffbbb5d-6dd5-426e-ab11-2678758860b8-kube-api-access-vvj59\") pod \"payload-processing-f9b8d9b9b-fbfx6\" (UID: \"9ffbbb5d-6dd5-426e-ab11-2678758860b8\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-fbfx6" Jul 15 12:30:12.563788 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:12.563754 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-fbfx6" Jul 15 12:30:12.710710 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:12.710600 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-fbfx6"] Jul 15 12:30:12.713062 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:30:12.713028 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ffbbb5d_6dd5_426e_ab11_2678758860b8.slice/crio-d0071da2fa52fc70ad25ab8d796c864689aa2c3ae2f2650e3e0a800115b52642 WatchSource:0}: Error finding container d0071da2fa52fc70ad25ab8d796c864689aa2c3ae2f2650e3e0a800115b52642: Status 404 returned error can't find the container with id d0071da2fa52fc70ad25ab8d796c864689aa2c3ae2f2650e3e0a800115b52642 Jul 15 12:30:13.388827 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:13.388796 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-fbfx6" event={"ID":"9ffbbb5d-6dd5-426e-ab11-2678758860b8","Type":"ContainerStarted","Data":"d0071da2fa52fc70ad25ab8d796c864689aa2c3ae2f2650e3e0a800115b52642"} Jul 15 12:30:14.396023 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:14.395984 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-fbfx6" event={"ID":"9ffbbb5d-6dd5-426e-ab11-2678758860b8","Type":"ContainerStarted","Data":"10f21859b74e9b8d4b0ad008bc4c60e3f72d51bc755988cecd9ed1b2e03a143e"} Jul 15 12:30:14.396428 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:14.396094 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-fbfx6" Jul 15 12:30:14.415003 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:14.414961 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-fbfx6" podStartSLOduration=0.942059818 podStartE2EDuration="2.41494718s" totalImagesPullingTime="1.472887362s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:30:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:30:14.413616828 +0000 UTC m=+587.554412967" watchObservedRunningTime="2026-07-15 12:30:14.41494718 +0000 UTC m=+587.555743291" Jul 15 12:30:25.409052 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:25.409026 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-fbfx6" Jul 15 12:30:27.297844 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:27.297814 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bg9fm_808669ad-89bb-473a-86e9-7562fcd99736/ovn-acl-logging/0.log" Jul 15 12:30:27.298816 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:27.298797 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bg9fm_808669ad-89bb-473a-86e9-7562fcd99736/ovn-acl-logging/0.log" Jul 15 12:30:41.851500 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:41.851469 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-qw62t"] Jul 15 12:30:41.854771 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:41.854756 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qw62t" Jul 15 12:30:41.869133 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:41.869113 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 15 12:30:41.875061 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:41.875042 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 15 12:30:41.875157 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:41.875089 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 15 12:30:41.875157 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:41.875144 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-f96dj\"" Jul 15 12:30:41.888987 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:41.888955 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/437dc02b-4909-4003-b9f4-8fac0bba4215-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qw62t\" (UID: \"437dc02b-4909-4003-b9f4-8fac0bba4215\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qw62t" Jul 15 12:30:41.888987 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:41.888991 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/437dc02b-4909-4003-b9f4-8fac0bba4215-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qw62t\" (UID: \"437dc02b-4909-4003-b9f4-8fac0bba4215\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qw62t" Jul 15 12:30:41.889171 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:41.889057 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/437dc02b-4909-4003-b9f4-8fac0bba4215-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qw62t\" (UID: \"437dc02b-4909-4003-b9f4-8fac0bba4215\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qw62t" Jul 15 12:30:41.889171 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:41.889117 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxkgs\" (UniqueName: \"kubernetes.io/projected/437dc02b-4909-4003-b9f4-8fac0bba4215-kube-api-access-nxkgs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qw62t\" (UID: \"437dc02b-4909-4003-b9f4-8fac0bba4215\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qw62t" Jul 15 12:30:41.889171 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:41.889152 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/437dc02b-4909-4003-b9f4-8fac0bba4215-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qw62t\" (UID: \"437dc02b-4909-4003-b9f4-8fac0bba4215\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qw62t" Jul 15 12:30:41.889320 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:41.889215 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/437dc02b-4909-4003-b9f4-8fac0bba4215-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qw62t\" (UID: \"437dc02b-4909-4003-b9f4-8fac0bba4215\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qw62t" Jul 15 12:30:41.891662 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:41.891624 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-qw62t"] Jul 15 12:30:41.990467 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:41.990435 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/437dc02b-4909-4003-b9f4-8fac0bba4215-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qw62t\" (UID: \"437dc02b-4909-4003-b9f4-8fac0bba4215\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qw62t" Jul 15 12:30:41.990467 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:41.990472 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/437dc02b-4909-4003-b9f4-8fac0bba4215-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qw62t\" (UID: \"437dc02b-4909-4003-b9f4-8fac0bba4215\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qw62t" Jul 15 12:30:41.990734 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:41.990498 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/437dc02b-4909-4003-b9f4-8fac0bba4215-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qw62t\" (UID: \"437dc02b-4909-4003-b9f4-8fac0bba4215\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qw62t" Jul 15 12:30:41.990734 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:41.990535 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/437dc02b-4909-4003-b9f4-8fac0bba4215-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qw62t\" (UID: \"437dc02b-4909-4003-b9f4-8fac0bba4215\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qw62t" Jul 15 12:30:41.990734 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:41.990585 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxkgs\" (UniqueName: \"kubernetes.io/projected/437dc02b-4909-4003-b9f4-8fac0bba4215-kube-api-access-nxkgs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qw62t\" (UID: \"437dc02b-4909-4003-b9f4-8fac0bba4215\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qw62t" Jul 15 12:30:41.990734 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:41.990622 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/437dc02b-4909-4003-b9f4-8fac0bba4215-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qw62t\" (UID: \"437dc02b-4909-4003-b9f4-8fac0bba4215\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qw62t" Jul 15 12:30:41.991000 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:41.990978 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/437dc02b-4909-4003-b9f4-8fac0bba4215-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qw62t\" (UID: \"437dc02b-4909-4003-b9f4-8fac0bba4215\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qw62t" Jul 15 12:30:41.991075 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:41.991054 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/437dc02b-4909-4003-b9f4-8fac0bba4215-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qw62t\" (UID: \"437dc02b-4909-4003-b9f4-8fac0bba4215\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qw62t" Jul 15 12:30:41.991130 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:41.991060 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/437dc02b-4909-4003-b9f4-8fac0bba4215-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qw62t\" (UID: \"437dc02b-4909-4003-b9f4-8fac0bba4215\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qw62t" Jul 15 12:30:41.992747 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:41.992726 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/437dc02b-4909-4003-b9f4-8fac0bba4215-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qw62t\" (UID: \"437dc02b-4909-4003-b9f4-8fac0bba4215\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qw62t" Jul 15 12:30:41.993003 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:41.992988 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/437dc02b-4909-4003-b9f4-8fac0bba4215-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qw62t\" (UID: \"437dc02b-4909-4003-b9f4-8fac0bba4215\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qw62t" Jul 15 12:30:41.999065 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:41.999046 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxkgs\" (UniqueName: \"kubernetes.io/projected/437dc02b-4909-4003-b9f4-8fac0bba4215-kube-api-access-nxkgs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qw62t\" (UID: \"437dc02b-4909-4003-b9f4-8fac0bba4215\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qw62t" Jul 15 12:30:42.164963 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:42.164883 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qw62t" Jul 15 12:30:42.302242 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:42.302215 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-qw62t"] Jul 15 12:30:42.303357 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:30:42.303332 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod437dc02b_4909_4003_b9f4_8fac0bba4215.slice/crio-0f3cb6ccdacfd4ecfb9d8c10844e5a45b55483d3a3c2e8304d0609e694f28e49 WatchSource:0}: Error finding container 0f3cb6ccdacfd4ecfb9d8c10844e5a45b55483d3a3c2e8304d0609e694f28e49: Status 404 returned error can't find the container with id 0f3cb6ccdacfd4ecfb9d8c10844e5a45b55483d3a3c2e8304d0609e694f28e49 Jul 15 12:30:42.304829 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:42.304811 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 12:30:42.547815 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:42.547780 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qw62t" event={"ID":"437dc02b-4909-4003-b9f4-8fac0bba4215","Type":"ContainerStarted","Data":"0f3cb6ccdacfd4ecfb9d8c10844e5a45b55483d3a3c2e8304d0609e694f28e49"} Jul 15 12:30:43.555197 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:43.555161 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qw62t" event={"ID":"437dc02b-4909-4003-b9f4-8fac0bba4215","Type":"ContainerStarted","Data":"1c712c7a146b066afe2d96edb5da2e62664b84b2ba225947a17454f33d0d4931"} Jul 15 12:30:43.602455 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:43.602403 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qw62t" podStartSLOduration=1.5381317079999999 podStartE2EDuration="2.602382209s" totalImagesPullingTime="1.064250501s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:30:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:30:43.601914654 +0000 UTC m=+616.742710767" watchObservedRunningTime="2026-07-15 12:30:43.602382209 +0000 UTC m=+616.743178319" Jul 15 12:30:52.165879 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:52.165790 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qw62t" Jul 15 12:30:52.165879 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:52.165855 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qw62t" Jul 15 12:30:52.178791 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:52.178764 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qw62t" Jul 15 12:30:52.617719 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:30:52.617690 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qw62t" Jul 15 12:31:17.411491 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:31:17.411453 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-m66vc"] Jul 15 12:31:17.414830 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:31:17.414811 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m66vc" Jul 15 12:31:17.432062 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:31:17.432040 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 15 12:31:17.467443 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:31:17.467417 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-m66vc"] Jul 15 12:31:17.516644 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:31:17.516619 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0e14eb8-ef78-4034-848e-ed97f4960d8e-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m66vc\" (UID: \"a0e14eb8-ef78-4034-848e-ed97f4960d8e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m66vc" Jul 15 12:31:17.516805 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:31:17.516684 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0e14eb8-ef78-4034-848e-ed97f4960d8e-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m66vc\" (UID: \"a0e14eb8-ef78-4034-848e-ed97f4960d8e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m66vc" Jul 15 12:31:17.516805 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:31:17.516706 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lkgd\" (UniqueName: \"kubernetes.io/projected/a0e14eb8-ef78-4034-848e-ed97f4960d8e-kube-api-access-2lkgd\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m66vc\" (UID: \"a0e14eb8-ef78-4034-848e-ed97f4960d8e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m66vc" Jul 15 12:31:17.516805 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:31:17.516729 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0e14eb8-ef78-4034-848e-ed97f4960d8e-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m66vc\" (UID: \"a0e14eb8-ef78-4034-848e-ed97f4960d8e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m66vc" Jul 15 12:31:17.516805 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:31:17.516755 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0e14eb8-ef78-4034-848e-ed97f4960d8e-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m66vc\" (UID: \"a0e14eb8-ef78-4034-848e-ed97f4960d8e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m66vc" Jul 15 12:31:17.516805 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:31:17.516791 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0e14eb8-ef78-4034-848e-ed97f4960d8e-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m66vc\" (UID: \"a0e14eb8-ef78-4034-848e-ed97f4960d8e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m66vc" Jul 15 12:31:17.617858 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:31:17.617827 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0e14eb8-ef78-4034-848e-ed97f4960d8e-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m66vc\" (UID: \"a0e14eb8-ef78-4034-848e-ed97f4960d8e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m66vc" Jul 15 12:31:17.618035 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:31:17.617872 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0e14eb8-ef78-4034-848e-ed97f4960d8e-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m66vc\" (UID: \"a0e14eb8-ef78-4034-848e-ed97f4960d8e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m66vc" Jul 15 12:31:17.618035 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:31:17.617892 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lkgd\" (UniqueName: \"kubernetes.io/projected/a0e14eb8-ef78-4034-848e-ed97f4960d8e-kube-api-access-2lkgd\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m66vc\" (UID: \"a0e14eb8-ef78-4034-848e-ed97f4960d8e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m66vc" Jul 15 12:31:17.618035 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:31:17.617915 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0e14eb8-ef78-4034-848e-ed97f4960d8e-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m66vc\" (UID: \"a0e14eb8-ef78-4034-848e-ed97f4960d8e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m66vc" Jul 15 12:31:17.618035 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:31:17.617949 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0e14eb8-ef78-4034-848e-ed97f4960d8e-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m66vc\" (UID: \"a0e14eb8-ef78-4034-848e-ed97f4960d8e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m66vc" Jul 15 12:31:17.618035 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:31:17.617966 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0e14eb8-ef78-4034-848e-ed97f4960d8e-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m66vc\" (UID: \"a0e14eb8-ef78-4034-848e-ed97f4960d8e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m66vc" Jul 15 12:31:17.618352 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:31:17.618330 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0e14eb8-ef78-4034-848e-ed97f4960d8e-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m66vc\" (UID: \"a0e14eb8-ef78-4034-848e-ed97f4960d8e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m66vc" Jul 15 12:31:17.618398 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:31:17.618381 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0e14eb8-ef78-4034-848e-ed97f4960d8e-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m66vc\" (UID: \"a0e14eb8-ef78-4034-848e-ed97f4960d8e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m66vc" Jul 15 12:31:17.618398 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:31:17.618392 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0e14eb8-ef78-4034-848e-ed97f4960d8e-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m66vc\" (UID: \"a0e14eb8-ef78-4034-848e-ed97f4960d8e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m66vc" Jul 15 12:31:17.620064 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:31:17.620032 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0e14eb8-ef78-4034-848e-ed97f4960d8e-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m66vc\" (UID: \"a0e14eb8-ef78-4034-848e-ed97f4960d8e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m66vc" Jul 15 12:31:17.620365 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:31:17.620345 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0e14eb8-ef78-4034-848e-ed97f4960d8e-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m66vc\" (UID: \"a0e14eb8-ef78-4034-848e-ed97f4960d8e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m66vc" Jul 15 12:31:17.633148 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:31:17.633123 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lkgd\" (UniqueName: \"kubernetes.io/projected/a0e14eb8-ef78-4034-848e-ed97f4960d8e-kube-api-access-2lkgd\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m66vc\" (UID: \"a0e14eb8-ef78-4034-848e-ed97f4960d8e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m66vc" Jul 15 12:31:17.724852 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:31:17.724771 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m66vc" Jul 15 12:31:17.878052 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:31:17.878023 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-m66vc"] Jul 15 12:31:17.879551 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:31:17.879526 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0e14eb8_ef78_4034_848e_ed97f4960d8e.slice/crio-ceff8dbb6468a4154d94628d6aab723b9d57b469816fbbb2e3d19b01ad28a50c WatchSource:0}: Error finding container ceff8dbb6468a4154d94628d6aab723b9d57b469816fbbb2e3d19b01ad28a50c: Status 404 returned error can't find the container with id ceff8dbb6468a4154d94628d6aab723b9d57b469816fbbb2e3d19b01ad28a50c Jul 15 12:31:18.749945 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:31:18.749908 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m66vc" event={"ID":"a0e14eb8-ef78-4034-848e-ed97f4960d8e","Type":"ContainerStarted","Data":"2ac1eb3b131c5e537381b40e083c62a45e0e768e8e47444d179b9b0f3ffa5a6b"} Jul 15 12:31:18.750323 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:31:18.749965 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m66vc" event={"ID":"a0e14eb8-ef78-4034-848e-ed97f4960d8e","Type":"ContainerStarted","Data":"ceff8dbb6468a4154d94628d6aab723b9d57b469816fbbb2e3d19b01ad28a50c"} Jul 15 12:31:18.774580 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:31:18.774503 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m66vc" podStartSLOduration=1.561845403 podStartE2EDuration="1.774483478s" totalImagesPullingTime="212.638075ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:31:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:31:18.774449791 +0000 UTC m=+651.915245914" watchObservedRunningTime="2026-07-15 12:31:18.774483478 +0000 UTC m=+651.915279590" Jul 15 12:31:27.725624 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:31:27.725586 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m66vc" Jul 15 12:31:27.726021 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:31:27.725676 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m66vc" Jul 15 12:31:27.738103 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:31:27.738082 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m66vc" Jul 15 12:31:27.810474 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:31:27.810447 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m66vc" Jul 15 12:35:27.344093 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:35:27.344064 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bg9fm_808669ad-89bb-473a-86e9-7562fcd99736/ovn-acl-logging/0.log" Jul 15 12:35:27.347069 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:35:27.347047 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bg9fm_808669ad-89bb-473a-86e9-7562fcd99736/ovn-acl-logging/0.log" Jul 15 12:39:14.702866 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:39:14.702838 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjqjw"] Jul 15 12:39:14.703293 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:39:14.703065 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjqjw" podUID="5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce" containerName="manager" containerID="cri-o://d6741e1d9b0565057cab8215d29936d1d5a5cbc1ec60fb7ac547466dd3314984" gracePeriod=10 Jul 15 12:39:17.660919 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:39:17.660894 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjqjw" Jul 15 12:39:17.709479 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:39:17.709454 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-km8qs\" (UniqueName: \"kubernetes.io/projected/5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce-kube-api-access-km8qs\") pod \"5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce\" (UID: \"5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce\") " Jul 15 12:39:17.709623 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:39:17.709536 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce-extensions-socket-volume\") pod \"5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce\" (UID: \"5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce\") " Jul 15 12:39:17.709933 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:39:17.709905 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce" (UID: "5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:39:17.711494 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:39:17.711421 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce-kube-api-access-km8qs" (OuterVolumeSpecName: "kube-api-access-km8qs") pod "5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce" (UID: "5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce"). InnerVolumeSpecName "kube-api-access-km8qs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:39:17.810496 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:39:17.810471 2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce-extensions-socket-volume\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:39:17.810496 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:39:17.810494 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-km8qs\" (UniqueName: \"kubernetes.io/projected/5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce-kube-api-access-km8qs\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:39:18.373122 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:39:18.373086 2560 generic.go:356] "Generic (PLEG): container finished" podID="5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce" containerID="d6741e1d9b0565057cab8215d29936d1d5a5cbc1ec60fb7ac547466dd3314984" exitCode=0 Jul 15 12:39:18.373122 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:39:18.373125 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjqjw" Jul 15 12:39:18.373323 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:39:18.373169 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjqjw" event={"ID":"5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce","Type":"ContainerDied","Data":"d6741e1d9b0565057cab8215d29936d1d5a5cbc1ec60fb7ac547466dd3314984"} Jul 15 12:39:18.373323 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:39:18.373219 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjqjw" event={"ID":"5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce","Type":"ContainerDied","Data":"6566b4e5a6b1cd5d5b096ef3293e6b8c49773022fd3643dd114c7ebb4f1e2e39"} Jul 15 12:39:18.373323 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:39:18.373244 2560 scope.go:117] "RemoveContainer" containerID="d6741e1d9b0565057cab8215d29936d1d5a5cbc1ec60fb7ac547466dd3314984" Jul 15 12:39:18.383877 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:39:18.383859 2560 scope.go:117] "RemoveContainer" containerID="d6741e1d9b0565057cab8215d29936d1d5a5cbc1ec60fb7ac547466dd3314984" Jul 15 12:39:18.384149 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:39:18.384120 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6741e1d9b0565057cab8215d29936d1d5a5cbc1ec60fb7ac547466dd3314984\": container with ID starting with d6741e1d9b0565057cab8215d29936d1d5a5cbc1ec60fb7ac547466dd3314984 not found: ID does not exist" containerID="d6741e1d9b0565057cab8215d29936d1d5a5cbc1ec60fb7ac547466dd3314984" Jul 15 12:39:18.384189 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:39:18.384157 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6741e1d9b0565057cab8215d29936d1d5a5cbc1ec60fb7ac547466dd3314984"} err="failed to get container status \"d6741e1d9b0565057cab8215d29936d1d5a5cbc1ec60fb7ac547466dd3314984\": rpc error: code = NotFound desc = could not find container \"d6741e1d9b0565057cab8215d29936d1d5a5cbc1ec60fb7ac547466dd3314984\": container with ID starting with d6741e1d9b0565057cab8215d29936d1d5a5cbc1ec60fb7ac547466dd3314984 not found: ID does not exist" Jul 15 12:39:18.395877 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:39:18.395853 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjqjw"] Jul 15 12:39:18.399925 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:39:18.399907 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjqjw"] Jul 15 12:39:19.408747 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:39:19.408711 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce" path="/var/lib/kubelet/pods/5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce/volumes" Jul 15 12:40:16.320507 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:40:16.320470 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cd7hl"] Jul 15 12:40:16.321095 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:40:16.321075 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce" containerName="manager" Jul 15 12:40:16.321170 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:40:16.321098 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce" containerName="manager" Jul 15 12:40:16.321241 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:40:16.321228 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b6d05b1-b2f9-4bb7-9615-7bca4b23c7ce" containerName="manager" Jul 15 12:40:16.324455 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:40:16.324433 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cd7hl" Jul 15 12:40:16.327211 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:40:16.327192 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-htfqg\"" Jul 15 12:40:16.340203 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:40:16.340181 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cd7hl"] Jul 15 12:40:16.437488 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:40:16.437458 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jtrw\" (UniqueName: \"kubernetes.io/projected/b2677353-0ada-4a87-b16f-8dade2040585-kube-api-access-9jtrw\") pod \"kuadrant-operator-controller-manager-86878c769f-cd7hl\" (UID: \"b2677353-0ada-4a87-b16f-8dade2040585\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cd7hl" Jul 15 12:40:16.437630 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:40:16.437582 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2677353-0ada-4a87-b16f-8dade2040585-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-cd7hl\" (UID: \"b2677353-0ada-4a87-b16f-8dade2040585\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cd7hl" Jul 15 12:40:16.538655 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:40:16.538610 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2677353-0ada-4a87-b16f-8dade2040585-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-cd7hl\" (UID: \"b2677353-0ada-4a87-b16f-8dade2040585\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cd7hl" Jul 15 12:40:16.538807 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:40:16.538735 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jtrw\" (UniqueName: \"kubernetes.io/projected/b2677353-0ada-4a87-b16f-8dade2040585-kube-api-access-9jtrw\") pod \"kuadrant-operator-controller-manager-86878c769f-cd7hl\" (UID: \"b2677353-0ada-4a87-b16f-8dade2040585\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cd7hl" Jul 15 12:40:16.538978 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:40:16.538955 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2677353-0ada-4a87-b16f-8dade2040585-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-cd7hl\" (UID: \"b2677353-0ada-4a87-b16f-8dade2040585\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cd7hl" Jul 15 12:40:16.553894 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:40:16.553864 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jtrw\" (UniqueName: \"kubernetes.io/projected/b2677353-0ada-4a87-b16f-8dade2040585-kube-api-access-9jtrw\") pod \"kuadrant-operator-controller-manager-86878c769f-cd7hl\" (UID: \"b2677353-0ada-4a87-b16f-8dade2040585\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cd7hl" Jul 15 12:40:16.634468 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:40:16.634409 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cd7hl" Jul 15 12:40:16.772741 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:40:16.772709 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cd7hl"] Jul 15 12:40:16.774769 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:40:16.774744 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2677353_0ada_4a87_b16f_8dade2040585.slice/crio-7a411f761b7b2fc573ed5376e00a60b559036e02c8d7441881fb17f052939f85 WatchSource:0}: Error finding container 7a411f761b7b2fc573ed5376e00a60b559036e02c8d7441881fb17f052939f85: Status 404 returned error can't find the container with id 7a411f761b7b2fc573ed5376e00a60b559036e02c8d7441881fb17f052939f85 Jul 15 12:40:17.697140 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:40:17.697107 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cd7hl" event={"ID":"b2677353-0ada-4a87-b16f-8dade2040585","Type":"ContainerStarted","Data":"805a858ae2dcc1821252cdc48d08c81e8a6c8dce97d2905c3131618f58528afa"} Jul 15 12:40:17.697535 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:40:17.697159 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cd7hl" event={"ID":"b2677353-0ada-4a87-b16f-8dade2040585","Type":"ContainerStarted","Data":"7a411f761b7b2fc573ed5376e00a60b559036e02c8d7441881fb17f052939f85"} Jul 15 12:40:17.697535 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:40:17.697180 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cd7hl" Jul 15 12:40:17.721858 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:40:17.721808 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cd7hl" podStartSLOduration=1.721792694 podStartE2EDuration="1.721792694s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:40:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:40:17.71867395 +0000 UTC m=+1190.859470085" watchObservedRunningTime="2026-07-15 12:40:17.721792694 +0000 UTC m=+1190.862588805" Jul 15 12:40:27.389947 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:40:27.389921 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bg9fm_808669ad-89bb-473a-86e9-7562fcd99736/ovn-acl-logging/0.log" Jul 15 12:40:27.394625 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:40:27.394601 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bg9fm_808669ad-89bb-473a-86e9-7562fcd99736/ovn-acl-logging/0.log" Jul 15 12:40:28.704712 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:40:28.704687 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cd7hl" Jul 15 12:45:27.434573 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:45:27.434541 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bg9fm_808669ad-89bb-473a-86e9-7562fcd99736/ovn-acl-logging/0.log" Jul 15 12:45:27.440544 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:45:27.440526 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bg9fm_808669ad-89bb-473a-86e9-7562fcd99736/ovn-acl-logging/0.log" Jul 15 12:46:18.143678 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.143600 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx"] Jul 15 12:46:18.147909 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.147887 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:18.150309 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.150284 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-2bbbbc27-openshift-default-dockercfg-dpml7\"" Jul 15 12:46:18.165166 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.165142 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx"] Jul 15 12:46:18.199347 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.199323 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7d083a7-aa96-4ead-a5e5-45a3436b569b-credential-socket\") pod \"e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:18.199473 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.199362 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfxxp\" (UniqueName: \"kubernetes.io/projected/b7d083a7-aa96-4ead-a5e5-45a3436b569b-kube-api-access-xfxxp\") pod \"e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:18.199473 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.199400 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7d083a7-aa96-4ead-a5e5-45a3436b569b-workload-certs\") pod \"e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:18.199473 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.199422 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7d083a7-aa96-4ead-a5e5-45a3436b569b-istiod-ca-cert\") pod \"e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:18.199589 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.199505 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7d083a7-aa96-4ead-a5e5-45a3436b569b-istio-envoy\") pod \"e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:18.199589 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.199533 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7d083a7-aa96-4ead-a5e5-45a3436b569b-workload-socket\") pod \"e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:18.199589 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.199557 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7d083a7-aa96-4ead-a5e5-45a3436b569b-istio-token\") pod \"e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:18.199878 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.199593 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7d083a7-aa96-4ead-a5e5-45a3436b569b-istio-data\") pod \"e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:18.199878 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.199612 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7d083a7-aa96-4ead-a5e5-45a3436b569b-istio-podinfo\") pod \"e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:18.300198 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.300164 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7d083a7-aa96-4ead-a5e5-45a3436b569b-workload-certs\") pod \"e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:18.300198 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.300200 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7d083a7-aa96-4ead-a5e5-45a3436b569b-istiod-ca-cert\") pod \"e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:18.300448 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.300242 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7d083a7-aa96-4ead-a5e5-45a3436b569b-istio-envoy\") pod \"e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:18.300448 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.300261 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7d083a7-aa96-4ead-a5e5-45a3436b569b-workload-socket\") pod \"e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:18.300448 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.300279 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7d083a7-aa96-4ead-a5e5-45a3436b569b-istio-token\") pod \"e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:18.300448 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.300312 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7d083a7-aa96-4ead-a5e5-45a3436b569b-istio-data\") pod \"e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:18.300448 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.300345 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7d083a7-aa96-4ead-a5e5-45a3436b569b-istio-podinfo\") pod \"e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:18.300448 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.300385 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7d083a7-aa96-4ead-a5e5-45a3436b569b-credential-socket\") pod \"e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:18.300448 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.300418 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfxxp\" (UniqueName: \"kubernetes.io/projected/b7d083a7-aa96-4ead-a5e5-45a3436b569b-kube-api-access-xfxxp\") pod \"e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:18.300849 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.300615 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7d083a7-aa96-4ead-a5e5-45a3436b569b-workload-certs\") pod \"e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:18.300849 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.300715 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7d083a7-aa96-4ead-a5e5-45a3436b569b-workload-socket\") pod \"e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:18.300849 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.300798 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7d083a7-aa96-4ead-a5e5-45a3436b569b-credential-socket\") pod \"e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:18.301009 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.300889 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7d083a7-aa96-4ead-a5e5-45a3436b569b-istio-data\") pod \"e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:18.301211 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.301191 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7d083a7-aa96-4ead-a5e5-45a3436b569b-istiod-ca-cert\") pod \"e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:18.302616 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.302594 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7d083a7-aa96-4ead-a5e5-45a3436b569b-istio-envoy\") pod \"e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:18.302970 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.302951 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7d083a7-aa96-4ead-a5e5-45a3436b569b-istio-podinfo\") pod \"e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:18.315082 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.315057 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7d083a7-aa96-4ead-a5e5-45a3436b569b-istio-token\") pod \"e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:18.315190 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.315154 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfxxp\" (UniqueName: \"kubernetes.io/projected/b7d083a7-aa96-4ead-a5e5-45a3436b569b-kube-api-access-xfxxp\") pod \"e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:18.460022 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.459908 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:18.609194 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.609170 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx"] Jul 15 12:46:18.610711 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:46:18.610682 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7d083a7_aa96_4ead_a5e5_45a3436b569b.slice/crio-087ce3eb3e060aec76d0ecba7120f26c66b1d228b20653da0493f1b8df95657f WatchSource:0}: Error finding container 087ce3eb3e060aec76d0ecba7120f26c66b1d228b20653da0493f1b8df95657f: Status 404 returned error can't find the container with id 087ce3eb3e060aec76d0ecba7120f26c66b1d228b20653da0493f1b8df95657f Jul 15 12:46:18.619457 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.613301 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:46:18.619457 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.613448 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:46:18.619457 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.613530 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:46:18.683369 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.683342 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" event={"ID":"b7d083a7-aa96-4ead-a5e5-45a3436b569b","Type":"ContainerStarted","Data":"08b2308bd89c91eabd0035700d7e37f0e60743b7bd5119d1ed789f2bf42e8a29"} Jul 15 12:46:18.683506 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.683414 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" event={"ID":"b7d083a7-aa96-4ead-a5e5-45a3436b569b","Type":"ContainerStarted","Data":"087ce3eb3e060aec76d0ecba7120f26c66b1d228b20653da0493f1b8df95657f"} Jul 15 12:46:18.717775 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:18.717621 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" podStartSLOduration=0.717605302 podStartE2EDuration="717.605302ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:46:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:46:18.715205104 +0000 UTC m=+1551.856001214" watchObservedRunningTime="2026-07-15 12:46:18.717605302 +0000 UTC m=+1551.858401413" Jul 15 12:46:19.460927 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:19.460898 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:19.466413 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:19.466133 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:19.688742 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:19.688713 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:19.689528 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:19.689504 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:24.449170 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:24.449138 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2bbbbc27-qdxrt"] Jul 15 12:46:24.453245 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:24.453225 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2bbbbc27-qdxrt" Jul 15 12:46:24.455955 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:24.455928 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 12:46:24.456184 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:24.456162 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-267b6\"" Jul 15 12:46:24.461344 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:24.461317 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2bbbbc27-qdxrt"] Jul 15 12:46:24.550933 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:24.550900 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81e1f446-779d-4d41-9d35-0757951875f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2bbbbc27-qdxrt\" (UID: \"81e1f446-779d-4d41-9d35-0757951875f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2bbbbc27-qdxrt" Jul 15 12:46:24.651936 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:24.651903 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81e1f446-779d-4d41-9d35-0757951875f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2bbbbc27-qdxrt\" (UID: \"81e1f446-779d-4d41-9d35-0757951875f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2bbbbc27-qdxrt" Jul 15 12:46:24.652527 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:24.652508 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81e1f446-779d-4d41-9d35-0757951875f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2bbbbc27-qdxrt\" (UID: \"81e1f446-779d-4d41-9d35-0757951875f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2bbbbc27-qdxrt" Jul 15 12:46:24.770799 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:24.770711 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2bbbbc27-qdxrt" Jul 15 12:46:24.935512 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:24.935489 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2bbbbc27-qdxrt"] Jul 15 12:46:24.937447 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:46:24.937421 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81e1f446_779d_4d41_9d35_0757951875f6.slice/crio-b76ca023d45cba356fe06a7b7679e4ed33e4d2a7b19692b61918da47949cdeeb WatchSource:0}: Error finding container b76ca023d45cba356fe06a7b7679e4ed33e4d2a7b19692b61918da47949cdeeb: Status 404 returned error can't find the container with id b76ca023d45cba356fe06a7b7679e4ed33e4d2a7b19692b61918da47949cdeeb Jul 15 12:46:24.939362 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:24.939344 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 12:46:25.721387 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:25.721358 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2bbbbc27-qdxrt" event={"ID":"81e1f446-779d-4d41-9d35-0757951875f6","Type":"ContainerStarted","Data":"b76ca023d45cba356fe06a7b7679e4ed33e4d2a7b19692b61918da47949cdeeb"} Jul 15 12:46:28.743135 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:28.743097 2560 generic.go:356] "Generic (PLEG): container finished" podID="81e1f446-779d-4d41-9d35-0757951875f6" containerID="0e42a0e631364de7e537f96b9e35ae683c2b72eb974c02517bba97b7924e2d26" exitCode=0 Jul 15 12:46:28.743530 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:28.743179 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2bbbbc27-qdxrt" event={"ID":"81e1f446-779d-4d41-9d35-0757951875f6","Type":"ContainerDied","Data":"0e42a0e631364de7e537f96b9e35ae683c2b72eb974c02517bba97b7924e2d26"} Jul 15 12:46:29.906890 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:29.906869 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2bbbbc27-qdxrt" Jul 15 12:46:30.020105 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:30.020019 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81e1f446-779d-4d41-9d35-0757951875f6-maas-api-service-ca\") pod \"81e1f446-779d-4d41-9d35-0757951875f6\" (UID: \"81e1f446-779d-4d41-9d35-0757951875f6\") " Jul 15 12:46:30.020438 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:30.020418 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81e1f446-779d-4d41-9d35-0757951875f6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "81e1f446-779d-4d41-9d35-0757951875f6" (UID: "81e1f446-779d-4d41-9d35-0757951875f6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:46:30.120971 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:30.120939 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81e1f446-779d-4d41-9d35-0757951875f6-maas-api-service-ca\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:46:30.757823 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:30.757788 2560 generic.go:356] "Generic (PLEG): container finished" podID="b7d083a7-aa96-4ead-a5e5-45a3436b569b" containerID="08b2308bd89c91eabd0035700d7e37f0e60743b7bd5119d1ed789f2bf42e8a29" exitCode=0 Jul 15 12:46:30.758057 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:30.757863 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" event={"ID":"b7d083a7-aa96-4ead-a5e5-45a3436b569b","Type":"ContainerDied","Data":"08b2308bd89c91eabd0035700d7e37f0e60743b7bd5119d1ed789f2bf42e8a29"} Jul 15 12:46:30.758351 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:30.758333 2560 scope.go:117] "RemoveContainer" containerID="08b2308bd89c91eabd0035700d7e37f0e60743b7bd5119d1ed789f2bf42e8a29" Jul 15 12:46:30.759506 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:30.759489 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2bbbbc27-qdxrt" Jul 15 12:46:30.759506 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:30.759486 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:46:30.759927 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:30.759520 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2bbbbc27-qdxrt" event={"ID":"81e1f446-779d-4d41-9d35-0757951875f6","Type":"ContainerDied","Data":"b76ca023d45cba356fe06a7b7679e4ed33e4d2a7b19692b61918da47949cdeeb"} Jul 15 12:46:30.759927 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:30.759560 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b76ca023d45cba356fe06a7b7679e4ed33e4d2a7b19692b61918da47949cdeeb" Jul 15 12:46:30.759927 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:30.759572 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:46:30.759927 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:30.759619 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:46:31.771480 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:31.771437 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" event={"ID":"b7d083a7-aa96-4ead-a5e5-45a3436b569b","Type":"ContainerStarted","Data":"67684323f51885c8089188efa19830fd2a26ebc66635601a16b71de7ecc7bc62"} Jul 15 12:46:32.460678 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:32.460626 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:32.462308 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:32.461977 2560 patch_prober.go:28] interesting pod/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 15 12:46:32.462308 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:32.462044 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" podUID="b7d083a7-aa96-4ead-a5e5-45a3436b569b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 15 12:46:33.468127 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:33.468090 2560 patch_prober.go:28] interesting pod/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 15 12:46:33.468619 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:33.468166 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" podUID="b7d083a7-aa96-4ead-a5e5-45a3436b569b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 15 12:46:34.461389 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:34.461344 2560 patch_prober.go:28] interesting pod/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 15 12:46:34.461599 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:34.461421 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" podUID="b7d083a7-aa96-4ead-a5e5-45a3436b569b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 15 12:46:34.688823 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:34.688785 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:35.461297 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:35.461263 2560 patch_prober.go:28] interesting pod/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 15 12:46:35.461582 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:35.461554 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" podUID="b7d083a7-aa96-4ead-a5e5-45a3436b569b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 15 12:46:36.464743 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:36.464711 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:36.465762 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:36.465742 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:49.754972 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:49.754937 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx"] Jul 15 12:46:49.755584 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:49.755152 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" podUID="b7d083a7-aa96-4ead-a5e5-45a3436b569b" containerName="istio-proxy" containerID="cri-o://67684323f51885c8089188efa19830fd2a26ebc66635601a16b71de7ecc7bc62" gracePeriod=30 Jul 15 12:46:51.467959 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:51.467914 2560 patch_prober.go:28] interesting pod/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 12:46:51.468349 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:51.467983 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" podUID="b7d083a7-aa96-4ead-a5e5-45a3436b569b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 12:46:54.904710 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:54.904675 2560 generic.go:356] "Generic (PLEG): container finished" podID="b7d083a7-aa96-4ead-a5e5-45a3436b569b" containerID="67684323f51885c8089188efa19830fd2a26ebc66635601a16b71de7ecc7bc62" exitCode=0 Jul 15 12:46:54.905084 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:54.904744 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" event={"ID":"b7d083a7-aa96-4ead-a5e5-45a3436b569b","Type":"ContainerDied","Data":"67684323f51885c8089188efa19830fd2a26ebc66635601a16b71de7ecc7bc62"} Jul 15 12:46:54.905084 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:54.904801 2560 scope.go:117] "RemoveContainer" containerID="08b2308bd89c91eabd0035700d7e37f0e60743b7bd5119d1ed789f2bf42e8a29" Jul 15 12:46:55.016257 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:55.016195 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:55.158705 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:55.158679 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7d083a7-aa96-4ead-a5e5-45a3436b569b-istio-token\") pod \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " Jul 15 12:46:55.158873 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:55.158737 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7d083a7-aa96-4ead-a5e5-45a3436b569b-workload-socket\") pod \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " Jul 15 12:46:55.158873 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:55.158784 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7d083a7-aa96-4ead-a5e5-45a3436b569b-istio-podinfo\") pod \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " Jul 15 12:46:55.158873 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:55.158817 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7d083a7-aa96-4ead-a5e5-45a3436b569b-credential-socket\") pod \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " Jul 15 12:46:55.158873 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:55.158848 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7d083a7-aa96-4ead-a5e5-45a3436b569b-istiod-ca-cert\") pod \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " Jul 15 12:46:55.159095 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:55.158910 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xfxxp\" (UniqueName: \"kubernetes.io/projected/b7d083a7-aa96-4ead-a5e5-45a3436b569b-kube-api-access-xfxxp\") pod \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " Jul 15 12:46:55.159095 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:55.158939 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7d083a7-aa96-4ead-a5e5-45a3436b569b-workload-certs\") pod \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " Jul 15 12:46:55.159095 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:55.158981 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7d083a7-aa96-4ead-a5e5-45a3436b569b-istio-envoy\") pod \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " Jul 15 12:46:55.159095 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:55.159009 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7d083a7-aa96-4ead-a5e5-45a3436b569b-istio-data\") pod \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\" (UID: \"b7d083a7-aa96-4ead-a5e5-45a3436b569b\") " Jul 15 12:46:55.159305 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:55.159124 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7d083a7-aa96-4ead-a5e5-45a3436b569b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b7d083a7-aa96-4ead-a5e5-45a3436b569b" (UID: "b7d083a7-aa96-4ead-a5e5-45a3436b569b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:46:55.159367 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:55.159325 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7d083a7-aa96-4ead-a5e5-45a3436b569b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b7d083a7-aa96-4ead-a5e5-45a3436b569b" (UID: "b7d083a7-aa96-4ead-a5e5-45a3436b569b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:46:55.159423 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:55.159391 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7d083a7-aa96-4ead-a5e5-45a3436b569b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b7d083a7-aa96-4ead-a5e5-45a3436b569b" (UID: "b7d083a7-aa96-4ead-a5e5-45a3436b569b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:46:55.159489 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:55.159431 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7d083a7-aa96-4ead-a5e5-45a3436b569b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b7d083a7-aa96-4ead-a5e5-45a3436b569b" (UID: "b7d083a7-aa96-4ead-a5e5-45a3436b569b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:46:55.159489 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:55.159452 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7d083a7-aa96-4ead-a5e5-45a3436b569b-credential-socket\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:46:55.159589 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:55.159490 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7d083a7-aa96-4ead-a5e5-45a3436b569b-workload-certs\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:46:55.160158 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:55.160128 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7d083a7-aa96-4ead-a5e5-45a3436b569b-istio-data" (OuterVolumeSpecName: "istio-data") pod "b7d083a7-aa96-4ead-a5e5-45a3436b569b" (UID: "b7d083a7-aa96-4ead-a5e5-45a3436b569b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:46:55.161265 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:55.161220 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7d083a7-aa96-4ead-a5e5-45a3436b569b-istio-token" (OuterVolumeSpecName: "istio-token") pod "b7d083a7-aa96-4ead-a5e5-45a3436b569b" (UID: "b7d083a7-aa96-4ead-a5e5-45a3436b569b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:46:55.161760 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:55.161736 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b7d083a7-aa96-4ead-a5e5-45a3436b569b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b7d083a7-aa96-4ead-a5e5-45a3436b569b" (UID: "b7d083a7-aa96-4ead-a5e5-45a3436b569b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 12:46:55.161760 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:55.161746 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7d083a7-aa96-4ead-a5e5-45a3436b569b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b7d083a7-aa96-4ead-a5e5-45a3436b569b" (UID: "b7d083a7-aa96-4ead-a5e5-45a3436b569b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:46:55.161890 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:55.161766 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7d083a7-aa96-4ead-a5e5-45a3436b569b-kube-api-access-xfxxp" (OuterVolumeSpecName: "kube-api-access-xfxxp") pod "b7d083a7-aa96-4ead-a5e5-45a3436b569b" (UID: "b7d083a7-aa96-4ead-a5e5-45a3436b569b"). InnerVolumeSpecName "kube-api-access-xfxxp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:46:55.260955 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:55.260919 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7d083a7-aa96-4ead-a5e5-45a3436b569b-istio-envoy\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:46:55.260955 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:55.260944 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7d083a7-aa96-4ead-a5e5-45a3436b569b-istio-data\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:46:55.260955 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:55.260953 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7d083a7-aa96-4ead-a5e5-45a3436b569b-istio-token\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:46:55.260955 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:55.260961 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7d083a7-aa96-4ead-a5e5-45a3436b569b-workload-socket\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:46:55.261271 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:55.260969 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7d083a7-aa96-4ead-a5e5-45a3436b569b-istio-podinfo\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:46:55.261271 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:55.260978 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7d083a7-aa96-4ead-a5e5-45a3436b569b-istiod-ca-cert\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:46:55.261271 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:55.260986 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfxxp\" (UniqueName: \"kubernetes.io/projected/b7d083a7-aa96-4ead-a5e5-45a3436b569b-kube-api-access-xfxxp\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:46:55.912683 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:55.912633 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" Jul 15 12:46:55.913176 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:55.912686 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx" event={"ID":"b7d083a7-aa96-4ead-a5e5-45a3436b569b","Type":"ContainerDied","Data":"087ce3eb3e060aec76d0ecba7120f26c66b1d228b20653da0493f1b8df95657f"} Jul 15 12:46:55.913176 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:55.912746 2560 scope.go:117] "RemoveContainer" containerID="67684323f51885c8089188efa19830fd2a26ebc66635601a16b71de7ecc7bc62" Jul 15 12:46:55.939466 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:55.939442 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx"] Jul 15 12:46:55.949788 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:55.949769 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-2bbbbc27-openshift-default-548f8cc795-ng5cx"] Jul 15 12:46:57.409330 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:46:57.409298 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7d083a7-aa96-4ead-a5e5-45a3436b569b" path="/var/lib/kubelet/pods/b7d083a7-aa96-4ead-a5e5-45a3436b569b/volumes" Jul 15 12:47:21.835709 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:21.835617 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp"] Jul 15 12:47:21.836196 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:21.836178 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7d083a7-aa96-4ead-a5e5-45a3436b569b" containerName="istio-proxy" Jul 15 12:47:21.836242 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:21.836198 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7d083a7-aa96-4ead-a5e5-45a3436b569b" containerName="istio-proxy" Jul 15 12:47:21.836242 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:21.836212 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7d083a7-aa96-4ead-a5e5-45a3436b569b" containerName="istio-proxy" Jul 15 12:47:21.836242 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:21.836218 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7d083a7-aa96-4ead-a5e5-45a3436b569b" containerName="istio-proxy" Jul 15 12:47:21.836242 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:21.836240 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81e1f446-779d-4d41-9d35-0757951875f6" containerName="revoke-keys" Jul 15 12:47:21.836366 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:21.836245 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="81e1f446-779d-4d41-9d35-0757951875f6" containerName="revoke-keys" Jul 15 12:47:21.836366 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:21.836303 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7d083a7-aa96-4ead-a5e5-45a3436b569b" containerName="istio-proxy" Jul 15 12:47:21.836366 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:21.836312 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="81e1f446-779d-4d41-9d35-0757951875f6" containerName="revoke-keys" Jul 15 12:47:21.836366 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:21.836321 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7d083a7-aa96-4ead-a5e5-45a3436b569b" containerName="istio-proxy" Jul 15 12:47:21.839392 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:21.839371 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:21.842561 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:21.842538 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-691083d1-openshift-default-dockercfg-8pd9w\"" Jul 15 12:47:21.871079 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:21.871050 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp"] Jul 15 12:47:21.906936 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:21.906910 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-workload-socket\") pod \"e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:21.907071 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:21.906950 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-istio-data\") pod \"e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:21.907071 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:21.906977 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-istio-podinfo\") pod \"e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:21.907071 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:21.906993 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-credential-socket\") pod \"e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:21.907071 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:21.907026 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-istiod-ca-cert\") pod \"e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:21.907071 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:21.907048 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-workload-certs\") pod \"e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:21.907237 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:21.907102 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-istio-token\") pod \"e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:21.907237 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:21.907145 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-istio-envoy\") pod \"e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:21.907237 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:21.907178 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl7m9\" (UniqueName: \"kubernetes.io/projected/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-kube-api-access-bl7m9\") pod \"e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:22.008536 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:22.008496 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-workload-certs\") pod \"e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:22.008761 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:22.008547 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-istio-token\") pod \"e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:22.008761 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:22.008674 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-istio-envoy\") pod \"e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:22.008761 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:22.008722 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bl7m9\" (UniqueName: \"kubernetes.io/projected/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-kube-api-access-bl7m9\") pod \"e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:22.008947 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:22.008795 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-workload-socket\") pod \"e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:22.008947 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:22.008835 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-istio-data\") pod \"e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:22.008947 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:22.008871 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-istio-podinfo\") pod \"e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:22.008947 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:22.008888 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-credential-socket\") pod \"e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:22.009124 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:22.008976 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-workload-certs\") pod \"e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:22.009124 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:22.009024 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-istiod-ca-cert\") pod \"e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:22.009231 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:22.009173 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-istio-data\") pod \"e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:22.009231 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:22.009211 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-workload-socket\") pod \"e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:22.009370 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:22.009341 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-credential-socket\") pod \"e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:22.009759 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:22.009738 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-istiod-ca-cert\") pod \"e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:22.011008 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:22.010987 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-istio-envoy\") pod \"e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:22.011205 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:22.011187 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-istio-podinfo\") pod \"e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:22.024296 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:22.024274 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl7m9\" (UniqueName: \"kubernetes.io/projected/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-kube-api-access-bl7m9\") pod \"e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:22.026946 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:22.026926 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-istio-token\") pod \"e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:22.150126 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:22.150037 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:22.332854 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:22.332830 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp"] Jul 15 12:47:22.334970 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:47:22.334946 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2678d220_fd2a_42ce_ac45_b96cb1a90a0a.slice/crio-64eb3e132d8f7eb9348ac6ef262b55ac5f5cd3a3d1df7408fc42218d5122eb4d WatchSource:0}: Error finding container 64eb3e132d8f7eb9348ac6ef262b55ac5f5cd3a3d1df7408fc42218d5122eb4d: Status 404 returned error can't find the container with id 64eb3e132d8f7eb9348ac6ef262b55ac5f5cd3a3d1df7408fc42218d5122eb4d Jul 15 12:47:22.337085 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:22.337055 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:47:22.337188 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:22.337136 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:47:22.337188 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:22.337168 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:47:23.075752 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:23.075711 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" event={"ID":"2678d220-fd2a-42ce-ac45-b96cb1a90a0a","Type":"ContainerStarted","Data":"9ef4018a6c3af88b9e32d3905eee34dfdf5bc2e4f4e8fe9efaa2e93945207a3f"} Jul 15 12:47:23.076137 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:23.075773 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" event={"ID":"2678d220-fd2a-42ce-ac45-b96cb1a90a0a","Type":"ContainerStarted","Data":"64eb3e132d8f7eb9348ac6ef262b55ac5f5cd3a3d1df7408fc42218d5122eb4d"} Jul 15 12:47:23.123947 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:23.123878 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" podStartSLOduration=2.123857789 podStartE2EDuration="2.123857789s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:47:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:47:23.122506132 +0000 UTC m=+1616.263302244" watchObservedRunningTime="2026-07-15 12:47:23.123857789 +0000 UTC m=+1616.264653904" Jul 15 12:47:23.150941 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:23.150909 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:23.155814 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:23.155768 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:24.081451 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:24.081419 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:24.082421 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:24.082399 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:35.144188 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:35.144154 2560 generic.go:356] "Generic (PLEG): container finished" podID="2678d220-fd2a-42ce-ac45-b96cb1a90a0a" containerID="9ef4018a6c3af88b9e32d3905eee34dfdf5bc2e4f4e8fe9efaa2e93945207a3f" exitCode=0 Jul 15 12:47:35.144663 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:35.144228 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" event={"ID":"2678d220-fd2a-42ce-ac45-b96cb1a90a0a","Type":"ContainerDied","Data":"9ef4018a6c3af88b9e32d3905eee34dfdf5bc2e4f4e8fe9efaa2e93945207a3f"} Jul 15 12:47:35.144663 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:35.144601 2560 scope.go:117] "RemoveContainer" containerID="9ef4018a6c3af88b9e32d3905eee34dfdf5bc2e4f4e8fe9efaa2e93945207a3f" Jul 15 12:47:35.145722 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:35.145695 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:47:35.145792 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:35.145776 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:47:35.145842 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:35.145826 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:47:36.151521 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:36.151491 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" event={"ID":"2678d220-fd2a-42ce-ac45-b96cb1a90a0a","Type":"ContainerStarted","Data":"ddd89e75d9047c49b0cdf5e33e7d33bb032aaf98e128d72db0d788d226bd7e36"} Jul 15 12:47:37.150701 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:37.150662 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:37.151843 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:37.151809 2560 patch_prober.go:28] interesting pod/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 15 12:47:37.152195 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:37.151870 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" podUID="2678d220-fd2a-42ce-ac45-b96cb1a90a0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 15 12:47:38.150864 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:38.150826 2560 patch_prober.go:28] interesting pod/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 15 12:47:38.151046 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:38.150876 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" podUID="2678d220-fd2a-42ce-ac45-b96cb1a90a0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 15 12:47:39.082128 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:39.082090 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:39.150775 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:39.150742 2560 patch_prober.go:28] interesting pod/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 15 12:47:39.150923 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:39.150794 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" podUID="2678d220-fd2a-42ce-ac45-b96cb1a90a0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 15 12:47:40.155360 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:40.155330 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:47:40.156460 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:47:40.156439 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:48:44.541840 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:44.541810 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp"] Jul 15 12:48:44.542347 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:44.542011 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" podUID="2678d220-fd2a-42ce-ac45-b96cb1a90a0a" containerName="istio-proxy" containerID="cri-o://ddd89e75d9047c49b0cdf5e33e7d33bb032aaf98e128d72db0d788d226bd7e36" gracePeriod=30 Jul 15 12:48:49.796458 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:49.796436 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:48:49.875926 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:49.875895 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-istio-token\") pod \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " Jul 15 12:48:49.876067 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:49.875964 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-istiod-ca-cert\") pod \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " Jul 15 12:48:49.876067 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:49.876026 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-workload-certs\") pod \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " Jul 15 12:48:49.876067 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:49.876064 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-istio-data\") pod \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " Jul 15 12:48:49.876217 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:49.876092 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-workload-socket\") pod \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " Jul 15 12:48:49.876217 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:49.876132 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-credential-socket\") pod \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " Jul 15 12:48:49.876217 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:49.876163 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bl7m9\" (UniqueName: \"kubernetes.io/projected/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-kube-api-access-bl7m9\") pod \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " Jul 15 12:48:49.876217 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:49.876214 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-istio-envoy\") pod \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " Jul 15 12:48:49.876417 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:49.876264 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-istio-podinfo\") pod \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\" (UID: \"2678d220-fd2a-42ce-ac45-b96cb1a90a0a\") " Jul 15 12:48:49.876417 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:49.876352 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2678d220-fd2a-42ce-ac45-b96cb1a90a0a" (UID: "2678d220-fd2a-42ce-ac45-b96cb1a90a0a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:48:49.876417 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:49.876395 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2678d220-fd2a-42ce-ac45-b96cb1a90a0a" (UID: "2678d220-fd2a-42ce-ac45-b96cb1a90a0a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:48:49.876568 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:49.876530 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2678d220-fd2a-42ce-ac45-b96cb1a90a0a" (UID: "2678d220-fd2a-42ce-ac45-b96cb1a90a0a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:48:49.876568 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:49.876541 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2678d220-fd2a-42ce-ac45-b96cb1a90a0a" (UID: "2678d220-fd2a-42ce-ac45-b96cb1a90a0a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:48:49.876788 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:49.876763 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-istiod-ca-cert\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:48:49.876916 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:49.876791 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-workload-certs\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:48:49.876916 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:49.876806 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-workload-socket\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:48:49.876916 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:49.876820 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-credential-socket\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:48:49.877248 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:49.877170 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-istio-data" (OuterVolumeSpecName: "istio-data") pod "2678d220-fd2a-42ce-ac45-b96cb1a90a0a" (UID: "2678d220-fd2a-42ce-ac45-b96cb1a90a0a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:48:49.878436 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:49.878409 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-istio-token" (OuterVolumeSpecName: "istio-token") pod "2678d220-fd2a-42ce-ac45-b96cb1a90a0a" (UID: "2678d220-fd2a-42ce-ac45-b96cb1a90a0a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:48:49.878552 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:49.878525 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-kube-api-access-bl7m9" (OuterVolumeSpecName: "kube-api-access-bl7m9") pod "2678d220-fd2a-42ce-ac45-b96cb1a90a0a" (UID: "2678d220-fd2a-42ce-ac45-b96cb1a90a0a"). InnerVolumeSpecName "kube-api-access-bl7m9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:48:49.878716 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:49.878691 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2678d220-fd2a-42ce-ac45-b96cb1a90a0a" (UID: "2678d220-fd2a-42ce-ac45-b96cb1a90a0a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 12:48:49.878844 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:49.878826 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2678d220-fd2a-42ce-ac45-b96cb1a90a0a" (UID: "2678d220-fd2a-42ce-ac45-b96cb1a90a0a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:48:49.978225 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:49.978198 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-istio-data\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:48:49.978225 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:49.978221 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bl7m9\" (UniqueName: \"kubernetes.io/projected/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-kube-api-access-bl7m9\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:48:49.978225 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:49.978231 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-istio-envoy\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:48:49.978431 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:49.978241 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-istio-podinfo\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:48:49.978431 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:49.978249 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2678d220-fd2a-42ce-ac45-b96cb1a90a0a-istio-token\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:48:50.574595 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:50.574558 2560 generic.go:356] "Generic (PLEG): container finished" podID="2678d220-fd2a-42ce-ac45-b96cb1a90a0a" containerID="ddd89e75d9047c49b0cdf5e33e7d33bb032aaf98e128d72db0d788d226bd7e36" exitCode=0 Jul 15 12:48:50.574784 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:50.574606 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" event={"ID":"2678d220-fd2a-42ce-ac45-b96cb1a90a0a","Type":"ContainerDied","Data":"ddd89e75d9047c49b0cdf5e33e7d33bb032aaf98e128d72db0d788d226bd7e36"} Jul 15 12:48:50.574784 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:50.574618 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" Jul 15 12:48:50.574784 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:50.574686 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp" event={"ID":"2678d220-fd2a-42ce-ac45-b96cb1a90a0a","Type":"ContainerDied","Data":"64eb3e132d8f7eb9348ac6ef262b55ac5f5cd3a3d1df7408fc42218d5122eb4d"} Jul 15 12:48:50.574784 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:50.574710 2560 scope.go:117] "RemoveContainer" containerID="ddd89e75d9047c49b0cdf5e33e7d33bb032aaf98e128d72db0d788d226bd7e36" Jul 15 12:48:50.584546 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:50.584529 2560 scope.go:117] "RemoveContainer" containerID="9ef4018a6c3af88b9e32d3905eee34dfdf5bc2e4f4e8fe9efaa2e93945207a3f" Jul 15 12:48:50.592957 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:50.592942 2560 scope.go:117] "RemoveContainer" containerID="ddd89e75d9047c49b0cdf5e33e7d33bb032aaf98e128d72db0d788d226bd7e36" Jul 15 12:48:50.593211 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:48:50.593195 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddd89e75d9047c49b0cdf5e33e7d33bb032aaf98e128d72db0d788d226bd7e36\": container with ID starting with ddd89e75d9047c49b0cdf5e33e7d33bb032aaf98e128d72db0d788d226bd7e36 not found: ID does not exist" containerID="ddd89e75d9047c49b0cdf5e33e7d33bb032aaf98e128d72db0d788d226bd7e36" Jul 15 12:48:50.593258 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:50.593219 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddd89e75d9047c49b0cdf5e33e7d33bb032aaf98e128d72db0d788d226bd7e36"} err="failed to get container status \"ddd89e75d9047c49b0cdf5e33e7d33bb032aaf98e128d72db0d788d226bd7e36\": rpc error: code = NotFound desc = could not find container \"ddd89e75d9047c49b0cdf5e33e7d33bb032aaf98e128d72db0d788d226bd7e36\": container with ID starting with ddd89e75d9047c49b0cdf5e33e7d33bb032aaf98e128d72db0d788d226bd7e36 not found: ID does not exist" Jul 15 12:48:50.593258 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:50.593237 2560 scope.go:117] "RemoveContainer" containerID="9ef4018a6c3af88b9e32d3905eee34dfdf5bc2e4f4e8fe9efaa2e93945207a3f" Jul 15 12:48:50.593441 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:48:50.593427 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ef4018a6c3af88b9e32d3905eee34dfdf5bc2e4f4e8fe9efaa2e93945207a3f\": container with ID starting with 9ef4018a6c3af88b9e32d3905eee34dfdf5bc2e4f4e8fe9efaa2e93945207a3f not found: ID does not exist" containerID="9ef4018a6c3af88b9e32d3905eee34dfdf5bc2e4f4e8fe9efaa2e93945207a3f" Jul 15 12:48:50.593480 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:50.593445 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ef4018a6c3af88b9e32d3905eee34dfdf5bc2e4f4e8fe9efaa2e93945207a3f"} err="failed to get container status \"9ef4018a6c3af88b9e32d3905eee34dfdf5bc2e4f4e8fe9efaa2e93945207a3f\": rpc error: code = NotFound desc = could not find container \"9ef4018a6c3af88b9e32d3905eee34dfdf5bc2e4f4e8fe9efaa2e93945207a3f\": container with ID starting with 9ef4018a6c3af88b9e32d3905eee34dfdf5bc2e4f4e8fe9efaa2e93945207a3f not found: ID does not exist" Jul 15 12:48:50.597715 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:50.597688 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp"] Jul 15 12:48:50.601853 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:50.601832 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-691083d1-openshift-default-9bb867bf5-x95jp"] Jul 15 12:48:51.409551 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:48:51.409518 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2678d220-fd2a-42ce-ac45-b96cb1a90a0a" path="/var/lib/kubelet/pods/2678d220-fd2a-42ce-ac45-b96cb1a90a0a/volumes" Jul 15 12:50:27.483252 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:50:27.483223 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bg9fm_808669ad-89bb-473a-86e9-7562fcd99736/ovn-acl-logging/0.log" Jul 15 12:50:27.488360 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:50:27.488343 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bg9fm_808669ad-89bb-473a-86e9-7562fcd99736/ovn-acl-logging/0.log" Jul 15 12:51:57.564530 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.564442 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb"] Jul 15 12:51:57.565141 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.564916 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2678d220-fd2a-42ce-ac45-b96cb1a90a0a" containerName="istio-proxy" Jul 15 12:51:57.565141 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.564932 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2678d220-fd2a-42ce-ac45-b96cb1a90a0a" containerName="istio-proxy" Jul 15 12:51:57.565141 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.564942 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2678d220-fd2a-42ce-ac45-b96cb1a90a0a" containerName="istio-proxy" Jul 15 12:51:57.565141 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.564948 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2678d220-fd2a-42ce-ac45-b96cb1a90a0a" containerName="istio-proxy" Jul 15 12:51:57.565141 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.565015 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2678d220-fd2a-42ce-ac45-b96cb1a90a0a" containerName="istio-proxy" Jul 15 12:51:57.565141 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.565023 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2678d220-fd2a-42ce-ac45-b96cb1a90a0a" containerName="istio-proxy" Jul 15 12:51:57.568018 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.567999 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:51:57.571678 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.571660 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-8a28be15-openshift-default-dockercfg-7nc2r\"" Jul 15 12:51:57.596577 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.596544 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb"] Jul 15 12:51:57.764331 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.764299 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9f00ca7-fb88-414a-a641-7d95d33c8a66-istio-envoy\") pod \"e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:51:57.764331 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.764336 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9f00ca7-fb88-414a-a641-7d95d33c8a66-istiod-ca-cert\") pod \"e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:51:57.764570 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.764358 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9f00ca7-fb88-414a-a641-7d95d33c8a66-istio-data\") pod \"e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:51:57.764570 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.764386 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-df599\" (UniqueName: \"kubernetes.io/projected/c9f00ca7-fb88-414a-a641-7d95d33c8a66-kube-api-access-df599\") pod \"e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:51:57.764570 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.764439 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9f00ca7-fb88-414a-a641-7d95d33c8a66-workload-socket\") pod \"e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:51:57.764570 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.764523 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9f00ca7-fb88-414a-a641-7d95d33c8a66-workload-certs\") pod \"e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:51:57.764570 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.764548 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9f00ca7-fb88-414a-a641-7d95d33c8a66-istio-podinfo\") pod \"e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:51:57.764802 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.764577 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9f00ca7-fb88-414a-a641-7d95d33c8a66-istio-token\") pod \"e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:51:57.764802 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.764605 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9f00ca7-fb88-414a-a641-7d95d33c8a66-credential-socket\") pod \"e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:51:57.865327 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.865225 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9f00ca7-fb88-414a-a641-7d95d33c8a66-istio-envoy\") pod \"e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:51:57.865327 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.865288 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9f00ca7-fb88-414a-a641-7d95d33c8a66-istiod-ca-cert\") pod \"e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:51:57.865563 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.865321 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9f00ca7-fb88-414a-a641-7d95d33c8a66-istio-data\") pod \"e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:51:57.865563 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.865382 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-df599\" (UniqueName: \"kubernetes.io/projected/c9f00ca7-fb88-414a-a641-7d95d33c8a66-kube-api-access-df599\") pod \"e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:51:57.865563 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.865407 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9f00ca7-fb88-414a-a641-7d95d33c8a66-workload-socket\") pod \"e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:51:57.865563 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.865433 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9f00ca7-fb88-414a-a641-7d95d33c8a66-workload-certs\") pod \"e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:51:57.865563 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.865468 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9f00ca7-fb88-414a-a641-7d95d33c8a66-istio-podinfo\") pod \"e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:51:57.865563 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.865502 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9f00ca7-fb88-414a-a641-7d95d33c8a66-istio-token\") pod \"e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:51:57.865563 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.865537 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9f00ca7-fb88-414a-a641-7d95d33c8a66-credential-socket\") pod \"e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:51:57.866021 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.865927 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9f00ca7-fb88-414a-a641-7d95d33c8a66-credential-socket\") pod \"e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:51:57.866595 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.866145 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9f00ca7-fb88-414a-a641-7d95d33c8a66-workload-socket\") pod \"e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:51:57.866595 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.866191 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9f00ca7-fb88-414a-a641-7d95d33c8a66-workload-certs\") pod \"e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:51:57.866595 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.866399 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9f00ca7-fb88-414a-a641-7d95d33c8a66-istio-data\") pod \"e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:51:57.867071 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.867045 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9f00ca7-fb88-414a-a641-7d95d33c8a66-istiod-ca-cert\") pod \"e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:51:57.868277 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.868255 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9f00ca7-fb88-414a-a641-7d95d33c8a66-istio-envoy\") pod \"e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:51:57.869018 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.868995 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9f00ca7-fb88-414a-a641-7d95d33c8a66-istio-podinfo\") pod \"e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:51:57.881907 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.881875 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-df599\" (UniqueName: \"kubernetes.io/projected/c9f00ca7-fb88-414a-a641-7d95d33c8a66-kube-api-access-df599\") pod \"e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:51:57.882001 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:57.881906 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9f00ca7-fb88-414a-a641-7d95d33c8a66-istio-token\") pod \"e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:51:58.178912 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:58.178824 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:51:58.327626 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:58.327600 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb"] Jul 15 12:51:58.328942 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:51:58.328916 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9f00ca7_fb88_414a_a641_7d95d33c8a66.slice/crio-1236427d2c8688759f908834c47231f1b9395a71463aca80b3a43ee103ecb854 WatchSource:0}: Error finding container 1236427d2c8688759f908834c47231f1b9395a71463aca80b3a43ee103ecb854: Status 404 returned error can't find the container with id 1236427d2c8688759f908834c47231f1b9395a71463aca80b3a43ee103ecb854 Jul 15 12:51:58.331002 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:58.330972 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:51:58.331078 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:58.331037 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:51:58.331078 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:58.331067 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:51:58.607249 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:58.607207 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" event={"ID":"c9f00ca7-fb88-414a-a641-7d95d33c8a66","Type":"ContainerStarted","Data":"958ca27aaa14099b177dd4cfdfc56dbaeec13b9c1f562c528a9f633f66e4e48c"} Jul 15 12:51:58.607661 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:58.607274 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" event={"ID":"c9f00ca7-fb88-414a-a641-7d95d33c8a66","Type":"ContainerStarted","Data":"1236427d2c8688759f908834c47231f1b9395a71463aca80b3a43ee103ecb854"} Jul 15 12:51:58.633596 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:58.633546 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" podStartSLOduration=1.633529448 podStartE2EDuration="1.633529448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:51:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:51:58.63207347 +0000 UTC m=+1891.772869592" watchObservedRunningTime="2026-07-15 12:51:58.633529448 +0000 UTC m=+1891.774325558" Jul 15 12:51:59.179945 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:59.179901 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:51:59.184765 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:59.184734 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:51:59.613733 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:59.613706 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:51:59.614771 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:51:59.614753 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:52:14.615405 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:14.615365 2560 patch_prober.go:28] interesting pod/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 15 12:52:14.615923 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:14.615436 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" podUID="c9f00ca7-fb88-414a-a641-7d95d33c8a66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 15 12:52:14.711650 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:14.711607 2560 generic.go:356] "Generic (PLEG): container finished" podID="c9f00ca7-fb88-414a-a641-7d95d33c8a66" containerID="958ca27aaa14099b177dd4cfdfc56dbaeec13b9c1f562c528a9f633f66e4e48c" exitCode=0 Jul 15 12:52:14.711818 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:14.711670 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" event={"ID":"c9f00ca7-fb88-414a-a641-7d95d33c8a66","Type":"ContainerDied","Data":"958ca27aaa14099b177dd4cfdfc56dbaeec13b9c1f562c528a9f633f66e4e48c"} Jul 15 12:52:14.712045 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:14.712032 2560 scope.go:117] "RemoveContainer" containerID="958ca27aaa14099b177dd4cfdfc56dbaeec13b9c1f562c528a9f633f66e4e48c" Jul 15 12:52:14.713013 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:14.712987 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:52:14.713068 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:14.713052 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:52:14.713104 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:14.713085 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:52:15.720336 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:15.720308 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" event={"ID":"c9f00ca7-fb88-414a-a641-7d95d33c8a66","Type":"ContainerStarted","Data":"d8ca62d4038a755ef2598e62527cc48dbb4451eff418dfa88406c2d1696d0d76"} Jul 15 12:52:16.180187 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:16.179673 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:52:16.180187 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:16.179841 2560 patch_prober.go:28] interesting pod/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 15 12:52:16.180187 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:16.179901 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" podUID="c9f00ca7-fb88-414a-a641-7d95d33c8a66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 15 12:52:17.179671 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:17.179609 2560 patch_prober.go:28] interesting pod/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 15 12:52:17.180298 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:17.180262 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" podUID="c9f00ca7-fb88-414a-a641-7d95d33c8a66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 15 12:52:18.179918 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:18.179879 2560 patch_prober.go:28] interesting pod/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 15 12:52:18.180544 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:18.180509 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" podUID="c9f00ca7-fb88-414a-a641-7d95d33c8a66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 15 12:52:19.179692 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:19.179655 2560 patch_prober.go:28] interesting pod/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 15 12:52:19.179995 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:19.179966 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" podUID="c9f00ca7-fb88-414a-a641-7d95d33c8a66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 15 12:52:20.183594 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:20.183566 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:52:20.184053 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:20.183844 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:52:20.184587 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:20.184572 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:52:38.535564 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:38.535533 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb"] Jul 15 12:52:38.536151 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:38.535752 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" podUID="c9f00ca7-fb88-414a-a641-7d95d33c8a66" containerName="istio-proxy" containerID="cri-o://d8ca62d4038a755ef2598e62527cc48dbb4451eff418dfa88406c2d1696d0d76" gracePeriod=30 Jul 15 12:52:43.795039 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.795012 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:52:43.806173 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.806107 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9f00ca7-fb88-414a-a641-7d95d33c8a66-workload-socket\") pod \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " Jul 15 12:52:43.806288 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.806169 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-df599\" (UniqueName: \"kubernetes.io/projected/c9f00ca7-fb88-414a-a641-7d95d33c8a66-kube-api-access-df599\") pod \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " Jul 15 12:52:43.806288 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.806253 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9f00ca7-fb88-414a-a641-7d95d33c8a66-istiod-ca-cert\") pod \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " Jul 15 12:52:43.806288 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.806283 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9f00ca7-fb88-414a-a641-7d95d33c8a66-workload-certs\") pod \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " Jul 15 12:52:43.806444 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.806312 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9f00ca7-fb88-414a-a641-7d95d33c8a66-istio-podinfo\") pod \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " Jul 15 12:52:43.806444 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.806347 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9f00ca7-fb88-414a-a641-7d95d33c8a66-credential-socket\") pod \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " Jul 15 12:52:43.806444 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.806346 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9f00ca7-fb88-414a-a641-7d95d33c8a66-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c9f00ca7-fb88-414a-a641-7d95d33c8a66" (UID: "c9f00ca7-fb88-414a-a641-7d95d33c8a66"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:52:43.806444 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.806425 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9f00ca7-fb88-414a-a641-7d95d33c8a66-istio-token\") pod \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " Jul 15 12:52:43.806672 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.806458 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9f00ca7-fb88-414a-a641-7d95d33c8a66-istio-envoy\") pod \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " Jul 15 12:52:43.806672 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.806485 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9f00ca7-fb88-414a-a641-7d95d33c8a66-istio-data\") pod \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\" (UID: \"c9f00ca7-fb88-414a-a641-7d95d33c8a66\") " Jul 15 12:52:43.806672 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.806609 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9f00ca7-fb88-414a-a641-7d95d33c8a66-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c9f00ca7-fb88-414a-a641-7d95d33c8a66" (UID: "c9f00ca7-fb88-414a-a641-7d95d33c8a66"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:52:43.806839 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.806716 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9f00ca7-fb88-414a-a641-7d95d33c8a66-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c9f00ca7-fb88-414a-a641-7d95d33c8a66" (UID: "c9f00ca7-fb88-414a-a641-7d95d33c8a66"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:52:43.806839 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.806728 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9f00ca7-fb88-414a-a641-7d95d33c8a66-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c9f00ca7-fb88-414a-a641-7d95d33c8a66" (UID: "c9f00ca7-fb88-414a-a641-7d95d33c8a66"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:52:43.807043 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.807019 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9f00ca7-fb88-414a-a641-7d95d33c8a66-istiod-ca-cert\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:52:43.807043 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.807045 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9f00ca7-fb88-414a-a641-7d95d33c8a66-workload-certs\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:52:43.807224 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.807058 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9f00ca7-fb88-414a-a641-7d95d33c8a66-credential-socket\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:52:43.807224 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.807072 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9f00ca7-fb88-414a-a641-7d95d33c8a66-workload-socket\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:52:43.807513 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.807487 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9f00ca7-fb88-414a-a641-7d95d33c8a66-istio-data" (OuterVolumeSpecName: "istio-data") pod "c9f00ca7-fb88-414a-a641-7d95d33c8a66" (UID: "c9f00ca7-fb88-414a-a641-7d95d33c8a66"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:52:43.808727 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.808697 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9f00ca7-fb88-414a-a641-7d95d33c8a66-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c9f00ca7-fb88-414a-a641-7d95d33c8a66" (UID: "c9f00ca7-fb88-414a-a641-7d95d33c8a66"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:52:43.808826 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.808792 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9f00ca7-fb88-414a-a641-7d95d33c8a66-istio-token" (OuterVolumeSpecName: "istio-token") pod "c9f00ca7-fb88-414a-a641-7d95d33c8a66" (UID: "c9f00ca7-fb88-414a-a641-7d95d33c8a66"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:52:43.808880 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.808853 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9f00ca7-fb88-414a-a641-7d95d33c8a66-kube-api-access-df599" (OuterVolumeSpecName: "kube-api-access-df599") pod "c9f00ca7-fb88-414a-a641-7d95d33c8a66" (UID: "c9f00ca7-fb88-414a-a641-7d95d33c8a66"). InnerVolumeSpecName "kube-api-access-df599". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:52:43.808920 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.808851 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c9f00ca7-fb88-414a-a641-7d95d33c8a66-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c9f00ca7-fb88-414a-a641-7d95d33c8a66" (UID: "c9f00ca7-fb88-414a-a641-7d95d33c8a66"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 12:52:43.872788 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.872755 2560 generic.go:356] "Generic (PLEG): container finished" podID="c9f00ca7-fb88-414a-a641-7d95d33c8a66" containerID="d8ca62d4038a755ef2598e62527cc48dbb4451eff418dfa88406c2d1696d0d76" exitCode=0 Jul 15 12:52:43.872927 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.872800 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" Jul 15 12:52:43.872927 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.872836 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" event={"ID":"c9f00ca7-fb88-414a-a641-7d95d33c8a66","Type":"ContainerDied","Data":"d8ca62d4038a755ef2598e62527cc48dbb4451eff418dfa88406c2d1696d0d76"} Jul 15 12:52:43.872927 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.872892 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb" event={"ID":"c9f00ca7-fb88-414a-a641-7d95d33c8a66","Type":"ContainerDied","Data":"1236427d2c8688759f908834c47231f1b9395a71463aca80b3a43ee103ecb854"} Jul 15 12:52:43.872927 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.872922 2560 scope.go:117] "RemoveContainer" containerID="d8ca62d4038a755ef2598e62527cc48dbb4451eff418dfa88406c2d1696d0d76" Jul 15 12:52:43.882525 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.882509 2560 scope.go:117] "RemoveContainer" containerID="958ca27aaa14099b177dd4cfdfc56dbaeec13b9c1f562c528a9f633f66e4e48c" Jul 15 12:52:43.890431 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.890415 2560 scope.go:117] "RemoveContainer" containerID="d8ca62d4038a755ef2598e62527cc48dbb4451eff418dfa88406c2d1696d0d76" Jul 15 12:52:43.890690 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:52:43.890666 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8ca62d4038a755ef2598e62527cc48dbb4451eff418dfa88406c2d1696d0d76\": container with ID starting with d8ca62d4038a755ef2598e62527cc48dbb4451eff418dfa88406c2d1696d0d76 not found: ID does not exist" containerID="d8ca62d4038a755ef2598e62527cc48dbb4451eff418dfa88406c2d1696d0d76" Jul 15 12:52:43.890758 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.890699 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8ca62d4038a755ef2598e62527cc48dbb4451eff418dfa88406c2d1696d0d76"} err="failed to get container status \"d8ca62d4038a755ef2598e62527cc48dbb4451eff418dfa88406c2d1696d0d76\": rpc error: code = NotFound desc = could not find container \"d8ca62d4038a755ef2598e62527cc48dbb4451eff418dfa88406c2d1696d0d76\": container with ID starting with d8ca62d4038a755ef2598e62527cc48dbb4451eff418dfa88406c2d1696d0d76 not found: ID does not exist" Jul 15 12:52:43.890758 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.890715 2560 scope.go:117] "RemoveContainer" containerID="958ca27aaa14099b177dd4cfdfc56dbaeec13b9c1f562c528a9f633f66e4e48c" Jul 15 12:52:43.890948 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:52:43.890933 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"958ca27aaa14099b177dd4cfdfc56dbaeec13b9c1f562c528a9f633f66e4e48c\": container with ID starting with 958ca27aaa14099b177dd4cfdfc56dbaeec13b9c1f562c528a9f633f66e4e48c not found: ID does not exist" containerID="958ca27aaa14099b177dd4cfdfc56dbaeec13b9c1f562c528a9f633f66e4e48c" Jul 15 12:52:43.890993 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.890951 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"958ca27aaa14099b177dd4cfdfc56dbaeec13b9c1f562c528a9f633f66e4e48c"} err="failed to get container status \"958ca27aaa14099b177dd4cfdfc56dbaeec13b9c1f562c528a9f633f66e4e48c\": rpc error: code = NotFound desc = could not find container \"958ca27aaa14099b177dd4cfdfc56dbaeec13b9c1f562c528a9f633f66e4e48c\": container with ID starting with 958ca27aaa14099b177dd4cfdfc56dbaeec13b9c1f562c528a9f633f66e4e48c not found: ID does not exist" Jul 15 12:52:43.896757 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.896738 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb"] Jul 15 12:52:43.902508 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.902490 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-8a28be15-openshift-default-558858c8c5-wgsnb"] Jul 15 12:52:43.908236 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.908218 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9f00ca7-fb88-414a-a641-7d95d33c8a66-istio-podinfo\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:52:43.908313 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.908238 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9f00ca7-fb88-414a-a641-7d95d33c8a66-istio-token\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:52:43.908313 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.908248 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9f00ca7-fb88-414a-a641-7d95d33c8a66-istio-envoy\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:52:43.908313 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.908256 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9f00ca7-fb88-414a-a641-7d95d33c8a66-istio-data\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:52:43.908313 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:43.908264 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-df599\" (UniqueName: \"kubernetes.io/projected/c9f00ca7-fb88-414a-a641-7d95d33c8a66-kube-api-access-df599\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:52:45.410003 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:52:45.409973 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9f00ca7-fb88-414a-a641-7d95d33c8a66" path="/var/lib/kubelet/pods/c9f00ca7-fb88-414a-a641-7d95d33c8a66/volumes" Jul 15 12:55:27.522381 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:27.522347 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bg9fm_808669ad-89bb-473a-86e9-7562fcd99736/ovn-acl-logging/0.log" Jul 15 12:55:27.530739 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:27.530719 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bg9fm_808669ad-89bb-473a-86e9-7562fcd99736/ovn-acl-logging/0.log" Jul 15 12:55:48.478019 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.477984 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn"] Jul 15 12:55:48.478651 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.478616 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9f00ca7-fb88-414a-a641-7d95d33c8a66" containerName="istio-proxy" Jul 15 12:55:48.478740 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.478656 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9f00ca7-fb88-414a-a641-7d95d33c8a66" containerName="istio-proxy" Jul 15 12:55:48.478740 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.478679 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9f00ca7-fb88-414a-a641-7d95d33c8a66" containerName="istio-proxy" Jul 15 12:55:48.478740 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.478687 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9f00ca7-fb88-414a-a641-7d95d33c8a66" containerName="istio-proxy" Jul 15 12:55:48.478890 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.478802 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9f00ca7-fb88-414a-a641-7d95d33c8a66" containerName="istio-proxy" Jul 15 12:55:48.478890 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.478814 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9f00ca7-fb88-414a-a641-7d95d33c8a66" containerName="istio-proxy" Jul 15 12:55:48.481935 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.481914 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:55:48.488614 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.488375 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-4060a9-openshift-default-dockercfg-lbvvv\"" Jul 15 12:55:48.502812 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.502788 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn"] Jul 15 12:55:48.512218 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.512195 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca949039-1b3d-4776-8578-59e036e7e628-istio-envoy\") pod \"e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:55:48.512314 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.512224 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca949039-1b3d-4776-8578-59e036e7e628-istiod-ca-cert\") pod \"e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:55:48.512314 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.512251 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca949039-1b3d-4776-8578-59e036e7e628-credential-socket\") pod \"e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:55:48.512314 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.512282 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca949039-1b3d-4776-8578-59e036e7e628-istio-token\") pod \"e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:55:48.512446 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.512356 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca949039-1b3d-4776-8578-59e036e7e628-istio-podinfo\") pod \"e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:55:48.512446 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.512396 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhl66\" (UniqueName: \"kubernetes.io/projected/ca949039-1b3d-4776-8578-59e036e7e628-kube-api-access-vhl66\") pod \"e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:55:48.512446 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.512424 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca949039-1b3d-4776-8578-59e036e7e628-workload-certs\") pod \"e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:55:48.512446 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.512442 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca949039-1b3d-4776-8578-59e036e7e628-istio-data\") pod \"e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:55:48.512671 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.512497 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca949039-1b3d-4776-8578-59e036e7e628-workload-socket\") pod \"e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:55:48.613740 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.613703 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca949039-1b3d-4776-8578-59e036e7e628-istio-podinfo\") pod \"e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:55:48.613908 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.613763 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhl66\" (UniqueName: \"kubernetes.io/projected/ca949039-1b3d-4776-8578-59e036e7e628-kube-api-access-vhl66\") pod \"e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:55:48.613908 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.613794 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca949039-1b3d-4776-8578-59e036e7e628-workload-certs\") pod \"e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:55:48.613908 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.613810 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca949039-1b3d-4776-8578-59e036e7e628-istio-data\") pod \"e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:55:48.613908 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.613854 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca949039-1b3d-4776-8578-59e036e7e628-workload-socket\") pod \"e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:55:48.614094 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.613928 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca949039-1b3d-4776-8578-59e036e7e628-istio-envoy\") pod \"e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:55:48.614094 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.613953 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca949039-1b3d-4776-8578-59e036e7e628-istiod-ca-cert\") pod \"e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:55:48.614094 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.613992 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca949039-1b3d-4776-8578-59e036e7e628-credential-socket\") pod \"e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:55:48.614094 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.614022 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca949039-1b3d-4776-8578-59e036e7e628-istio-token\") pod \"e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:55:48.614376 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.614275 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca949039-1b3d-4776-8578-59e036e7e628-workload-certs\") pod \"e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:55:48.614376 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.614310 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca949039-1b3d-4776-8578-59e036e7e628-istio-data\") pod \"e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:55:48.614590 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.614566 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca949039-1b3d-4776-8578-59e036e7e628-workload-socket\") pod \"e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:55:48.614691 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.614593 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca949039-1b3d-4776-8578-59e036e7e628-istiod-ca-cert\") pod \"e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:55:48.614759 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.614703 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca949039-1b3d-4776-8578-59e036e7e628-credential-socket\") pod \"e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:55:48.616221 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.616191 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca949039-1b3d-4776-8578-59e036e7e628-istio-envoy\") pod \"e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:55:48.616346 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.616198 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca949039-1b3d-4776-8578-59e036e7e628-istio-podinfo\") pod \"e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:55:48.622449 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.622423 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhl66\" (UniqueName: \"kubernetes.io/projected/ca949039-1b3d-4776-8578-59e036e7e628-kube-api-access-vhl66\") pod \"e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:55:48.622568 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.622450 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca949039-1b3d-4776-8578-59e036e7e628-istio-token\") pod \"e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:55:48.791942 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.791909 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:55:48.944753 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.944726 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn"] Jul 15 12:55:48.946553 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:55:48.946521 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca949039_1b3d_4776_8578_59e036e7e628.slice/crio-c23b3b1761cbecc370e10c773483b596f20364f6578ef5682ac48397629ad229 WatchSource:0}: Error finding container c23b3b1761cbecc370e10c773483b596f20364f6578ef5682ac48397629ad229: Status 404 returned error can't find the container with id c23b3b1761cbecc370e10c773483b596f20364f6578ef5682ac48397629ad229 Jul 15 12:55:48.948759 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.948728 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:55:48.948841 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.948791 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:55:48.948841 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:48.948817 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:55:49.906852 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:49.906814 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" event={"ID":"ca949039-1b3d-4776-8578-59e036e7e628","Type":"ContainerStarted","Data":"e871e19f337f808ebacd34f960891992f583b58b7bfe30b25e5467d4635d181c"} Jul 15 12:55:49.907231 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:49.906867 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" event={"ID":"ca949039-1b3d-4776-8578-59e036e7e628","Type":"ContainerStarted","Data":"c23b3b1761cbecc370e10c773483b596f20364f6578ef5682ac48397629ad229"} Jul 15 12:55:49.931648 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:49.931588 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" podStartSLOduration=1.931573827 podStartE2EDuration="1.931573827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:55:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:55:49.929029311 +0000 UTC m=+2123.069825422" watchObservedRunningTime="2026-07-15 12:55:49.931573827 +0000 UTC m=+2123.072369940" Jul 15 12:55:50.792680 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:50.792626 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:55:50.797898 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:50.797875 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:55:50.911891 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:50.911858 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:55:50.912833 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:55:50.912807 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:56:06.953902 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:56:06.953852 2560 patch_prober.go:28] interesting pod/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 15 12:56:06.954348 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:56:06.953939 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" podUID="ca949039-1b3d-4776-8578-59e036e7e628" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 15 12:56:08.018857 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:56:08.018822 2560 generic.go:356] "Generic (PLEG): container finished" podID="ca949039-1b3d-4776-8578-59e036e7e628" containerID="e871e19f337f808ebacd34f960891992f583b58b7bfe30b25e5467d4635d181c" exitCode=0 Jul 15 12:56:08.019256 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:56:08.018894 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" event={"ID":"ca949039-1b3d-4776-8578-59e036e7e628","Type":"ContainerDied","Data":"e871e19f337f808ebacd34f960891992f583b58b7bfe30b25e5467d4635d181c"} Jul 15 12:56:08.019316 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:56:08.019257 2560 scope.go:117] "RemoveContainer" containerID="e871e19f337f808ebacd34f960891992f583b58b7bfe30b25e5467d4635d181c" Jul 15 12:56:08.020327 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:56:08.020301 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:56:08.020408 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:56:08.020380 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:56:08.020444 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:56:08.020433 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:56:09.026365 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:56:09.026336 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" event={"ID":"ca949039-1b3d-4776-8578-59e036e7e628","Type":"ContainerStarted","Data":"14c8e30f666779c610b45180fc742a084469513a7be9b4d1e3f89508a8925a39"} Jul 15 12:56:09.792332 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:56:09.792298 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:56:09.792537 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:56:09.792515 2560 patch_prober.go:28] interesting pod/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 15 12:56:09.792593 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:56:09.792573 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" podUID="ca949039-1b3d-4776-8578-59e036e7e628" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 15 12:56:10.793012 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:56:10.792974 2560 patch_prober.go:28] interesting pod/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 15 12:56:10.793416 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:56:10.793045 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" podUID="ca949039-1b3d-4776-8578-59e036e7e628" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 15 12:56:11.793036 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:56:11.792996 2560 patch_prober.go:28] interesting pod/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 15 12:56:11.793532 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:56:11.793058 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" podUID="ca949039-1b3d-4776-8578-59e036e7e628" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 15 12:56:12.792618 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:56:12.792584 2560 patch_prober.go:28] interesting pod/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 15 12:56:12.792833 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:56:12.792657 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" podUID="ca949039-1b3d-4776-8578-59e036e7e628" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 15 12:56:13.796258 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:56:13.796230 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:56:13.796736 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:56:13.796494 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:56:13.797312 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:56:13.797294 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:57:15.071901 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:15.071867 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4060a9-v8mlt"] Jul 15 12:57:15.076034 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:15.076015 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4060a9-v8mlt" Jul 15 12:57:15.082400 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:15.082380 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 12:57:15.082531 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:15.082435 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-267b6\"" Jul 15 12:57:15.099831 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:15.099808 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4060a9-v8mlt"] Jul 15 12:57:15.159079 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:15.159049 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f11da84-53fc-41bf-92f5-00b65f16f402-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4060a9-v8mlt\" (UID: \"4f11da84-53fc-41bf-92f5-00b65f16f402\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4060a9-v8mlt" Jul 15 12:57:15.260118 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:15.260077 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f11da84-53fc-41bf-92f5-00b65f16f402-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4060a9-v8mlt\" (UID: \"4f11da84-53fc-41bf-92f5-00b65f16f402\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4060a9-v8mlt" Jul 15 12:57:15.260815 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:15.260789 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f11da84-53fc-41bf-92f5-00b65f16f402-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4060a9-v8mlt\" (UID: \"4f11da84-53fc-41bf-92f5-00b65f16f402\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4060a9-v8mlt" Jul 15 12:57:15.385860 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:15.385775 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4060a9-v8mlt" Jul 15 12:57:15.518236 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:15.518203 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4060a9-v8mlt"] Jul 15 12:57:15.519991 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:57:15.519966 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f11da84_53fc_41bf_92f5_00b65f16f402.slice/crio-77bc51b0489739807be7d26236254640efa5fdb4c28e1017b23c78af0e175971 WatchSource:0}: Error finding container 77bc51b0489739807be7d26236254640efa5fdb4c28e1017b23c78af0e175971: Status 404 returned error can't find the container with id 77bc51b0489739807be7d26236254640efa5fdb4c28e1017b23c78af0e175971 Jul 15 12:57:16.407501 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:16.407471 2560 generic.go:356] "Generic (PLEG): container finished" podID="4f11da84-53fc-41bf-92f5-00b65f16f402" containerID="276606bf810fb57ffcc7b8478eedddf1863af82e2c7ce5d2271b86feab85025b" exitCode=0 Jul 15 12:57:16.407878 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:16.407527 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4060a9-v8mlt" event={"ID":"4f11da84-53fc-41bf-92f5-00b65f16f402","Type":"ContainerDied","Data":"276606bf810fb57ffcc7b8478eedddf1863af82e2c7ce5d2271b86feab85025b"} Jul 15 12:57:16.407878 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:16.407567 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4060a9-v8mlt" event={"ID":"4f11da84-53fc-41bf-92f5-00b65f16f402","Type":"ContainerStarted","Data":"77bc51b0489739807be7d26236254640efa5fdb4c28e1017b23c78af0e175971"} Jul 15 12:57:17.544464 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:17.544439 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4060a9-v8mlt" Jul 15 12:57:17.685475 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:17.685378 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f11da84-53fc-41bf-92f5-00b65f16f402-maas-api-service-ca\") pod \"4f11da84-53fc-41bf-92f5-00b65f16f402\" (UID: \"4f11da84-53fc-41bf-92f5-00b65f16f402\") " Jul 15 12:57:17.685805 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:17.685783 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f11da84-53fc-41bf-92f5-00b65f16f402-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4f11da84-53fc-41bf-92f5-00b65f16f402" (UID: "4f11da84-53fc-41bf-92f5-00b65f16f402"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:57:17.786951 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:17.786920 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f11da84-53fc-41bf-92f5-00b65f16f402-maas-api-service-ca\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:57:18.422538 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:18.422515 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4060a9-v8mlt" Jul 15 12:57:18.422745 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:18.422515 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4060a9-v8mlt" event={"ID":"4f11da84-53fc-41bf-92f5-00b65f16f402","Type":"ContainerDied","Data":"77bc51b0489739807be7d26236254640efa5fdb4c28e1017b23c78af0e175971"} Jul 15 12:57:18.422745 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:18.422660 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="77bc51b0489739807be7d26236254640efa5fdb4c28e1017b23c78af0e175971" Jul 15 12:57:18.467297 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:57:18.467266 2560 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f11da84_53fc_41bf_92f5_00b65f16f402.slice\": RecentStats: unable to find data in memory cache]" Jul 15 12:57:40.951331 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:40.951300 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn"] Jul 15 12:57:40.951784 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:40.951511 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" podUID="ca949039-1b3d-4776-8578-59e036e7e628" containerName="istio-proxy" containerID="cri-o://14c8e30f666779c610b45180fc742a084469513a7be9b4d1e3f89508a8925a39" gracePeriod=30 Jul 15 12:57:43.799320 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:43.799282 2560 patch_prober.go:28] interesting pod/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 12:57:43.799711 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:43.799347 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" podUID="ca949039-1b3d-4776-8578-59e036e7e628" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 12:57:46.203254 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.203229 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:57:46.243817 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.243746 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca949039-1b3d-4776-8578-59e036e7e628-workload-socket\") pod \"ca949039-1b3d-4776-8578-59e036e7e628\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " Jul 15 12:57:46.243817 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.243801 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca949039-1b3d-4776-8578-59e036e7e628-credential-socket\") pod \"ca949039-1b3d-4776-8578-59e036e7e628\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " Jul 15 12:57:46.244002 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.243842 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca949039-1b3d-4776-8578-59e036e7e628-istio-podinfo\") pod \"ca949039-1b3d-4776-8578-59e036e7e628\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " Jul 15 12:57:46.244002 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.243884 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca949039-1b3d-4776-8578-59e036e7e628-istio-data\") pod \"ca949039-1b3d-4776-8578-59e036e7e628\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " Jul 15 12:57:46.244002 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.243947 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca949039-1b3d-4776-8578-59e036e7e628-istio-envoy\") pod \"ca949039-1b3d-4776-8578-59e036e7e628\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " Jul 15 12:57:46.244002 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.243981 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca949039-1b3d-4776-8578-59e036e7e628-istio-token\") pod \"ca949039-1b3d-4776-8578-59e036e7e628\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " Jul 15 12:57:46.244207 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.244006 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca949039-1b3d-4776-8578-59e036e7e628-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ca949039-1b3d-4776-8578-59e036e7e628" (UID: "ca949039-1b3d-4776-8578-59e036e7e628"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:57:46.244207 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.244037 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca949039-1b3d-4776-8578-59e036e7e628-workload-certs\") pod \"ca949039-1b3d-4776-8578-59e036e7e628\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " Jul 15 12:57:46.244207 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.244094 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca949039-1b3d-4776-8578-59e036e7e628-istiod-ca-cert\") pod \"ca949039-1b3d-4776-8578-59e036e7e628\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " Jul 15 12:57:46.244207 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.244128 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vhl66\" (UniqueName: \"kubernetes.io/projected/ca949039-1b3d-4776-8578-59e036e7e628-kube-api-access-vhl66\") pod \"ca949039-1b3d-4776-8578-59e036e7e628\" (UID: \"ca949039-1b3d-4776-8578-59e036e7e628\") " Jul 15 12:57:46.244207 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.244130 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca949039-1b3d-4776-8578-59e036e7e628-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ca949039-1b3d-4776-8578-59e036e7e628" (UID: "ca949039-1b3d-4776-8578-59e036e7e628"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:57:46.244446 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.244311 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca949039-1b3d-4776-8578-59e036e7e628-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ca949039-1b3d-4776-8578-59e036e7e628" (UID: "ca949039-1b3d-4776-8578-59e036e7e628"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:57:46.244446 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.244405 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca949039-1b3d-4776-8578-59e036e7e628-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ca949039-1b3d-4776-8578-59e036e7e628" (UID: "ca949039-1b3d-4776-8578-59e036e7e628"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:57:46.244559 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.244474 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca949039-1b3d-4776-8578-59e036e7e628-workload-socket\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:57:46.244559 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.244493 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca949039-1b3d-4776-8578-59e036e7e628-credential-socket\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:57:46.244559 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.244509 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca949039-1b3d-4776-8578-59e036e7e628-workload-certs\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:57:46.244559 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.244522 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca949039-1b3d-4776-8578-59e036e7e628-istiod-ca-cert\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:57:46.245083 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.245062 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca949039-1b3d-4776-8578-59e036e7e628-istio-data" (OuterVolumeSpecName: "istio-data") pod "ca949039-1b3d-4776-8578-59e036e7e628" (UID: "ca949039-1b3d-4776-8578-59e036e7e628"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:57:46.246338 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.246313 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca949039-1b3d-4776-8578-59e036e7e628-kube-api-access-vhl66" (OuterVolumeSpecName: "kube-api-access-vhl66") pod "ca949039-1b3d-4776-8578-59e036e7e628" (UID: "ca949039-1b3d-4776-8578-59e036e7e628"). InnerVolumeSpecName "kube-api-access-vhl66". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:57:46.246531 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.246504 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ca949039-1b3d-4776-8578-59e036e7e628-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ca949039-1b3d-4776-8578-59e036e7e628" (UID: "ca949039-1b3d-4776-8578-59e036e7e628"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 12:57:46.246607 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.246503 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca949039-1b3d-4776-8578-59e036e7e628-istio-token" (OuterVolumeSpecName: "istio-token") pod "ca949039-1b3d-4776-8578-59e036e7e628" (UID: "ca949039-1b3d-4776-8578-59e036e7e628"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:57:46.246607 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.246543 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca949039-1b3d-4776-8578-59e036e7e628-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ca949039-1b3d-4776-8578-59e036e7e628" (UID: "ca949039-1b3d-4776-8578-59e036e7e628"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:57:46.345012 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.344971 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca949039-1b3d-4776-8578-59e036e7e628-istio-envoy\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:57:46.345012 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.345007 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca949039-1b3d-4776-8578-59e036e7e628-istio-token\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:57:46.345012 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.345020 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vhl66\" (UniqueName: \"kubernetes.io/projected/ca949039-1b3d-4776-8578-59e036e7e628-kube-api-access-vhl66\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:57:46.345213 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.345034 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca949039-1b3d-4776-8578-59e036e7e628-istio-podinfo\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:57:46.345213 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.345045 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca949039-1b3d-4776-8578-59e036e7e628-istio-data\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:57:46.578017 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.577985 2560 generic.go:356] "Generic (PLEG): container finished" podID="ca949039-1b3d-4776-8578-59e036e7e628" containerID="14c8e30f666779c610b45180fc742a084469513a7be9b4d1e3f89508a8925a39" exitCode=0 Jul 15 12:57:46.578228 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.578040 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" Jul 15 12:57:46.578228 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.578054 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" event={"ID":"ca949039-1b3d-4776-8578-59e036e7e628","Type":"ContainerDied","Data":"14c8e30f666779c610b45180fc742a084469513a7be9b4d1e3f89508a8925a39"} Jul 15 12:57:46.578228 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.578096 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn" event={"ID":"ca949039-1b3d-4776-8578-59e036e7e628","Type":"ContainerDied","Data":"c23b3b1761cbecc370e10c773483b596f20364f6578ef5682ac48397629ad229"} Jul 15 12:57:46.578228 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.578116 2560 scope.go:117] "RemoveContainer" containerID="14c8e30f666779c610b45180fc742a084469513a7be9b4d1e3f89508a8925a39" Jul 15 12:57:46.588008 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.587989 2560 scope.go:117] "RemoveContainer" containerID="e871e19f337f808ebacd34f960891992f583b58b7bfe30b25e5467d4635d181c" Jul 15 12:57:46.598267 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.598243 2560 scope.go:117] "RemoveContainer" containerID="14c8e30f666779c610b45180fc742a084469513a7be9b4d1e3f89508a8925a39" Jul 15 12:57:46.598786 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:57:46.598764 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14c8e30f666779c610b45180fc742a084469513a7be9b4d1e3f89508a8925a39\": container with ID starting with 14c8e30f666779c610b45180fc742a084469513a7be9b4d1e3f89508a8925a39 not found: ID does not exist" containerID="14c8e30f666779c610b45180fc742a084469513a7be9b4d1e3f89508a8925a39" Jul 15 12:57:46.598907 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.598795 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14c8e30f666779c610b45180fc742a084469513a7be9b4d1e3f89508a8925a39"} err="failed to get container status \"14c8e30f666779c610b45180fc742a084469513a7be9b4d1e3f89508a8925a39\": rpc error: code = NotFound desc = could not find container \"14c8e30f666779c610b45180fc742a084469513a7be9b4d1e3f89508a8925a39\": container with ID starting with 14c8e30f666779c610b45180fc742a084469513a7be9b4d1e3f89508a8925a39 not found: ID does not exist" Jul 15 12:57:46.598907 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.598875 2560 scope.go:117] "RemoveContainer" containerID="e871e19f337f808ebacd34f960891992f583b58b7bfe30b25e5467d4635d181c" Jul 15 12:57:46.599417 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:57:46.599377 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e871e19f337f808ebacd34f960891992f583b58b7bfe30b25e5467d4635d181c\": container with ID starting with e871e19f337f808ebacd34f960891992f583b58b7bfe30b25e5467d4635d181c not found: ID does not exist" containerID="e871e19f337f808ebacd34f960891992f583b58b7bfe30b25e5467d4635d181c" Jul 15 12:57:46.599519 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.599441 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e871e19f337f808ebacd34f960891992f583b58b7bfe30b25e5467d4635d181c"} err="failed to get container status \"e871e19f337f808ebacd34f960891992f583b58b7bfe30b25e5467d4635d181c\": rpc error: code = NotFound desc = could not find container \"e871e19f337f808ebacd34f960891992f583b58b7bfe30b25e5467d4635d181c\": container with ID starting with e871e19f337f808ebacd34f960891992f583b58b7bfe30b25e5467d4635d181c not found: ID does not exist" Jul 15 12:57:46.602371 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.602352 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn"] Jul 15 12:57:46.606981 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:46.606964 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-4060a9-openshift-default-5bdf7fd665-2g6mn"] Jul 15 12:57:47.415019 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:57:47.414983 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca949039-1b3d-4776-8578-59e036e7e628" path="/var/lib/kubelet/pods/ca949039-1b3d-4776-8578-59e036e7e628/volumes" Jul 15 12:58:08.178897 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.178860 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc"] Jul 15 12:58:08.179255 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.179246 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca949039-1b3d-4776-8578-59e036e7e628" containerName="istio-proxy" Jul 15 12:58:08.179305 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.179256 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca949039-1b3d-4776-8578-59e036e7e628" containerName="istio-proxy" Jul 15 12:58:08.179305 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.179284 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f11da84-53fc-41bf-92f5-00b65f16f402" containerName="revoke-keys" Jul 15 12:58:08.179305 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.179290 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f11da84-53fc-41bf-92f5-00b65f16f402" containerName="revoke-keys" Jul 15 12:58:08.179397 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.179344 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f11da84-53fc-41bf-92f5-00b65f16f402" containerName="revoke-keys" Jul 15 12:58:08.179397 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.179353 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca949039-1b3d-4776-8578-59e036e7e628" containerName="istio-proxy" Jul 15 12:58:08.179397 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.179361 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca949039-1b3d-4776-8578-59e036e7e628" containerName="istio-proxy" Jul 15 12:58:08.182351 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.182334 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:08.192252 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.192231 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-d0ed97-openshift-default-dockercfg-rklsh\"" Jul 15 12:58:08.242496 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.242466 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc"] Jul 15 12:58:08.356603 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.356564 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9wk9\" (UniqueName: \"kubernetes.io/projected/91d319cb-c145-457c-a2cb-fcf94f6c39a7-kube-api-access-s9wk9\") pod \"e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:08.356603 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.356604 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91d319cb-c145-457c-a2cb-fcf94f6c39a7-credential-socket\") pod \"e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:08.356850 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.356700 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91d319cb-c145-457c-a2cb-fcf94f6c39a7-istio-envoy\") pod \"e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:08.356850 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.356747 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91d319cb-c145-457c-a2cb-fcf94f6c39a7-workload-socket\") pod \"e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:08.356850 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.356764 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91d319cb-c145-457c-a2cb-fcf94f6c39a7-workload-certs\") pod \"e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:08.356850 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.356783 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91d319cb-c145-457c-a2cb-fcf94f6c39a7-istiod-ca-cert\") pod \"e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:08.356850 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.356805 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91d319cb-c145-457c-a2cb-fcf94f6c39a7-istio-podinfo\") pod \"e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:08.357019 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.356903 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91d319cb-c145-457c-a2cb-fcf94f6c39a7-istio-token\") pod \"e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:08.357019 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.356944 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91d319cb-c145-457c-a2cb-fcf94f6c39a7-istio-data\") pod \"e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:08.458347 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.458267 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91d319cb-c145-457c-a2cb-fcf94f6c39a7-istio-envoy\") pod \"e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:08.458347 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.458325 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91d319cb-c145-457c-a2cb-fcf94f6c39a7-workload-socket\") pod \"e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:08.458588 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.458354 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91d319cb-c145-457c-a2cb-fcf94f6c39a7-workload-certs\") pod \"e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:08.458588 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.458384 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91d319cb-c145-457c-a2cb-fcf94f6c39a7-istiod-ca-cert\") pod \"e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:08.458588 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.458421 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91d319cb-c145-457c-a2cb-fcf94f6c39a7-istio-podinfo\") pod \"e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:08.458588 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.458452 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91d319cb-c145-457c-a2cb-fcf94f6c39a7-istio-token\") pod \"e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:08.458588 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.458478 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91d319cb-c145-457c-a2cb-fcf94f6c39a7-istio-data\") pod \"e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:08.458588 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.458519 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9wk9\" (UniqueName: \"kubernetes.io/projected/91d319cb-c145-457c-a2cb-fcf94f6c39a7-kube-api-access-s9wk9\") pod \"e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:08.458588 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.458552 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91d319cb-c145-457c-a2cb-fcf94f6c39a7-credential-socket\") pod \"e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:08.458968 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.458807 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91d319cb-c145-457c-a2cb-fcf94f6c39a7-workload-socket\") pod \"e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:08.459194 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.459144 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91d319cb-c145-457c-a2cb-fcf94f6c39a7-istiod-ca-cert\") pod \"e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:08.459327 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.459270 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91d319cb-c145-457c-a2cb-fcf94f6c39a7-istio-data\") pod \"e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:08.459327 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.459283 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91d319cb-c145-457c-a2cb-fcf94f6c39a7-workload-certs\") pod \"e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:08.459436 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.459331 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91d319cb-c145-457c-a2cb-fcf94f6c39a7-credential-socket\") pod \"e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:08.461139 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.461118 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91d319cb-c145-457c-a2cb-fcf94f6c39a7-istio-envoy\") pod \"e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:08.461389 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.461367 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91d319cb-c145-457c-a2cb-fcf94f6c39a7-istio-podinfo\") pod \"e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:08.474505 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.474483 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9wk9\" (UniqueName: \"kubernetes.io/projected/91d319cb-c145-457c-a2cb-fcf94f6c39a7-kube-api-access-s9wk9\") pod \"e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:08.474608 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.474564 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91d319cb-c145-457c-a2cb-fcf94f6c39a7-istio-token\") pod \"e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:08.492374 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.492354 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:08.637505 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.637476 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc"] Jul 15 12:58:08.639322 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:58:08.639293 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91d319cb_c145_457c_a2cb_fcf94f6c39a7.slice/crio-65d54205e792cf883103a3ef2703cdf2b6723fcf61c48b9bf405dcb17d062567 WatchSource:0}: Error finding container 65d54205e792cf883103a3ef2703cdf2b6723fcf61c48b9bf405dcb17d062567: Status 404 returned error can't find the container with id 65d54205e792cf883103a3ef2703cdf2b6723fcf61c48b9bf405dcb17d062567 Jul 15 12:58:08.641415 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.641388 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:58:08.641473 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.641453 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:58:08.641509 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.641482 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:58:08.705715 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:08.705667 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" event={"ID":"91d319cb-c145-457c-a2cb-fcf94f6c39a7","Type":"ContainerStarted","Data":"65d54205e792cf883103a3ef2703cdf2b6723fcf61c48b9bf405dcb17d062567"} Jul 15 12:58:09.713104 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:09.713070 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" event={"ID":"91d319cb-c145-457c-a2cb-fcf94f6c39a7","Type":"ContainerStarted","Data":"605cbb6bd9efc1f4c07d34a449963c8c9127ef77460b13fd100bf895a8fb73b3"} Jul 15 12:58:09.751441 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:09.751390 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" podStartSLOduration=1.751374124 podStartE2EDuration="1.751374124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:58:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:58:09.748621095 +0000 UTC m=+2262.889417233" watchObservedRunningTime="2026-07-15 12:58:09.751374124 +0000 UTC m=+2262.892170236" Jul 15 12:58:10.493078 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:10.493046 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:10.497968 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:10.497946 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:10.719977 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:10.719951 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:10.720871 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:10.720853 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:25.721909 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:25.721877 2560 patch_prober.go:28] interesting pod/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 15 12:58:25.722320 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:25.721941 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" podUID="91d319cb-c145-457c-a2cb-fcf94f6c39a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 15 12:58:25.818260 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:25.818225 2560 generic.go:356] "Generic (PLEG): container finished" podID="91d319cb-c145-457c-a2cb-fcf94f6c39a7" containerID="605cbb6bd9efc1f4c07d34a449963c8c9127ef77460b13fd100bf895a8fb73b3" exitCode=0 Jul 15 12:58:25.818440 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:25.818297 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" event={"ID":"91d319cb-c145-457c-a2cb-fcf94f6c39a7","Type":"ContainerDied","Data":"605cbb6bd9efc1f4c07d34a449963c8c9127ef77460b13fd100bf895a8fb73b3"} Jul 15 12:58:25.818838 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:25.818820 2560 scope.go:117] "RemoveContainer" containerID="605cbb6bd9efc1f4c07d34a449963c8c9127ef77460b13fd100bf895a8fb73b3" Jul 15 12:58:25.819901 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:25.819876 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:58:25.819963 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:25.819942 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:58:25.820003 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:25.819975 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:58:26.825439 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:26.825405 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" event={"ID":"91d319cb-c145-457c-a2cb-fcf94f6c39a7","Type":"ContainerStarted","Data":"0044d739b105c801df2c71ad93c168c7bc4b6a5c0a655b83c6bd56e97e3fc645"} Jul 15 12:58:27.492890 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:27.492855 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:27.493085 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:27.493065 2560 patch_prober.go:28] interesting pod/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 15 12:58:27.493138 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:27.493112 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" podUID="91d319cb-c145-457c-a2cb-fcf94f6c39a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 15 12:58:28.492926 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:28.492884 2560 patch_prober.go:28] interesting pod/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 15 12:58:28.493334 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:28.492960 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" podUID="91d319cb-c145-457c-a2cb-fcf94f6c39a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 15 12:58:29.493202 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:29.493163 2560 patch_prober.go:28] interesting pod/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 15 12:58:29.493594 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:29.493217 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" podUID="91d319cb-c145-457c-a2cb-fcf94f6c39a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 15 12:58:30.493568 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:30.493534 2560 patch_prober.go:28] interesting pod/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 15 12:58:30.493994 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:30.493586 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" podUID="91d319cb-c145-457c-a2cb-fcf94f6c39a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 15 12:58:31.497109 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:31.497080 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:31.497516 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:31.497448 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:31.498077 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:31.498061 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:58:59.071352 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:59.071316 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc"] Jul 15 12:58:59.071907 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:58:59.071566 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" podUID="91d319cb-c145-457c-a2cb-fcf94f6c39a7" containerName="istio-proxy" containerID="cri-o://0044d739b105c801df2c71ad93c168c7bc4b6a5c0a655b83c6bd56e97e3fc645" gracePeriod=30 Jul 15 12:59:01.499305 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:01.499273 2560 patch_prober.go:28] interesting pod/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 12:59:01.499740 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:01.499325 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" podUID="91d319cb-c145-457c-a2cb-fcf94f6c39a7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 12:59:04.327916 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:04.327892 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:59:04.389137 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:04.389055 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91d319cb-c145-457c-a2cb-fcf94f6c39a7-credential-socket\") pod \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " Jul 15 12:59:04.389137 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:04.389099 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s9wk9\" (UniqueName: \"kubernetes.io/projected/91d319cb-c145-457c-a2cb-fcf94f6c39a7-kube-api-access-s9wk9\") pod \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " Jul 15 12:59:04.389368 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:04.389148 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91d319cb-c145-457c-a2cb-fcf94f6c39a7-istio-envoy\") pod \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " Jul 15 12:59:04.389368 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:04.389171 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91d319cb-c145-457c-a2cb-fcf94f6c39a7-istio-token\") pod \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " Jul 15 12:59:04.389368 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:04.389235 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91d319cb-c145-457c-a2cb-fcf94f6c39a7-istio-data\") pod \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " Jul 15 12:59:04.389368 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:04.389281 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91d319cb-c145-457c-a2cb-fcf94f6c39a7-workload-socket\") pod \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " Jul 15 12:59:04.389368 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:04.389283 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91d319cb-c145-457c-a2cb-fcf94f6c39a7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "91d319cb-c145-457c-a2cb-fcf94f6c39a7" (UID: "91d319cb-c145-457c-a2cb-fcf94f6c39a7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:59:04.389631 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:04.389372 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91d319cb-c145-457c-a2cb-fcf94f6c39a7-workload-certs\") pod \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " Jul 15 12:59:04.389631 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:04.389434 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91d319cb-c145-457c-a2cb-fcf94f6c39a7-istio-podinfo\") pod \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " Jul 15 12:59:04.389631 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:04.389489 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91d319cb-c145-457c-a2cb-fcf94f6c39a7-istiod-ca-cert\") pod \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\" (UID: \"91d319cb-c145-457c-a2cb-fcf94f6c39a7\") " Jul 15 12:59:04.389631 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:04.389516 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91d319cb-c145-457c-a2cb-fcf94f6c39a7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "91d319cb-c145-457c-a2cb-fcf94f6c39a7" (UID: "91d319cb-c145-457c-a2cb-fcf94f6c39a7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:59:04.389856 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:04.389721 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91d319cb-c145-457c-a2cb-fcf94f6c39a7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "91d319cb-c145-457c-a2cb-fcf94f6c39a7" (UID: "91d319cb-c145-457c-a2cb-fcf94f6c39a7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:59:04.389909 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:04.389863 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91d319cb-c145-457c-a2cb-fcf94f6c39a7-credential-socket\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:59:04.389958 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:04.389907 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91d319cb-c145-457c-a2cb-fcf94f6c39a7-workload-socket\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:59:04.389958 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:04.389921 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91d319cb-c145-457c-a2cb-fcf94f6c39a7-workload-certs\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:59:04.390052 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:04.390030 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91d319cb-c145-457c-a2cb-fcf94f6c39a7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "91d319cb-c145-457c-a2cb-fcf94f6c39a7" (UID: "91d319cb-c145-457c-a2cb-fcf94f6c39a7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:59:04.390250 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:04.390219 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91d319cb-c145-457c-a2cb-fcf94f6c39a7-istio-data" (OuterVolumeSpecName: "istio-data") pod "91d319cb-c145-457c-a2cb-fcf94f6c39a7" (UID: "91d319cb-c145-457c-a2cb-fcf94f6c39a7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:59:04.391559 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:04.391526 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91d319cb-c145-457c-a2cb-fcf94f6c39a7-kube-api-access-s9wk9" (OuterVolumeSpecName: "kube-api-access-s9wk9") pod "91d319cb-c145-457c-a2cb-fcf94f6c39a7" (UID: "91d319cb-c145-457c-a2cb-fcf94f6c39a7"). InnerVolumeSpecName "kube-api-access-s9wk9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:59:04.391559 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:04.391526 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91d319cb-c145-457c-a2cb-fcf94f6c39a7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "91d319cb-c145-457c-a2cb-fcf94f6c39a7" (UID: "91d319cb-c145-457c-a2cb-fcf94f6c39a7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:59:04.391761 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:04.391709 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91d319cb-c145-457c-a2cb-fcf94f6c39a7-istio-token" (OuterVolumeSpecName: "istio-token") pod "91d319cb-c145-457c-a2cb-fcf94f6c39a7" (UID: "91d319cb-c145-457c-a2cb-fcf94f6c39a7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:59:04.391845 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:04.391829 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/91d319cb-c145-457c-a2cb-fcf94f6c39a7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "91d319cb-c145-457c-a2cb-fcf94f6c39a7" (UID: "91d319cb-c145-457c-a2cb-fcf94f6c39a7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 12:59:04.490853 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:04.490817 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91d319cb-c145-457c-a2cb-fcf94f6c39a7-istio-podinfo\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:59:04.490853 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:04.490843 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91d319cb-c145-457c-a2cb-fcf94f6c39a7-istiod-ca-cert\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:59:04.490853 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:04.490853 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9wk9\" (UniqueName: \"kubernetes.io/projected/91d319cb-c145-457c-a2cb-fcf94f6c39a7-kube-api-access-s9wk9\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:59:04.490853 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:04.490863 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91d319cb-c145-457c-a2cb-fcf94f6c39a7-istio-envoy\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:59:04.491125 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:04.490871 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91d319cb-c145-457c-a2cb-fcf94f6c39a7-istio-token\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:59:04.491125 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:04.490879 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91d319cb-c145-457c-a2cb-fcf94f6c39a7-istio-data\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 12:59:05.045031 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:05.044997 2560 generic.go:356] "Generic (PLEG): container finished" podID="91d319cb-c145-457c-a2cb-fcf94f6c39a7" containerID="0044d739b105c801df2c71ad93c168c7bc4b6a5c0a655b83c6bd56e97e3fc645" exitCode=0 Jul 15 12:59:05.045190 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:05.045101 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" Jul 15 12:59:05.045190 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:05.045132 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" event={"ID":"91d319cb-c145-457c-a2cb-fcf94f6c39a7","Type":"ContainerDied","Data":"0044d739b105c801df2c71ad93c168c7bc4b6a5c0a655b83c6bd56e97e3fc645"} Jul 15 12:59:05.045190 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:05.045186 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc" event={"ID":"91d319cb-c145-457c-a2cb-fcf94f6c39a7","Type":"ContainerDied","Data":"65d54205e792cf883103a3ef2703cdf2b6723fcf61c48b9bf405dcb17d062567"} Jul 15 12:59:05.045299 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:05.045209 2560 scope.go:117] "RemoveContainer" containerID="0044d739b105c801df2c71ad93c168c7bc4b6a5c0a655b83c6bd56e97e3fc645" Jul 15 12:59:05.056901 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:05.056885 2560 scope.go:117] "RemoveContainer" containerID="605cbb6bd9efc1f4c07d34a449963c8c9127ef77460b13fd100bf895a8fb73b3" Jul 15 12:59:05.064946 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:05.064930 2560 scope.go:117] "RemoveContainer" containerID="0044d739b105c801df2c71ad93c168c7bc4b6a5c0a655b83c6bd56e97e3fc645" Jul 15 12:59:05.065182 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:59:05.065158 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0044d739b105c801df2c71ad93c168c7bc4b6a5c0a655b83c6bd56e97e3fc645\": container with ID starting with 0044d739b105c801df2c71ad93c168c7bc4b6a5c0a655b83c6bd56e97e3fc645 not found: ID does not exist" containerID="0044d739b105c801df2c71ad93c168c7bc4b6a5c0a655b83c6bd56e97e3fc645" Jul 15 12:59:05.065237 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:05.065190 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0044d739b105c801df2c71ad93c168c7bc4b6a5c0a655b83c6bd56e97e3fc645"} err="failed to get container status \"0044d739b105c801df2c71ad93c168c7bc4b6a5c0a655b83c6bd56e97e3fc645\": rpc error: code = NotFound desc = could not find container \"0044d739b105c801df2c71ad93c168c7bc4b6a5c0a655b83c6bd56e97e3fc645\": container with ID starting with 0044d739b105c801df2c71ad93c168c7bc4b6a5c0a655b83c6bd56e97e3fc645 not found: ID does not exist" Jul 15 12:59:05.065237 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:05.065207 2560 scope.go:117] "RemoveContainer" containerID="605cbb6bd9efc1f4c07d34a449963c8c9127ef77460b13fd100bf895a8fb73b3" Jul 15 12:59:05.065441 ip-10-0-139-188 kubenswrapper[2560]: E0715 12:59:05.065420 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"605cbb6bd9efc1f4c07d34a449963c8c9127ef77460b13fd100bf895a8fb73b3\": container with ID starting with 605cbb6bd9efc1f4c07d34a449963c8c9127ef77460b13fd100bf895a8fb73b3 not found: ID does not exist" containerID="605cbb6bd9efc1f4c07d34a449963c8c9127ef77460b13fd100bf895a8fb73b3" Jul 15 12:59:05.065487 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:05.065448 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"605cbb6bd9efc1f4c07d34a449963c8c9127ef77460b13fd100bf895a8fb73b3"} err="failed to get container status \"605cbb6bd9efc1f4c07d34a449963c8c9127ef77460b13fd100bf895a8fb73b3\": rpc error: code = NotFound desc = could not find container \"605cbb6bd9efc1f4c07d34a449963c8c9127ef77460b13fd100bf895a8fb73b3\": container with ID starting with 605cbb6bd9efc1f4c07d34a449963c8c9127ef77460b13fd100bf895a8fb73b3 not found: ID does not exist" Jul 15 12:59:05.076744 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:05.076722 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc"] Jul 15 12:59:05.086512 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:05.086492 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-d0ed97-openshift-default-7777c68977-s58dc"] Jul 15 12:59:05.409394 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:05.409316 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91d319cb-c145-457c-a2cb-fcf94f6c39a7" path="/var/lib/kubelet/pods/91d319cb-c145-457c-a2cb-fcf94f6c39a7/volumes" Jul 15 12:59:26.385912 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.385797 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn"] Jul 15 12:59:26.386392 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.386338 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91d319cb-c145-457c-a2cb-fcf94f6c39a7" containerName="istio-proxy" Jul 15 12:59:26.386392 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.386358 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="91d319cb-c145-457c-a2cb-fcf94f6c39a7" containerName="istio-proxy" Jul 15 12:59:26.386392 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.386378 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca949039-1b3d-4776-8578-59e036e7e628" containerName="istio-proxy" Jul 15 12:59:26.386392 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.386386 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca949039-1b3d-4776-8578-59e036e7e628" containerName="istio-proxy" Jul 15 12:59:26.386392 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.386392 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91d319cb-c145-457c-a2cb-fcf94f6c39a7" containerName="istio-proxy" Jul 15 12:59:26.386577 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.386397 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="91d319cb-c145-457c-a2cb-fcf94f6c39a7" containerName="istio-proxy" Jul 15 12:59:26.386577 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.386472 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="91d319cb-c145-457c-a2cb-fcf94f6c39a7" containerName="istio-proxy" Jul 15 12:59:26.386577 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.386480 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="91d319cb-c145-457c-a2cb-fcf94f6c39a7" containerName="istio-proxy" Jul 15 12:59:26.391218 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.391201 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:26.395164 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.395143 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-3af497-openshift-default-dockercfg-mgpld\"" Jul 15 12:59:26.422472 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.422441 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn"] Jul 15 12:59:26.502526 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.502488 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-istio-podinfo\") pod \"e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:26.502740 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.502538 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-credential-socket\") pod \"e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:26.502740 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.502605 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-istio-data\") pod \"e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:26.502740 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.502697 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbjbx\" (UniqueName: \"kubernetes.io/projected/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-kube-api-access-vbjbx\") pod \"e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:26.502887 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.502746 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-workload-certs\") pod \"e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:26.502887 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.502795 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-istio-token\") pod \"e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:26.502887 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.502822 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-workload-socket\") pod \"e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:26.503046 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.502914 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-istio-envoy\") pod \"e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:26.503046 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.502946 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-istiod-ca-cert\") pod \"e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:26.603878 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.603842 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-istio-podinfo\") pod \"e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:26.604066 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.603892 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-credential-socket\") pod \"e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:26.604066 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.603918 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-istio-data\") pod \"e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:26.604184 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.604104 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbjbx\" (UniqueName: \"kubernetes.io/projected/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-kube-api-access-vbjbx\") pod \"e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:26.604184 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.604173 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-workload-certs\") pod \"e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:26.604299 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.604245 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-istio-token\") pod \"e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:26.604299 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.604277 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-workload-socket\") pod \"e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:26.604403 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.604332 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-istio-envoy\") pod \"e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:26.604403 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.604362 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-istiod-ca-cert\") pod \"e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:26.604507 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.604400 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-istio-data\") pod \"e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:26.604507 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.604334 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-credential-socket\") pod \"e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:26.604618 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.604493 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-workload-certs\") pod \"e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:26.604725 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.604701 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-workload-socket\") pod \"e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:26.605018 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.604997 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-istiod-ca-cert\") pod \"e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:26.606442 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.606420 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-istio-envoy\") pod \"e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:26.606504 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.606444 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-istio-podinfo\") pod \"e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:26.627248 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.627221 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbjbx\" (UniqueName: \"kubernetes.io/projected/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-kube-api-access-vbjbx\") pod \"e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:26.634102 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.634082 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-istio-token\") pod \"e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:26.701464 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.701363 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:26.861617 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.861589 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn"] Jul 15 12:59:26.863413 ip-10-0-139-188 kubenswrapper[2560]: W0715 12:59:26.863386 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef5e6d30_25e0_415c_8a55_6f97be4e4caf.slice/crio-24bfb8b8eb7aa56a1eeda5596d8d95b5dd34376021105b61add31f0ce47df2d4 WatchSource:0}: Error finding container 24bfb8b8eb7aa56a1eeda5596d8d95b5dd34376021105b61add31f0ce47df2d4: Status 404 returned error can't find the container with id 24bfb8b8eb7aa56a1eeda5596d8d95b5dd34376021105b61add31f0ce47df2d4 Jul 15 12:59:26.865604 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.865573 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:59:26.865716 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.865663 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:59:26.865716 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:26.865702 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:59:27.173627 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:27.173570 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" event={"ID":"ef5e6d30-25e0-415c-8a55-6f97be4e4caf","Type":"ContainerStarted","Data":"0a573c3959e2758e4a13506a60701b36751c0dce69af931ec0def4515a65fcee"} Jul 15 12:59:27.173849 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:27.173667 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" event={"ID":"ef5e6d30-25e0-415c-8a55-6f97be4e4caf","Type":"ContainerStarted","Data":"24bfb8b8eb7aa56a1eeda5596d8d95b5dd34376021105b61add31f0ce47df2d4"} Jul 15 12:59:27.204767 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:27.204717 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" podStartSLOduration=1.20469702 podStartE2EDuration="1.20469702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:59:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:59:27.204044933 +0000 UTC m=+2340.344841076" watchObservedRunningTime="2026-07-15 12:59:27.20469702 +0000 UTC m=+2340.345493131" Jul 15 12:59:27.701754 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:27.701722 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:27.706467 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:27.706436 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:28.180165 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:28.180136 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:28.181224 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:28.181206 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:42.269741 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:42.269705 2560 generic.go:356] "Generic (PLEG): container finished" podID="ef5e6d30-25e0-415c-8a55-6f97be4e4caf" containerID="0a573c3959e2758e4a13506a60701b36751c0dce69af931ec0def4515a65fcee" exitCode=0 Jul 15 12:59:42.270183 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:42.269774 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" event={"ID":"ef5e6d30-25e0-415c-8a55-6f97be4e4caf","Type":"ContainerDied","Data":"0a573c3959e2758e4a13506a60701b36751c0dce69af931ec0def4515a65fcee"} Jul 15 12:59:42.270183 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:42.270151 2560 scope.go:117] "RemoveContainer" containerID="0a573c3959e2758e4a13506a60701b36751c0dce69af931ec0def4515a65fcee" Jul 15 12:59:42.271144 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:42.271116 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:59:42.271236 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:42.271185 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:59:42.271236 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:42.271218 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 12:59:43.180469 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:43.180429 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:43.283546 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:43.283511 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" event={"ID":"ef5e6d30-25e0-415c-8a55-6f97be4e4caf","Type":"ContainerStarted","Data":"a0e8be59ea715f21755259e8ed9a49de8692177b984180696bf959611a17b4f3"} Jul 15 12:59:43.702576 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:43.702542 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:43.703884 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:43.703857 2560 patch_prober.go:28] interesting pod/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 15 12:59:43.704018 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:43.703913 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" podUID="ef5e6d30-25e0-415c-8a55-6f97be4e4caf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 15 12:59:44.702538 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:44.702496 2560 patch_prober.go:28] interesting pod/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 15 12:59:44.702960 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:44.702556 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" podUID="ef5e6d30-25e0-415c-8a55-6f97be4e4caf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 15 12:59:45.701987 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:45.701957 2560 patch_prober.go:28] interesting pod/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 15 12:59:45.702206 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:45.702014 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" podUID="ef5e6d30-25e0-415c-8a55-6f97be4e4caf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 15 12:59:46.702761 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:46.702720 2560 patch_prober.go:28] interesting pod/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 15 12:59:46.703150 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:46.702781 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" podUID="ef5e6d30-25e0-415c-8a55-6f97be4e4caf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 15 12:59:47.705761 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:47.705731 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:47.706145 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:47.706016 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 12:59:47.706799 ip-10-0-139-188 kubenswrapper[2560]: I0715 12:59:47.706782 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 13:00:27.559500 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:00:27.559469 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bg9fm_808669ad-89bb-473a-86e9-7562fcd99736/ovn-acl-logging/0.log" Jul 15 13:00:27.568356 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:00:27.568336 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bg9fm_808669ad-89bb-473a-86e9-7562fcd99736/ovn-acl-logging/0.log" Jul 15 13:05:27.598592 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:05:27.598516 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bg9fm_808669ad-89bb-473a-86e9-7562fcd99736/ovn-acl-logging/0.log" Jul 15 13:05:27.610857 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:05:27.610835 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bg9fm_808669ad-89bb-473a-86e9-7562fcd99736/ovn-acl-logging/0.log" Jul 15 13:06:42.141195 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:42.141162 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7606-139635308448816"] Jul 15 13:06:42.144990 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:42.144975 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7606-139635308448816" Jul 15 13:06:42.147746 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:42.147722 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-664xg\"" Jul 15 13:06:42.150565 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:42.150538 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7606-139635308448816"] Jul 15 13:06:42.213047 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:42.213013 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cptd5\" (UniqueName: \"kubernetes.io/projected/f83278de-4f4a-4191-a58f-f5568b8c20e3-kube-api-access-cptd5\") pod \"test-curl-7606-139635308448816\" (UID: \"f83278de-4f4a-4191-a58f-f5568b8c20e3\") " pod="opendatahub/test-curl-7606-139635308448816" Jul 15 13:06:42.313839 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:42.313805 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cptd5\" (UniqueName: \"kubernetes.io/projected/f83278de-4f4a-4191-a58f-f5568b8c20e3-kube-api-access-cptd5\") pod \"test-curl-7606-139635308448816\" (UID: \"f83278de-4f4a-4191-a58f-f5568b8c20e3\") " pod="opendatahub/test-curl-7606-139635308448816" Jul 15 13:06:42.322892 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:42.322869 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cptd5\" (UniqueName: \"kubernetes.io/projected/f83278de-4f4a-4191-a58f-f5568b8c20e3-kube-api-access-cptd5\") pod \"test-curl-7606-139635308448816\" (UID: \"f83278de-4f4a-4191-a58f-f5568b8c20e3\") " pod="opendatahub/test-curl-7606-139635308448816" Jul 15 13:06:42.456042 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:42.455954 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7606-139635308448816" Jul 15 13:06:42.592125 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:42.592098 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7606-139635308448816"] Jul 15 13:06:42.594232 ip-10-0-139-188 kubenswrapper[2560]: W0715 13:06:42.594208 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf83278de_4f4a_4191_a58f_f5568b8c20e3.slice/crio-613057603c55d2f6814fe7910498f9d083c7f176e0978a5cf551bf01fda6b152 WatchSource:0}: Error finding container 613057603c55d2f6814fe7910498f9d083c7f176e0978a5cf551bf01fda6b152: Status 404 returned error can't find the container with id 613057603c55d2f6814fe7910498f9d083c7f176e0978a5cf551bf01fda6b152 Jul 15 13:06:42.595499 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:42.595481 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 13:06:42.644035 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:42.644011 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7606-139635308448816" event={"ID":"f83278de-4f4a-4191-a58f-f5568b8c20e3","Type":"ContainerStarted","Data":"613057603c55d2f6814fe7910498f9d083c7f176e0978a5cf551bf01fda6b152"} Jul 15 13:06:43.652008 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:43.651975 2560 generic.go:356] "Generic (PLEG): container finished" podID="f83278de-4f4a-4191-a58f-f5568b8c20e3" containerID="359edbbf490ac056bf2a2c7c438a6e832c19d02d9d46dfb535c47abc00a1f326" exitCode=0 Jul 15 13:06:43.652536 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:43.652024 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7606-139635308448816" event={"ID":"f83278de-4f4a-4191-a58f-f5568b8c20e3","Type":"ContainerDied","Data":"359edbbf490ac056bf2a2c7c438a6e832c19d02d9d46dfb535c47abc00a1f326"} Jul 15 13:06:44.800443 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:44.800412 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7606-139635308448816" Jul 15 13:06:44.838600 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:44.838572 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7606-139635308448816_f83278de-4f4a-4191-a58f-f5568b8c20e3/test-curl-7606-139635308448816/0.log" Jul 15 13:06:44.839858 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:44.839839 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cptd5\" (UniqueName: \"kubernetes.io/projected/f83278de-4f4a-4191-a58f-f5568b8c20e3-kube-api-access-cptd5\") pod \"f83278de-4f4a-4191-a58f-f5568b8c20e3\" (UID: \"f83278de-4f4a-4191-a58f-f5568b8c20e3\") " Jul 15 13:06:44.841925 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:44.841901 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f83278de-4f4a-4191-a58f-f5568b8c20e3-kube-api-access-cptd5" (OuterVolumeSpecName: "kube-api-access-cptd5") pod "f83278de-4f4a-4191-a58f-f5568b8c20e3" (UID: "f83278de-4f4a-4191-a58f-f5568b8c20e3"). InnerVolumeSpecName "kube-api-access-cptd5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 13:06:44.868657 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:44.868604 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7606-139635308448816"] Jul 15 13:06:44.870898 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:44.870879 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7606-139635308448816"] Jul 15 13:06:44.941578 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:44.941505 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cptd5\" (UniqueName: \"kubernetes.io/projected/f83278de-4f4a-4191-a58f-f5568b8c20e3-kube-api-access-cptd5\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 13:06:45.409322 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:45.409288 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f83278de-4f4a-4191-a58f-f5568b8c20e3" path="/var/lib/kubelet/pods/f83278de-4f4a-4191-a58f-f5568b8c20e3/volumes" Jul 15 13:06:45.666557 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:45.666477 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7606-139635308448816" Jul 15 13:06:45.666557 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:45.666527 2560 scope.go:117] "RemoveContainer" containerID="359edbbf490ac056bf2a2c7c438a6e832c19d02d9d46dfb535c47abc00a1f326" Jul 15 13:06:51.202135 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:51.202057 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7606-139635315250352"] Jul 15 13:06:51.202519 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:51.202464 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f83278de-4f4a-4191-a58f-f5568b8c20e3" containerName="test-curl-7606-139635308448816" Jul 15 13:06:51.202519 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:51.202477 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f83278de-4f4a-4191-a58f-f5568b8c20e3" containerName="test-curl-7606-139635308448816" Jul 15 13:06:51.202592 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:51.202551 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f83278de-4f4a-4191-a58f-f5568b8c20e3" containerName="test-curl-7606-139635308448816" Jul 15 13:06:51.206944 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:51.206927 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7606-139635315250352" Jul 15 13:06:51.209897 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:51.209871 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-664xg\"" Jul 15 13:06:51.212743 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:51.212723 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7606-139635315250352"] Jul 15 13:06:51.296592 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:51.296558 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmjtb\" (UniqueName: \"kubernetes.io/projected/c60d78c6-086e-4dfc-b6af-422726b6ea5a-kube-api-access-pmjtb\") pod \"test-curl-7606-139635315250352\" (UID: \"c60d78c6-086e-4dfc-b6af-422726b6ea5a\") " pod="opendatahub/test-curl-7606-139635315250352" Jul 15 13:06:51.397847 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:51.397812 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmjtb\" (UniqueName: \"kubernetes.io/projected/c60d78c6-086e-4dfc-b6af-422726b6ea5a-kube-api-access-pmjtb\") pod \"test-curl-7606-139635315250352\" (UID: \"c60d78c6-086e-4dfc-b6af-422726b6ea5a\") " pod="opendatahub/test-curl-7606-139635315250352" Jul 15 13:06:51.406118 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:51.406095 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmjtb\" (UniqueName: \"kubernetes.io/projected/c60d78c6-086e-4dfc-b6af-422726b6ea5a-kube-api-access-pmjtb\") pod \"test-curl-7606-139635315250352\" (UID: \"c60d78c6-086e-4dfc-b6af-422726b6ea5a\") " pod="opendatahub/test-curl-7606-139635315250352" Jul 15 13:06:51.517391 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:51.517311 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7606-139635315250352" Jul 15 13:06:51.653051 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:51.653026 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7606-139635315250352"] Jul 15 13:06:51.655884 ip-10-0-139-188 kubenswrapper[2560]: W0715 13:06:51.655855 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc60d78c6_086e_4dfc_b6af_422726b6ea5a.slice/crio-57a367a6c90c6bf662475bdec333de8b6839d63c41db22f5a4a41ffa4fbfd13c WatchSource:0}: Error finding container 57a367a6c90c6bf662475bdec333de8b6839d63c41db22f5a4a41ffa4fbfd13c: Status 404 returned error can't find the container with id 57a367a6c90c6bf662475bdec333de8b6839d63c41db22f5a4a41ffa4fbfd13c Jul 15 13:06:51.708863 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:51.708829 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7606-139635315250352" event={"ID":"c60d78c6-086e-4dfc-b6af-422726b6ea5a","Type":"ContainerStarted","Data":"57a367a6c90c6bf662475bdec333de8b6839d63c41db22f5a4a41ffa4fbfd13c"} Jul 15 13:06:52.716135 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:52.716046 2560 generic.go:356] "Generic (PLEG): container finished" podID="c60d78c6-086e-4dfc-b6af-422726b6ea5a" containerID="23a332b4c837d427469eac9e20c36b701042ae0ac493cdd139a6c0a87f82de51" exitCode=0 Jul 15 13:06:52.716135 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:52.716100 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7606-139635315250352" event={"ID":"c60d78c6-086e-4dfc-b6af-422726b6ea5a","Type":"ContainerDied","Data":"23a332b4c837d427469eac9e20c36b701042ae0ac493cdd139a6c0a87f82de51"} Jul 15 13:06:53.862991 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:53.862963 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7606-139635315250352" Jul 15 13:06:53.884252 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:53.884146 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7606-139635315250352_c60d78c6-086e-4dfc-b6af-422726b6ea5a/test-curl-7606-139635315250352/0.log" Jul 15 13:06:53.912462 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:53.912418 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7606-139635315250352"] Jul 15 13:06:53.915691 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:53.915670 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7606-139635315250352"] Jul 15 13:06:53.925835 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:53.925816 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmjtb\" (UniqueName: \"kubernetes.io/projected/c60d78c6-086e-4dfc-b6af-422726b6ea5a-kube-api-access-pmjtb\") pod \"c60d78c6-086e-4dfc-b6af-422726b6ea5a\" (UID: \"c60d78c6-086e-4dfc-b6af-422726b6ea5a\") " Jul 15 13:06:53.927833 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:53.927813 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c60d78c6-086e-4dfc-b6af-422726b6ea5a-kube-api-access-pmjtb" (OuterVolumeSpecName: "kube-api-access-pmjtb") pod "c60d78c6-086e-4dfc-b6af-422726b6ea5a" (UID: "c60d78c6-086e-4dfc-b6af-422726b6ea5a"). InnerVolumeSpecName "kube-api-access-pmjtb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 13:06:54.026991 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:54.026890 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmjtb\" (UniqueName: \"kubernetes.io/projected/c60d78c6-086e-4dfc-b6af-422726b6ea5a-kube-api-access-pmjtb\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 13:06:54.728232 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:54.728204 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7606-139635315250352" Jul 15 13:06:54.728427 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:54.728203 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="57a367a6c90c6bf662475bdec333de8b6839d63c41db22f5a4a41ffa4fbfd13c" Jul 15 13:06:54.738070 ip-10-0-139-188 kubenswrapper[2560]: E0715 13:06:54.738045 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7606-139635315250352\" is forbidden: User \"system:node:ip-10-0-139-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-188.ec2.internal' and this object" podUID="c60d78c6-086e-4dfc-b6af-422726b6ea5a" pod="opendatahub/test-curl-7606-139635315250352" Jul 15 13:06:55.409320 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:06:55.409290 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c60d78c6-086e-4dfc-b6af-422726b6ea5a" path="/var/lib/kubelet/pods/c60d78c6-086e-4dfc-b6af-422726b6ea5a/volumes" Jul 15 13:07:23.130996 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:23.130966 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn"] Jul 15 13:07:23.131541 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:23.131179 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" podUID="ef5e6d30-25e0-415c-8a55-6f97be4e4caf" containerName="istio-proxy" containerID="cri-o://a0e8be59ea715f21755259e8ed9a49de8692177b984180696bf959611a17b4f3" gracePeriod=30 Jul 15 13:07:28.401040 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.400975 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 13:07:28.462074 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.462041 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-istio-data\") pod \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " Jul 15 13:07:28.462278 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.462090 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-istio-envoy\") pod \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " Jul 15 13:07:28.462278 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.462106 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-istio-token\") pod \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " Jul 15 13:07:28.462278 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.462125 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-istiod-ca-cert\") pod \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " Jul 15 13:07:28.462278 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.462230 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-credential-socket\") pod \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " Jul 15 13:07:28.462506 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.462289 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-workload-socket\") pod \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " Jul 15 13:07:28.462506 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.462354 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-workload-certs\") pod \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " Jul 15 13:07:28.462506 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.462384 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-istio-podinfo\") pod \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " Jul 15 13:07:28.462506 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.462444 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbjbx\" (UniqueName: \"kubernetes.io/projected/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-kube-api-access-vbjbx\") pod \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\" (UID: \"ef5e6d30-25e0-415c-8a55-6f97be4e4caf\") " Jul 15 13:07:28.462849 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.462706 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ef5e6d30-25e0-415c-8a55-6f97be4e4caf" (UID: "ef5e6d30-25e0-415c-8a55-6f97be4e4caf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 13:07:28.462849 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.462768 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ef5e6d30-25e0-415c-8a55-6f97be4e4caf" (UID: "ef5e6d30-25e0-415c-8a55-6f97be4e4caf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 13:07:28.462849 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.462822 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-credential-socket\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 13:07:28.463079 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.463038 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ef5e6d30-25e0-415c-8a55-6f97be4e4caf" (UID: "ef5e6d30-25e0-415c-8a55-6f97be4e4caf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 13:07:28.463133 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.463077 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-istio-data" (OuterVolumeSpecName: "istio-data") pod "ef5e6d30-25e0-415c-8a55-6f97be4e4caf" (UID: "ef5e6d30-25e0-415c-8a55-6f97be4e4caf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 13:07:28.463205 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.463183 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ef5e6d30-25e0-415c-8a55-6f97be4e4caf" (UID: "ef5e6d30-25e0-415c-8a55-6f97be4e4caf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 13:07:28.464527 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.464495 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ef5e6d30-25e0-415c-8a55-6f97be4e4caf" (UID: "ef5e6d30-25e0-415c-8a55-6f97be4e4caf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 13:07:28.464814 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.464790 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-istio-token" (OuterVolumeSpecName: "istio-token") pod "ef5e6d30-25e0-415c-8a55-6f97be4e4caf" (UID: "ef5e6d30-25e0-415c-8a55-6f97be4e4caf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 13:07:28.464926 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.464893 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-kube-api-access-vbjbx" (OuterVolumeSpecName: "kube-api-access-vbjbx") pod "ef5e6d30-25e0-415c-8a55-6f97be4e4caf" (UID: "ef5e6d30-25e0-415c-8a55-6f97be4e4caf"). InnerVolumeSpecName "kube-api-access-vbjbx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 13:07:28.465352 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.465333 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ef5e6d30-25e0-415c-8a55-6f97be4e4caf" (UID: "ef5e6d30-25e0-415c-8a55-6f97be4e4caf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 13:07:28.563886 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.563851 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-workload-socket\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 13:07:28.563886 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.563878 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-workload-certs\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 13:07:28.563886 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.563887 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-istio-podinfo\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 13:07:28.564118 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.563895 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbjbx\" (UniqueName: \"kubernetes.io/projected/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-kube-api-access-vbjbx\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 13:07:28.564118 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.563906 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-istio-data\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 13:07:28.564118 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.563914 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-istio-envoy\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 13:07:28.564118 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.563921 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-istio-token\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 13:07:28.564118 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.563929 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef5e6d30-25e0-415c-8a55-6f97be4e4caf-istiod-ca-cert\") on node \"ip-10-0-139-188.ec2.internal\" DevicePath \"\"" Jul 15 13:07:28.922108 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.922070 2560 generic.go:356] "Generic (PLEG): container finished" podID="ef5e6d30-25e0-415c-8a55-6f97be4e4caf" containerID="a0e8be59ea715f21755259e8ed9a49de8692177b984180696bf959611a17b4f3" exitCode=0 Jul 15 13:07:28.922294 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.922131 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" Jul 15 13:07:28.922294 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.922147 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" event={"ID":"ef5e6d30-25e0-415c-8a55-6f97be4e4caf","Type":"ContainerDied","Data":"a0e8be59ea715f21755259e8ed9a49de8692177b984180696bf959611a17b4f3"} Jul 15 13:07:28.922294 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.922242 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn" event={"ID":"ef5e6d30-25e0-415c-8a55-6f97be4e4caf","Type":"ContainerDied","Data":"24bfb8b8eb7aa56a1eeda5596d8d95b5dd34376021105b61add31f0ce47df2d4"} Jul 15 13:07:28.922294 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.922268 2560 scope.go:117] "RemoveContainer" containerID="a0e8be59ea715f21755259e8ed9a49de8692177b984180696bf959611a17b4f3" Jul 15 13:07:28.931963 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.931943 2560 scope.go:117] "RemoveContainer" containerID="0a573c3959e2758e4a13506a60701b36751c0dce69af931ec0def4515a65fcee" Jul 15 13:07:28.940270 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.940251 2560 scope.go:117] "RemoveContainer" containerID="a0e8be59ea715f21755259e8ed9a49de8692177b984180696bf959611a17b4f3" Jul 15 13:07:28.940505 ip-10-0-139-188 kubenswrapper[2560]: E0715 13:07:28.940485 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0e8be59ea715f21755259e8ed9a49de8692177b984180696bf959611a17b4f3\": container with ID starting with a0e8be59ea715f21755259e8ed9a49de8692177b984180696bf959611a17b4f3 not found: ID does not exist" containerID="a0e8be59ea715f21755259e8ed9a49de8692177b984180696bf959611a17b4f3" Jul 15 13:07:28.940553 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.940515 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0e8be59ea715f21755259e8ed9a49de8692177b984180696bf959611a17b4f3"} err="failed to get container status \"a0e8be59ea715f21755259e8ed9a49de8692177b984180696bf959611a17b4f3\": rpc error: code = NotFound desc = could not find container \"a0e8be59ea715f21755259e8ed9a49de8692177b984180696bf959611a17b4f3\": container with ID starting with a0e8be59ea715f21755259e8ed9a49de8692177b984180696bf959611a17b4f3 not found: ID does not exist" Jul 15 13:07:28.940553 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.940533 2560 scope.go:117] "RemoveContainer" containerID="0a573c3959e2758e4a13506a60701b36751c0dce69af931ec0def4515a65fcee" Jul 15 13:07:28.940803 ip-10-0-139-188 kubenswrapper[2560]: E0715 13:07:28.940786 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a573c3959e2758e4a13506a60701b36751c0dce69af931ec0def4515a65fcee\": container with ID starting with 0a573c3959e2758e4a13506a60701b36751c0dce69af931ec0def4515a65fcee not found: ID does not exist" containerID="0a573c3959e2758e4a13506a60701b36751c0dce69af931ec0def4515a65fcee" Jul 15 13:07:28.940872 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.940812 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a573c3959e2758e4a13506a60701b36751c0dce69af931ec0def4515a65fcee"} err="failed to get container status \"0a573c3959e2758e4a13506a60701b36751c0dce69af931ec0def4515a65fcee\": rpc error: code = NotFound desc = could not find container \"0a573c3959e2758e4a13506a60701b36751c0dce69af931ec0def4515a65fcee\": container with ID starting with 0a573c3959e2758e4a13506a60701b36751c0dce69af931ec0def4515a65fcee not found: ID does not exist" Jul 15 13:07:28.944845 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.944825 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn"] Jul 15 13:07:28.949865 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:28.949841 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-3af497-openshift-default-56d59dd547-m4jdn"] Jul 15 13:07:29.408696 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:29.408663 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef5e6d30-25e0-415c-8a55-6f97be4e4caf" path="/var/lib/kubelet/pods/ef5e6d30-25e0-415c-8a55-6f97be4e4caf/volumes" Jul 15 13:07:55.376713 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:55.376663 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-vrxrj_baf577c8-f3f4-4d3a-b525-dfa341a0a771/manager/0.log" Jul 15 13:07:55.750695 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:55.750600 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-gk6xz_65e23de7-f6fd-4018-b78e-fa388c118295/server/0.log" Jul 15 13:07:55.867130 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:55.867102 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-5dc4q_a7189dee-11f0-4215-ac65-1b3979234a21/manager/0.log" Jul 15 13:07:56.111867 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:56.111838 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f56ddd455-vzpvx_a5d0d0db-7435-4204-9255-12e3d3ed844d/copy-manifests/0.log" Jul 15 13:07:56.118943 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:56.118920 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f56ddd455-vzpvx_a5d0d0db-7435-4204-9255-12e3d3ed844d/manager/0.log" Jul 15 13:07:56.930953 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:56.930926 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-2bbbbc27-qdxrt_81e1f446-779d-4d41-9d35-0757951875f6/revoke-keys/0.log" Jul 15 13:07:57.481058 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:57.481025 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-4060a9-v8mlt_4f11da84-53fc-41bf-92f5-00b65f16f402/revoke-keys/0.log" Jul 15 13:07:59.029150 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:59.029119 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8_77f52d1d-5d55-4e20-90fd-e79abe8c1734/util/0.log" Jul 15 13:07:59.035905 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:59.035885 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8_77f52d1d-5d55-4e20-90fd-e79abe8c1734/pull/0.log" Jul 15 13:07:59.042838 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:59.042818 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8_77f52d1d-5d55-4e20-90fd-e79abe8c1734/extract/0.log" Jul 15 13:07:59.156012 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:59.155982 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq_5c2ec5a7-8c26-4c79-9118-f8a8af840042/util/0.log" Jul 15 13:07:59.162809 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:59.162788 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq_5c2ec5a7-8c26-4c79-9118-f8a8af840042/pull/0.log" Jul 15 13:07:59.169459 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:59.169437 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq_5c2ec5a7-8c26-4c79-9118-f8a8af840042/extract/0.log" Jul 15 13:07:59.280807 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:59.280729 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49_050ecdb1-3d3e-454d-abb1-723157f29c5a/pull/0.log" Jul 15 13:07:59.287798 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:59.287784 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49_050ecdb1-3d3e-454d-abb1-723157f29c5a/extract/0.log" Jul 15 13:07:59.294284 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:59.294268 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49_050ecdb1-3d3e-454d-abb1-723157f29c5a/util/0.log" Jul 15 13:07:59.403695 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:59.403666 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87_fed32aad-0e99-44a1-86cf-43839588c5e2/util/0.log" Jul 15 13:07:59.409886 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:59.409863 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87_fed32aad-0e99-44a1-86cf-43839588c5e2/pull/0.log" Jul 15 13:07:59.416006 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:59.415987 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87_fed32aad-0e99-44a1-86cf-43839588c5e2/extract/0.log" Jul 15 13:07:59.875881 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:59.875846 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-9fplt_1dde5304-3dde-4ae0-bb1e-1b9422a3a447/kuadrant-console-plugin/0.log" Jul 15 13:07:59.991377 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:07:59.991345 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-r95g2_51609fa2-4c5e-4031-8df6-11319aad0f4c/registry-server/0.log" Jul 15 13:08:00.111561 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:00.111537 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-cd7hl_b2677353-0ada-4a87-b16f-8dade2040585/manager/0.log" Jul 15 13:08:00.352344 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:00.352318 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-m8ht4_4f16448a-6aa2-425b-9449-138af341ded4/manager/0.log" Jul 15 13:08:00.707216 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:00.707134 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678l2zsr_c6fa3be8-3559-4e2f-882c-a4bc7b84c55a/istio-proxy/0.log" Jul 15 13:08:00.828911 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:00.828883 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-7bzkz_63ea41ea-c57e-4adc-a87f-6ac5436abfc1/discovery/0.log" Jul 15 13:08:01.056843 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:01.056819 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c484c84b4-qxxxh_0a6955c9-30b4-42be-a3a8-e1debd48a8b0/kube-auth-proxy/0.log" Jul 15 13:08:01.415965 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:01.415891 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-fbfx6_9ffbbb5d-6dd5-426e-ab11-2678758860b8/payload-processing/0.log" Jul 15 13:08:02.098430 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:02.098399 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-qw62t_437dc02b-4909-4003-b9f4-8fac0bba4215/main/0.log" Jul 15 13:08:02.328060 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:02.328032 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-m66vc_a0e14eb8-ef78-4034-848e-ed97f4960d8e/main/0.log" Jul 15 13:08:11.052320 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:11.052289 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-79bqk_a919b128-8dca-4cee-81a2-51a65300820c/global-pull-secret-syncer/0.log" Jul 15 13:08:11.241453 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:11.241426 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-rw5hk_7c84d9e6-e6f4-4bc0-a95e-707e501a2d80/konnectivity-agent/0.log" Jul 15 13:08:11.325241 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:11.325161 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-188.ec2.internal_a30df3ddb10f29f1462b23c270b4893e/haproxy/0.log" Jul 15 13:08:15.256246 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:15.256214 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8_77f52d1d-5d55-4e20-90fd-e79abe8c1734/extract/0.log" Jul 15 13:08:15.283116 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:15.283093 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8_77f52d1d-5d55-4e20-90fd-e79abe8c1734/util/0.log" Jul 15 13:08:15.316258 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:15.316236 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595zns8_77f52d1d-5d55-4e20-90fd-e79abe8c1734/pull/0.log" Jul 15 13:08:15.352251 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:15.352228 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq_5c2ec5a7-8c26-4c79-9118-f8a8af840042/extract/0.log" Jul 15 13:08:15.375804 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:15.375783 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq_5c2ec5a7-8c26-4c79-9118-f8a8af840042/util/0.log" Jul 15 13:08:15.397655 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:15.397615 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zbhtq_5c2ec5a7-8c26-4c79-9118-f8a8af840042/pull/0.log" Jul 15 13:08:15.424769 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:15.424752 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49_050ecdb1-3d3e-454d-abb1-723157f29c5a/extract/0.log" Jul 15 13:08:15.448202 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:15.448182 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49_050ecdb1-3d3e-454d-abb1-723157f29c5a/util/0.log" Jul 15 13:08:15.470388 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:15.470370 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cct49_050ecdb1-3d3e-454d-abb1-723157f29c5a/pull/0.log" Jul 15 13:08:15.498243 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:15.498221 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87_fed32aad-0e99-44a1-86cf-43839588c5e2/extract/0.log" Jul 15 13:08:15.521246 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:15.521184 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87_fed32aad-0e99-44a1-86cf-43839588c5e2/util/0.log" Jul 15 13:08:15.545858 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:15.545839 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s9q87_fed32aad-0e99-44a1-86cf-43839588c5e2/pull/0.log" Jul 15 13:08:16.100773 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:16.100747 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-9fplt_1dde5304-3dde-4ae0-bb1e-1b9422a3a447/kuadrant-console-plugin/0.log" Jul 15 13:08:16.146417 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:16.146388 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-r95g2_51609fa2-4c5e-4031-8df6-11319aad0f4c/registry-server/0.log" Jul 15 13:08:16.248781 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:16.248752 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-cd7hl_b2677353-0ada-4a87-b16f-8dade2040585/manager/0.log" Jul 15 13:08:16.335555 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:16.335528 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-m8ht4_4f16448a-6aa2-425b-9449-138af341ded4/manager/0.log" Jul 15 13:08:18.136436 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:18.136406 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_792323f7-96a2-4633-91b8-94a6c9f42e30/alertmanager/0.log" Jul 15 13:08:18.167283 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:18.167260 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_792323f7-96a2-4633-91b8-94a6c9f42e30/config-reloader/0.log" Jul 15 13:08:18.192076 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:18.192052 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_792323f7-96a2-4633-91b8-94a6c9f42e30/kube-rbac-proxy-web/0.log" Jul 15 13:08:18.222111 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:18.222085 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_792323f7-96a2-4633-91b8-94a6c9f42e30/kube-rbac-proxy/0.log" Jul 15 13:08:18.249472 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:18.249450 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_792323f7-96a2-4633-91b8-94a6c9f42e30/kube-rbac-proxy-metric/0.log" Jul 15 13:08:18.273680 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:18.273656 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_792323f7-96a2-4633-91b8-94a6c9f42e30/prom-label-proxy/0.log" Jul 15 13:08:18.301014 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:18.300993 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_792323f7-96a2-4633-91b8-94a6c9f42e30/init-config-reloader/0.log" Jul 15 13:08:18.381771 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:18.381747 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-vrglv_a104d556-6b6d-4d2c-b54d-5c4567fa05cf/kube-state-metrics/0.log" Jul 15 13:08:18.408213 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:18.408136 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-vrglv_a104d556-6b6d-4d2c-b54d-5c4567fa05cf/kube-rbac-proxy-main/0.log" Jul 15 13:08:18.434382 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:18.434355 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-vrglv_a104d556-6b6d-4d2c-b54d-5c4567fa05cf/kube-rbac-proxy-self/0.log" Jul 15 13:08:18.477477 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:18.477453 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-788f455ccc-8tj2l_d958af7b-382f-499e-86ac-0f7ea1f3c2dd/metrics-server/0.log" Jul 15 13:08:18.540648 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:18.540621 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-96pbm_cad0427d-526a-408e-8735-9dd31593821e/node-exporter/0.log" Jul 15 13:08:18.565777 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:18.565732 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-96pbm_cad0427d-526a-408e-8735-9dd31593821e/kube-rbac-proxy/0.log" Jul 15 13:08:18.596485 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:18.596456 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-96pbm_cad0427d-526a-408e-8735-9dd31593821e/init-textfile/0.log" Jul 15 13:08:18.813459 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:18.813391 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-qr5rr_268eb654-ee5f-408e-bf43-7476e962fdf4/kube-rbac-proxy-main/0.log" Jul 15 13:08:18.838609 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:18.838587 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-qr5rr_268eb654-ee5f-408e-bf43-7476e962fdf4/kube-rbac-proxy-self/0.log" Jul 15 13:08:18.865221 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:18.865201 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-qr5rr_268eb654-ee5f-408e-bf43-7476e962fdf4/openshift-state-metrics/0.log" Jul 15 13:08:19.182848 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:19.182753 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-78c4b465b9-9qbqf_498584a2-e27b-48c5-9eb0-14c6245268f2/telemeter-client/0.log" Jul 15 13:08:19.208433 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:19.208408 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-78c4b465b9-9qbqf_498584a2-e27b-48c5-9eb0-14c6245268f2/reload/0.log" Jul 15 13:08:19.234752 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:19.234727 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-78c4b465b9-9qbqf_498584a2-e27b-48c5-9eb0-14c6245268f2/kube-rbac-proxy/0.log" Jul 15 13:08:19.267867 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:19.267844 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-776d57b8cc-mh8ll_38e690a5-da92-4aa4-9d66-b4de8a6748f6/thanos-query/0.log" Jul 15 13:08:19.299725 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:19.299705 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-776d57b8cc-mh8ll_38e690a5-da92-4aa4-9d66-b4de8a6748f6/kube-rbac-proxy-web/0.log" Jul 15 13:08:19.331498 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:19.331478 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-776d57b8cc-mh8ll_38e690a5-da92-4aa4-9d66-b4de8a6748f6/kube-rbac-proxy/0.log" Jul 15 13:08:19.358015 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:19.357995 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-776d57b8cc-mh8ll_38e690a5-da92-4aa4-9d66-b4de8a6748f6/prom-label-proxy/0.log" Jul 15 13:08:19.382654 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:19.382620 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-776d57b8cc-mh8ll_38e690a5-da92-4aa4-9d66-b4de8a6748f6/kube-rbac-proxy-rules/0.log" Jul 15 13:08:19.407106 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:19.407088 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-776d57b8cc-mh8ll_38e690a5-da92-4aa4-9d66-b4de8a6748f6/kube-rbac-proxy-metrics/0.log" Jul 15 13:08:20.928737 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:20.928701 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8mxws/perf-node-gather-daemonset-5bz46"] Jul 15 13:08:20.929306 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:20.929287 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef5e6d30-25e0-415c-8a55-6f97be4e4caf" containerName="istio-proxy" Jul 15 13:08:20.929397 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:20.929336 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef5e6d30-25e0-415c-8a55-6f97be4e4caf" containerName="istio-proxy" Jul 15 13:08:20.929397 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:20.929362 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef5e6d30-25e0-415c-8a55-6f97be4e4caf" containerName="istio-proxy" Jul 15 13:08:20.929397 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:20.929371 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef5e6d30-25e0-415c-8a55-6f97be4e4caf" containerName="istio-proxy" Jul 15 13:08:20.929397 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:20.929379 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c60d78c6-086e-4dfc-b6af-422726b6ea5a" containerName="test-curl-7606-139635315250352" Jul 15 13:08:20.929397 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:20.929389 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c60d78c6-086e-4dfc-b6af-422726b6ea5a" containerName="test-curl-7606-139635315250352" Jul 15 13:08:20.929667 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:20.929498 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c60d78c6-086e-4dfc-b6af-422726b6ea5a" containerName="test-curl-7606-139635315250352" Jul 15 13:08:20.929667 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:20.929514 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef5e6d30-25e0-415c-8a55-6f97be4e4caf" containerName="istio-proxy" Jul 15 13:08:20.929667 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:20.929525 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef5e6d30-25e0-415c-8a55-6f97be4e4caf" containerName="istio-proxy" Jul 15 13:08:20.932679 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:20.932659 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-5bz46" Jul 15 13:08:20.935198 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:20.935182 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8mxws\"/\"openshift-service-ca.crt\"" Jul 15 13:08:20.935357 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:20.935338 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8mxws\"/\"kube-root-ca.crt\"" Jul 15 13:08:20.936134 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:20.936120 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8mxws\"/\"default-dockercfg-7bjhq\"" Jul 15 13:08:20.943248 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:20.943226 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8mxws/perf-node-gather-daemonset-5bz46"] Jul 15 13:08:21.070330 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:21.070304 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d82mp\" (UniqueName: \"kubernetes.io/projected/47819d5c-f76f-4189-b70a-1121e2246960-kube-api-access-d82mp\") pod \"perf-node-gather-daemonset-5bz46\" (UID: \"47819d5c-f76f-4189-b70a-1121e2246960\") " pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-5bz46" Jul 15 13:08:21.070501 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:21.070353 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/47819d5c-f76f-4189-b70a-1121e2246960-proc\") pod \"perf-node-gather-daemonset-5bz46\" (UID: \"47819d5c-f76f-4189-b70a-1121e2246960\") " pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-5bz46" Jul 15 13:08:21.070501 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:21.070370 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/47819d5c-f76f-4189-b70a-1121e2246960-lib-modules\") pod \"perf-node-gather-daemonset-5bz46\" (UID: \"47819d5c-f76f-4189-b70a-1121e2246960\") " pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-5bz46" Jul 15 13:08:21.070501 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:21.070413 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/47819d5c-f76f-4189-b70a-1121e2246960-sys\") pod \"perf-node-gather-daemonset-5bz46\" (UID: \"47819d5c-f76f-4189-b70a-1121e2246960\") " pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-5bz46" Jul 15 13:08:21.070501 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:21.070487 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/47819d5c-f76f-4189-b70a-1121e2246960-podres\") pod \"perf-node-gather-daemonset-5bz46\" (UID: \"47819d5c-f76f-4189-b70a-1121e2246960\") " pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-5bz46" Jul 15 13:08:21.171600 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:21.171568 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/47819d5c-f76f-4189-b70a-1121e2246960-proc\") pod \"perf-node-gather-daemonset-5bz46\" (UID: \"47819d5c-f76f-4189-b70a-1121e2246960\") " pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-5bz46" Jul 15 13:08:21.171600 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:21.171600 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/47819d5c-f76f-4189-b70a-1121e2246960-lib-modules\") pod \"perf-node-gather-daemonset-5bz46\" (UID: \"47819d5c-f76f-4189-b70a-1121e2246960\") " pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-5bz46" Jul 15 13:08:21.171809 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:21.171619 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/47819d5c-f76f-4189-b70a-1121e2246960-sys\") pod \"perf-node-gather-daemonset-5bz46\" (UID: \"47819d5c-f76f-4189-b70a-1121e2246960\") " pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-5bz46" Jul 15 13:08:21.171809 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:21.171699 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/47819d5c-f76f-4189-b70a-1121e2246960-podres\") pod \"perf-node-gather-daemonset-5bz46\" (UID: \"47819d5c-f76f-4189-b70a-1121e2246960\") " pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-5bz46" Jul 15 13:08:21.171809 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:21.171707 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/47819d5c-f76f-4189-b70a-1121e2246960-proc\") pod \"perf-node-gather-daemonset-5bz46\" (UID: \"47819d5c-f76f-4189-b70a-1121e2246960\") " pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-5bz46" Jul 15 13:08:21.171809 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:21.171769 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/47819d5c-f76f-4189-b70a-1121e2246960-sys\") pod \"perf-node-gather-daemonset-5bz46\" (UID: \"47819d5c-f76f-4189-b70a-1121e2246960\") " pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-5bz46" Jul 15 13:08:21.171809 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:21.171785 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/47819d5c-f76f-4189-b70a-1121e2246960-lib-modules\") pod \"perf-node-gather-daemonset-5bz46\" (UID: \"47819d5c-f76f-4189-b70a-1121e2246960\") " pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-5bz46" Jul 15 13:08:21.171809 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:21.171805 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/47819d5c-f76f-4189-b70a-1121e2246960-podres\") pod \"perf-node-gather-daemonset-5bz46\" (UID: \"47819d5c-f76f-4189-b70a-1121e2246960\") " pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-5bz46" Jul 15 13:08:21.172010 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:21.171861 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d82mp\" (UniqueName: \"kubernetes.io/projected/47819d5c-f76f-4189-b70a-1121e2246960-kube-api-access-d82mp\") pod \"perf-node-gather-daemonset-5bz46\" (UID: \"47819d5c-f76f-4189-b70a-1121e2246960\") " pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-5bz46" Jul 15 13:08:21.181135 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:21.181059 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d82mp\" (UniqueName: \"kubernetes.io/projected/47819d5c-f76f-4189-b70a-1121e2246960-kube-api-access-d82mp\") pod \"perf-node-gather-daemonset-5bz46\" (UID: \"47819d5c-f76f-4189-b70a-1121e2246960\") " pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-5bz46" Jul 15 13:08:21.244514 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:21.244482 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-5bz46" Jul 15 13:08:21.377241 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:21.377212 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8mxws/perf-node-gather-daemonset-5bz46"] Jul 15 13:08:21.379421 ip-10-0-139-188 kubenswrapper[2560]: W0715 13:08:21.379392 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod47819d5c_f76f_4189_b70a_1121e2246960.slice/crio-d8db7fe11b832e20bcec4052d170507281bcd8e0fdfb275196f893dc412f71e8 WatchSource:0}: Error finding container d8db7fe11b832e20bcec4052d170507281bcd8e0fdfb275196f893dc412f71e8: Status 404 returned error can't find the container with id d8db7fe11b832e20bcec4052d170507281bcd8e0fdfb275196f893dc412f71e8 Jul 15 13:08:22.222883 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:22.222853 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-5bz46" event={"ID":"47819d5c-f76f-4189-b70a-1121e2246960","Type":"ContainerStarted","Data":"74b48d724eb5707cd964a37700c40af95fafdc620dd5f908684487038262cf15"} Jul 15 13:08:22.223342 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:22.222909 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-5bz46" event={"ID":"47819d5c-f76f-4189-b70a-1121e2246960","Type":"ContainerStarted","Data":"d8db7fe11b832e20bcec4052d170507281bcd8e0fdfb275196f893dc412f71e8"} Jul 15 13:08:22.223342 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:22.222950 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-5bz46" Jul 15 13:08:22.240931 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:22.240887 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-5bz46" podStartSLOduration=2.240871056 podStartE2EDuration="2.240871056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:08:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:08:22.238715336 +0000 UTC m=+2875.379511474" watchObservedRunningTime="2026-07-15 13:08:22.240871056 +0000 UTC m=+2875.381667186" Jul 15 13:08:23.560009 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:23.559939 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5cpsl_5f742cf0-bd7c-4cf1-9e4a-1214def20eff/dns/0.log" Jul 15 13:08:23.584368 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:23.584347 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5cpsl_5f742cf0-bd7c-4cf1-9e4a-1214def20eff/kube-rbac-proxy/0.log" Jul 15 13:08:23.718890 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:23.718863 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6xkhw_74f12cb2-ed42-4446-bb5d-25cb7a992fa3/dns-node-resolver/0.log" Jul 15 13:08:24.342269 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:24.342239 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5dff78dbc9-kf6kz_f8d0552b-2236-45a7-b3bb-57681252e2e5/registry/0.log" Jul 15 13:08:24.422662 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:24.422617 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-m7tbh_05b26d09-b957-4b76-80fd-b30046fbbfc5/node-ca/0.log" Jul 15 13:08:25.459599 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:25.459570 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678l2zsr_c6fa3be8-3559-4e2f-882c-a4bc7b84c55a/istio-proxy/0.log" Jul 15 13:08:25.577494 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:25.577466 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-7bzkz_63ea41ea-c57e-4adc-a87f-6ac5436abfc1/discovery/0.log" Jul 15 13:08:25.629046 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:25.629022 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c484c84b4-qxxxh_0a6955c9-30b4-42be-a3a8-e1debd48a8b0/kube-auth-proxy/0.log" Jul 15 13:08:25.749647 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:25.749557 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-fbfx6_9ffbbb5d-6dd5-426e-ab11-2678758860b8/payload-processing/0.log" Jul 15 13:08:26.543733 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:26.543707 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rt7x5_4efdeca1-8ed8-43bb-9ec0-d7a54a30e3f9/serve-healthcheck-canary/0.log" Jul 15 13:08:27.120741 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:27.120716 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7g6wj_c883a479-e3e8-45b1-bcc3-027f8637183e/kube-rbac-proxy/0.log" Jul 15 13:08:27.145219 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:27.145190 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7g6wj_c883a479-e3e8-45b1-bcc3-027f8637183e/exporter/0.log" Jul 15 13:08:27.168491 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:27.168462 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7g6wj_c883a479-e3e8-45b1-bcc3-027f8637183e/extractor/0.log" Jul 15 13:08:28.238217 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:28.238188 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-5bz46" Jul 15 13:08:29.713205 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:29.713178 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-vrxrj_baf577c8-f3f4-4d3a-b525-dfa341a0a771/manager/0.log" Jul 15 13:08:29.967432 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:29.967402 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-gk6xz_65e23de7-f6fd-4018-b78e-fa388c118295/server/0.log" Jul 15 13:08:30.040321 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:30.040294 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-5dc4q_a7189dee-11f0-4215-ac65-1b3979234a21/manager/0.log" Jul 15 13:08:30.181061 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:30.181035 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f56ddd455-vzpvx_a5d0d0db-7435-4204-9255-12e3d3ed844d/manager/0.log" Jul 15 13:08:30.205068 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:30.205040 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f56ddd455-vzpvx_a5d0d0db-7435-4204-9255-12e3d3ed844d/copy-manifests/0.log" Jul 15 13:08:31.535847 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:31.535816 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678l2zsr_c6fa3be8-3559-4e2f-882c-a4bc7b84c55a/istio-proxy/0.log" Jul 15 13:08:31.646608 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:31.646581 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-7bzkz_63ea41ea-c57e-4adc-a87f-6ac5436abfc1/discovery/0.log" Jul 15 13:08:31.675599 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:31.675575 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c484c84b4-qxxxh_0a6955c9-30b4-42be-a3a8-e1debd48a8b0/kube-auth-proxy/0.log" Jul 15 13:08:31.683469 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:31.683445 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-n29vc_1d8f0d3c-1d10-4039-a30d-76892ed8839b/openshift-lws-operator/0.log" Jul 15 13:08:31.783178 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:31.783154 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-fbfx6_9ffbbb5d-6dd5-426e-ab11-2678758860b8/payload-processing/0.log" Jul 15 13:08:33.115382 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:33.115356 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678l2zsr_c6fa3be8-3559-4e2f-882c-a4bc7b84c55a/istio-proxy/0.log" Jul 15 13:08:33.239526 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:33.239499 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-7bzkz_63ea41ea-c57e-4adc-a87f-6ac5436abfc1/discovery/0.log" Jul 15 13:08:33.269612 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:33.269585 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c484c84b4-qxxxh_0a6955c9-30b4-42be-a3a8-e1debd48a8b0/kube-auth-proxy/0.log" Jul 15 13:08:33.364999 ip-10-0-139-188 kubenswrapper[2560]: I0715 13:08:33.364968 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-fbfx6_9ffbbb5d-6dd5-426e-ab11-2678758860b8/payload-processing/0.log"