May 28 16:04:55.572040 ip-10-0-143-102 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory May 28 16:04:55.572053 ip-10-0-143-102 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory May 28 16:04:55.572063 ip-10-0-143-102 systemd[1]: kubelet.service: Failed with result 'resources'. May 28 16:04:55.572435 ip-10-0-143-102 systemd[1]: Failed to start Kubernetes Kubelet. May 28 16:05:05.615746 ip-10-0-143-102 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. May 28 16:05:05.615767 ip-10-0-143-102 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot b0348a7219c744bfbd50f17cd9309085 -- May 28 16:07:40.662640 ip-10-0-143-102 systemd[1]: Starting Kubernetes Kubelet... May 28 16:07:41.072084 ip-10-0-143-102 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 28 16:07:41.072084 ip-10-0-143-102 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. May 28 16:07:41.072084 ip-10-0-143-102 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 28 16:07:41.072084 ip-10-0-143-102 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. May 28 16:07:41.072084 ip-10-0-143-102 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 28 16:07:41.072894 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.072802 2566 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" May 28 16:07:41.081416 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081393 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 28 16:07:41.081416 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081415 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 28 16:07:41.081496 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081419 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 28 16:07:41.081496 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081423 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 28 16:07:41.081496 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081426 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 28 16:07:41.081496 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081429 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 28 16:07:41.081496 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081432 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 28 16:07:41.081496 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081435 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 28 16:07:41.081496 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081438 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 28 16:07:41.081496 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081441 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 28 16:07:41.081496 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081443 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 28 16:07:41.081496 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081446 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 28 16:07:41.081496 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081449 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 28 16:07:41.081496 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081452 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 28 16:07:41.081496 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081454 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 28 16:07:41.081496 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081457 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 28 16:07:41.081496 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081460 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 28 16:07:41.081496 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081462 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 28 16:07:41.081496 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081464 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 28 16:07:41.081496 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081467 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 28 16:07:41.081496 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081470 2566 feature_gate.go:328] unrecognized feature gate: NewOLM May 28 16:07:41.081496 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081472 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 28 16:07:41.081979 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081475 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 28 16:07:41.081979 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081477 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 28 16:07:41.081979 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081480 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 28 16:07:41.081979 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081483 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 28 16:07:41.081979 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081485 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 28 16:07:41.081979 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081488 2566 feature_gate.go:328] unrecognized feature gate: Example2 May 28 16:07:41.081979 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081506 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 28 16:07:41.081979 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081510 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 28 16:07:41.081979 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081513 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 28 16:07:41.081979 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081517 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 28 16:07:41.081979 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081520 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 28 16:07:41.081979 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081522 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 28 16:07:41.081979 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081526 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 28 16:07:41.081979 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081529 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 28 16:07:41.081979 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081532 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 28 16:07:41.081979 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081535 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 28 16:07:41.081979 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081537 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages May 28 16:07:41.081979 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081539 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 28 16:07:41.081979 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081542 2566 feature_gate.go:328] unrecognized feature gate: Example May 28 16:07:41.081979 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081546 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 28 16:07:41.082490 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081550 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 28 16:07:41.082490 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081552 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 28 16:07:41.082490 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081555 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 28 16:07:41.082490 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081558 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 28 16:07:41.082490 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081560 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 28 16:07:41.082490 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081563 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 28 16:07:41.082490 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081565 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 28 16:07:41.082490 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081567 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 28 16:07:41.082490 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081570 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 28 16:07:41.082490 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081572 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 28 16:07:41.082490 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081575 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 28 16:07:41.082490 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081578 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 28 16:07:41.082490 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081581 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 28 16:07:41.082490 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081584 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability May 28 16:07:41.082490 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081587 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 28 16:07:41.082490 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081590 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 28 16:07:41.082490 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081592 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 28 16:07:41.082490 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081595 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 28 16:07:41.082490 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081597 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 28 16:07:41.082977 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081601 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 28 16:07:41.082977 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081605 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 28 16:07:41.082977 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081608 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 28 16:07:41.082977 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081611 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 28 16:07:41.082977 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081613 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 28 16:07:41.082977 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081616 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 28 16:07:41.082977 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081619 2566 feature_gate.go:328] unrecognized feature gate: DualReplica May 28 16:07:41.082977 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081621 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 28 16:07:41.082977 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081624 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 28 16:07:41.082977 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081626 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 28 16:07:41.082977 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081630 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 28 16:07:41.082977 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081633 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 28 16:07:41.082977 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081635 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 28 16:07:41.082977 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081638 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores May 28 16:07:41.082977 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081641 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 28 16:07:41.082977 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081643 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 28 16:07:41.082977 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081646 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 28 16:07:41.082977 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081648 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 28 16:07:41.082977 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081651 2566 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 28 16:07:41.083447 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081653 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 28 16:07:41.083447 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081656 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 28 16:07:41.083447 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081658 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 28 16:07:41.083447 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081661 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 28 16:07:41.083447 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081664 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 28 16:07:41.083447 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.081667 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 28 16:07:41.083447 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082092 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability May 28 16:07:41.083447 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082098 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 28 16:07:41.083447 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082101 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 28 16:07:41.083447 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082103 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 28 16:07:41.083447 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082106 2566 feature_gate.go:328] unrecognized feature gate: Example2 May 28 16:07:41.083447 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082109 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 28 16:07:41.083447 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082111 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 28 16:07:41.083447 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082114 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 28 16:07:41.083447 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082116 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 28 16:07:41.083447 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082119 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 28 16:07:41.083447 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082121 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 28 16:07:41.083447 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082124 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 28 16:07:41.083447 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082127 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 28 16:07:41.083447 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082129 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 28 16:07:41.083952 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082132 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 28 16:07:41.083952 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082134 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 28 16:07:41.083952 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082137 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 28 16:07:41.083952 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082139 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 28 16:07:41.083952 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082142 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 28 16:07:41.083952 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082144 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 28 16:07:41.083952 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082147 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 28 16:07:41.083952 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082150 2566 feature_gate.go:328] unrecognized feature gate: DualReplica May 28 16:07:41.083952 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082153 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 28 16:07:41.083952 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082155 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 28 16:07:41.083952 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082157 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 28 16:07:41.083952 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082160 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 28 16:07:41.083952 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082162 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 28 16:07:41.083952 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082166 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 28 16:07:41.083952 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082168 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 28 16:07:41.083952 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082171 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 28 16:07:41.083952 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082173 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 28 16:07:41.083952 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082176 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 28 16:07:41.083952 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082178 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores May 28 16:07:41.083952 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082182 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 28 16:07:41.084461 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082184 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 28 16:07:41.084461 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082187 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 28 16:07:41.084461 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082189 2566 feature_gate.go:328] unrecognized feature gate: NewOLM May 28 16:07:41.084461 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082192 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 28 16:07:41.084461 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082195 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 28 16:07:41.084461 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082197 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 28 16:07:41.084461 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082199 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 28 16:07:41.084461 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082202 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 28 16:07:41.084461 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082204 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 28 16:07:41.084461 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082207 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 28 16:07:41.084461 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082209 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 28 16:07:41.084461 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082211 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 28 16:07:41.084461 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082213 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 28 16:07:41.084461 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082216 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 28 16:07:41.084461 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082218 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 28 16:07:41.084461 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082221 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 28 16:07:41.084461 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082223 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 28 16:07:41.084461 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082226 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 28 16:07:41.084461 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082228 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 28 16:07:41.084957 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082230 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 28 16:07:41.084957 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082233 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 28 16:07:41.084957 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082236 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 28 16:07:41.084957 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082238 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 28 16:07:41.084957 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082240 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 28 16:07:41.084957 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082248 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 28 16:07:41.084957 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082251 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 28 16:07:41.084957 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082254 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 28 16:07:41.084957 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082256 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 28 16:07:41.084957 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082258 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 28 16:07:41.084957 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082261 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 28 16:07:41.084957 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082265 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 28 16:07:41.084957 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082269 2566 feature_gate.go:328] unrecognized feature gate: Example May 28 16:07:41.084957 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082272 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 28 16:07:41.084957 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082275 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 28 16:07:41.084957 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082295 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 28 16:07:41.084957 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082298 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 28 16:07:41.084957 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082301 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 28 16:07:41.084957 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082304 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 28 16:07:41.085443 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082307 2566 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 28 16:07:41.085443 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082310 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 28 16:07:41.085443 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082313 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 28 16:07:41.085443 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082317 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 28 16:07:41.085443 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082321 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 28 16:07:41.085443 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082327 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages May 28 16:07:41.085443 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082331 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 28 16:07:41.085443 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082335 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 28 16:07:41.085443 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082340 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 28 16:07:41.085443 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082344 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 28 16:07:41.085443 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082348 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 28 16:07:41.085443 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082351 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 28 16:07:41.085443 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082354 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 28 16:07:41.085443 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.082356 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 28 16:07:41.085443 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082457 2566 flags.go:64] FLAG: --address="0.0.0.0" May 28 16:07:41.085443 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082467 2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" May 28 16:07:41.085443 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082474 2566 flags.go:64] FLAG: --anonymous-auth="true" May 28 16:07:41.085443 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082479 2566 flags.go:64] FLAG: --application-metrics-count-limit="100" May 28 16:07:41.085443 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082486 2566 flags.go:64] FLAG: --authentication-token-webhook="false" May 28 16:07:41.085443 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082490 2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" May 28 16:07:41.085443 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082495 2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" May 28 16:07:41.085946 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082499 2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" May 28 16:07:41.085946 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082504 2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" May 28 16:07:41.085946 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082510 2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" May 28 16:07:41.085946 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082516 2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" May 28 16:07:41.085946 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082522 2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" May 28 16:07:41.085946 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082526 2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs" May 28 16:07:41.085946 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082529 2566 flags.go:64] FLAG: --cgroup-root="" May 28 16:07:41.085946 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082532 2566 flags.go:64] FLAG: --cgroups-per-qos="true" May 28 16:07:41.085946 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082535 2566 flags.go:64] FLAG: --client-ca-file="" May 28 16:07:41.085946 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082537 2566 flags.go:64] FLAG: --cloud-config="" May 28 16:07:41.085946 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082540 2566 flags.go:64] FLAG: --cloud-provider="external" May 28 16:07:41.085946 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082543 2566 flags.go:64] FLAG: --cluster-dns="[]" May 28 16:07:41.085946 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082549 2566 flags.go:64] FLAG: --cluster-domain="" May 28 16:07:41.085946 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082552 2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" May 28 16:07:41.085946 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082555 2566 flags.go:64] FLAG: --config-dir="" May 28 16:07:41.085946 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082558 2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" May 28 16:07:41.085946 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082561 2566 flags.go:64] FLAG: --container-log-max-files="5" May 28 16:07:41.085946 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082565 2566 flags.go:64] FLAG: --container-log-max-size="10Mi" May 28 16:07:41.085946 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082568 2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" May 28 16:07:41.085946 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082571 2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" May 28 16:07:41.085946 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082574 2566 flags.go:64] FLAG: --containerd-namespace="k8s.io" May 28 16:07:41.085946 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082577 2566 flags.go:64] FLAG: --contention-profiling="false" May 28 16:07:41.085946 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082580 2566 flags.go:64] FLAG: --cpu-cfs-quota="true" May 28 16:07:41.085946 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082584 2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" May 28 16:07:41.086549 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082590 2566 flags.go:64] FLAG: --cpu-manager-policy="none" May 28 16:07:41.086549 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082595 2566 flags.go:64] FLAG: --cpu-manager-policy-options="" May 28 16:07:41.086549 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082601 2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" May 28 16:07:41.086549 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082606 2566 flags.go:64] FLAG: --enable-controller-attach-detach="true" May 28 16:07:41.086549 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082609 2566 flags.go:64] FLAG: --enable-debugging-handlers="true" May 28 16:07:41.086549 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082613 2566 flags.go:64] FLAG: --enable-load-reader="false" May 28 16:07:41.086549 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082616 2566 flags.go:64] FLAG: --enable-server="true" May 28 16:07:41.086549 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082619 2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" May 28 16:07:41.086549 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082624 2566 flags.go:64] FLAG: --event-burst="100" May 28 16:07:41.086549 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082627 2566 flags.go:64] FLAG: --event-qps="50" May 28 16:07:41.086549 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082629 2566 flags.go:64] FLAG: --event-storage-age-limit="default=0" May 28 16:07:41.086549 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082633 2566 flags.go:64] FLAG: --event-storage-event-limit="default=0" May 28 16:07:41.086549 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082636 2566 flags.go:64] FLAG: --eviction-hard="" May 28 16:07:41.086549 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082640 2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" May 28 16:07:41.086549 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082643 2566 flags.go:64] FLAG: --eviction-minimum-reclaim="" May 28 16:07:41.086549 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082646 2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" May 28 16:07:41.086549 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082650 2566 flags.go:64] FLAG: --eviction-soft="" May 28 16:07:41.086549 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082652 2566 flags.go:64] FLAG: --eviction-soft-grace-period="" May 28 16:07:41.086549 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082655 2566 flags.go:64] FLAG: --exit-on-lock-contention="false" May 28 16:07:41.086549 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082658 2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" May 28 16:07:41.086549 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082661 2566 flags.go:64] FLAG: --experimental-mounter-path="" May 28 16:07:41.086549 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082667 2566 flags.go:64] FLAG: --fail-cgroupv1="false" May 28 16:07:41.086549 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082672 2566 flags.go:64] FLAG: --fail-swap-on="true" May 28 16:07:41.086549 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082678 2566 flags.go:64] FLAG: --feature-gates="" May 28 16:07:41.086549 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082683 2566 flags.go:64] FLAG: --file-check-frequency="20s" May 28 16:07:41.087167 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082686 2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" May 28 16:07:41.087167 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082689 2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" May 28 16:07:41.087167 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082692 2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" May 28 16:07:41.087167 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082695 2566 flags.go:64] FLAG: --healthz-port="10248" May 28 16:07:41.087167 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082698 2566 flags.go:64] FLAG: --help="false" May 28 16:07:41.087167 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082701 2566 flags.go:64] FLAG: --hostname-override="ip-10-0-143-102.ec2.internal" May 28 16:07:41.087167 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082705 2566 flags.go:64] FLAG: --housekeeping-interval="10s" May 28 16:07:41.087167 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082708 2566 flags.go:64] FLAG: --http-check-frequency="20s" May 28 16:07:41.087167 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082710 2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" May 28 16:07:41.087167 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082714 2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" May 28 16:07:41.087167 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082718 2566 flags.go:64] FLAG: --image-gc-high-threshold="85" May 28 16:07:41.087167 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082721 2566 flags.go:64] FLAG: --image-gc-low-threshold="80" May 28 16:07:41.087167 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082725 2566 flags.go:64] FLAG: --image-service-endpoint="" May 28 16:07:41.087167 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082728 2566 flags.go:64] FLAG: --kernel-memcg-notification="false" May 28 16:07:41.087167 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082731 2566 flags.go:64] FLAG: --kube-api-burst="100" May 28 16:07:41.087167 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082734 2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" May 28 16:07:41.087167 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082737 2566 flags.go:64] FLAG: --kube-api-qps="50" May 28 16:07:41.087167 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082740 2566 flags.go:64] FLAG: --kube-reserved="" May 28 16:07:41.087167 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082743 2566 flags.go:64] FLAG: --kube-reserved-cgroup="" May 28 16:07:41.087167 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082748 2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" May 28 16:07:41.087167 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082753 2566 flags.go:64] FLAG: --kubelet-cgroups="" May 28 16:07:41.087167 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082758 2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true" May 28 16:07:41.087167 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082762 2566 flags.go:64] FLAG: --lock-file="" May 28 16:07:41.087167 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082765 2566 flags.go:64] FLAG: --log-cadvisor-usage="false" May 28 16:07:41.087790 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082768 2566 flags.go:64] FLAG: --log-flush-frequency="5s" May 28 16:07:41.087790 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082770 2566 flags.go:64] FLAG: --log-json-info-buffer-size="0" May 28 16:07:41.087790 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082777 2566 flags.go:64] FLAG: --log-json-split-stream="false" May 28 16:07:41.087790 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082780 2566 flags.go:64] FLAG: --log-text-info-buffer-size="0" May 28 16:07:41.087790 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082783 2566 flags.go:64] FLAG: --log-text-split-stream="false" May 28 16:07:41.087790 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082786 2566 flags.go:64] FLAG: --logging-format="text" May 28 16:07:41.087790 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082788 2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" May 28 16:07:41.087790 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082792 2566 flags.go:64] FLAG: --make-iptables-util-chains="true" May 28 16:07:41.087790 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082795 2566 flags.go:64] FLAG: --manifest-url="" May 28 16:07:41.087790 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082798 2566 flags.go:64] FLAG: --manifest-url-header="" May 28 16:07:41.087790 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082802 2566 flags.go:64] FLAG: --max-housekeeping-interval="15s" May 28 16:07:41.087790 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082805 2566 flags.go:64] FLAG: --max-open-files="1000000" May 28 16:07:41.087790 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082809 2566 flags.go:64] FLAG: --max-pods="110" May 28 16:07:41.087790 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082812 2566 flags.go:64] FLAG: --maximum-dead-containers="-1" May 28 16:07:41.087790 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082815 2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" May 28 16:07:41.087790 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082818 2566 flags.go:64] FLAG: --memory-manager-policy="None" May 28 16:07:41.087790 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082821 2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" May 28 16:07:41.087790 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082827 2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" May 28 16:07:41.087790 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082832 2566 flags.go:64] FLAG: --node-ip="0.0.0.0" May 28 16:07:41.087790 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082837 2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" May 28 16:07:41.087790 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082850 2566 flags.go:64] FLAG: --node-status-max-images="50" May 28 16:07:41.087790 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082853 2566 flags.go:64] FLAG: --node-status-update-frequency="10s" May 28 16:07:41.087790 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082856 2566 flags.go:64] FLAG: --oom-score-adj="-999" May 28 16:07:41.087790 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082859 2566 flags.go:64] FLAG: --pod-cidr="" May 28 16:07:41.088372 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082862 2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc" May 28 16:07:41.088372 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082868 2566 flags.go:64] FLAG: --pod-manifest-path="" May 28 16:07:41.088372 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082871 2566 flags.go:64] FLAG: --pod-max-pids="-1" May 28 16:07:41.088372 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082874 2566 flags.go:64] FLAG: --pods-per-core="0" May 28 16:07:41.088372 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082877 2566 flags.go:64] FLAG: --port="10250" May 28 16:07:41.088372 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082880 2566 flags.go:64] FLAG: --protect-kernel-defaults="false" May 28 16:07:41.088372 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082883 2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02eaaadd59554ecee" May 28 16:07:41.088372 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082886 2566 flags.go:64] FLAG: --qos-reserved="" May 28 16:07:41.088372 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082889 2566 flags.go:64] FLAG: --read-only-port="10255" May 28 16:07:41.088372 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082892 2566 flags.go:64] FLAG: --register-node="true" May 28 16:07:41.088372 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082895 2566 flags.go:64] FLAG: --register-schedulable="true" May 28 16:07:41.088372 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082904 2566 flags.go:64] FLAG: --register-with-taints="" May 28 16:07:41.088372 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082910 2566 flags.go:64] FLAG: --registry-burst="10" May 28 16:07:41.088372 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082915 2566 flags.go:64] FLAG: --registry-qps="5" May 28 16:07:41.088372 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082919 2566 flags.go:64] FLAG: --reserved-cpus="" May 28 16:07:41.088372 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082923 2566 flags.go:64] FLAG: --reserved-memory="" May 28 16:07:41.088372 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082926 2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" May 28 16:07:41.088372 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082929 2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" May 28 16:07:41.088372 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082932 2566 flags.go:64] FLAG: --rotate-certificates="false" May 28 16:07:41.088372 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082935 2566 flags.go:64] FLAG: --rotate-server-certificates="false" May 28 16:07:41.088372 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082938 2566 flags.go:64] FLAG: --runonce="false" May 28 16:07:41.088372 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082941 2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" May 28 16:07:41.088372 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082944 2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s" May 28 16:07:41.088372 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082947 2566 flags.go:64] FLAG: --seccomp-default="false" May 28 16:07:41.088372 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082950 2566 flags.go:64] FLAG: --serialize-image-pulls="true" May 28 16:07:41.088971 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082953 2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" May 28 16:07:41.088971 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082956 2566 flags.go:64] FLAG: --storage-driver-db="cadvisor" May 28 16:07:41.088971 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082959 2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086" May 28 16:07:41.088971 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082962 2566 flags.go:64] FLAG: --storage-driver-password="root" May 28 16:07:41.088971 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082966 2566 flags.go:64] FLAG: --storage-driver-secure="false" May 28 16:07:41.088971 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082970 2566 flags.go:64] FLAG: --storage-driver-table="stats" May 28 16:07:41.088971 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082973 2566 flags.go:64] FLAG: --storage-driver-user="root" May 28 16:07:41.088971 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082976 2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" May 28 16:07:41.088971 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082979 2566 flags.go:64] FLAG: --sync-frequency="1m0s" May 28 16:07:41.088971 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082982 2566 flags.go:64] FLAG: --system-cgroups="" May 28 16:07:41.088971 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082988 2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" May 28 16:07:41.088971 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.082997 2566 flags.go:64] FLAG: --system-reserved-cgroup="" May 28 16:07:41.088971 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.083001 2566 flags.go:64] FLAG: --tls-cert-file="" May 28 16:07:41.088971 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.083004 2566 flags.go:64] FLAG: --tls-cipher-suites="[]" May 28 16:07:41.088971 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.083009 2566 flags.go:64] FLAG: --tls-min-version="" May 28 16:07:41.088971 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.083012 2566 flags.go:64] FLAG: --tls-private-key-file="" May 28 16:07:41.088971 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.083015 2566 flags.go:64] FLAG: --topology-manager-policy="none" May 28 16:07:41.088971 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.083018 2566 flags.go:64] FLAG: --topology-manager-policy-options="" May 28 16:07:41.088971 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.083022 2566 flags.go:64] FLAG: --topology-manager-scope="container" May 28 16:07:41.088971 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.083025 2566 flags.go:64] FLAG: --v="2" May 28 16:07:41.088971 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.083030 2566 flags.go:64] FLAG: --version="false" May 28 16:07:41.088971 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.083034 2566 flags.go:64] FLAG: --vmodule="" May 28 16:07:41.088971 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.083039 2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" May 28 16:07:41.088971 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.083042 2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" May 28 16:07:41.088971 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083159 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 28 16:07:41.089588 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083164 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 28 16:07:41.089588 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083167 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 28 16:07:41.089588 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083170 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 28 16:07:41.089588 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083173 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 28 16:07:41.089588 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083176 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 28 16:07:41.089588 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083178 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 28 16:07:41.089588 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083181 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 28 16:07:41.089588 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083184 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 28 16:07:41.089588 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083186 2566 feature_gate.go:328] unrecognized feature gate: Example2 May 28 16:07:41.089588 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083189 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 28 16:07:41.089588 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083192 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 28 16:07:41.089588 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083196 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 28 16:07:41.089588 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083200 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 28 16:07:41.089588 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083202 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 28 16:07:41.089588 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083205 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 28 16:07:41.089588 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083207 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 28 16:07:41.089588 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083210 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 28 16:07:41.089588 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083212 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 28 16:07:41.089588 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083215 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 28 16:07:41.089588 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083217 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 28 16:07:41.090146 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083220 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 28 16:07:41.090146 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083225 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 28 16:07:41.090146 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083230 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 28 16:07:41.090146 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083235 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 28 16:07:41.090146 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083240 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 28 16:07:41.090146 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083243 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 28 16:07:41.090146 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083246 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 28 16:07:41.090146 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083248 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 28 16:07:41.090146 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083251 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 28 16:07:41.090146 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083254 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 28 16:07:41.090146 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083256 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 28 16:07:41.090146 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083259 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 28 16:07:41.090146 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083261 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 28 16:07:41.090146 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083264 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 28 16:07:41.090146 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083268 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 28 16:07:41.090146 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083272 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 28 16:07:41.090146 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083274 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 28 16:07:41.090146 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083293 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 28 16:07:41.090701 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083296 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 28 16:07:41.090701 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083299 2566 feature_gate.go:328] unrecognized feature gate: DualReplica May 28 16:07:41.090701 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083302 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 28 16:07:41.090701 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083306 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 28 16:07:41.090701 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083310 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 28 16:07:41.090701 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083316 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 28 16:07:41.090701 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083320 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 28 16:07:41.090701 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083325 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 28 16:07:41.090701 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083327 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 28 16:07:41.090701 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083330 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 28 16:07:41.090701 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083333 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 28 16:07:41.090701 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083335 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 28 16:07:41.090701 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083338 2566 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 28 16:07:41.090701 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083340 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 28 16:07:41.090701 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083342 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages May 28 16:07:41.090701 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083345 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 28 16:07:41.090701 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083347 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 28 16:07:41.090701 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083350 2566 feature_gate.go:328] unrecognized feature gate: NewOLM May 28 16:07:41.090701 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083352 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 28 16:07:41.090701 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083357 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 28 16:07:41.091197 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083360 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 28 16:07:41.091197 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083362 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 28 16:07:41.091197 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083365 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability May 28 16:07:41.091197 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083367 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 28 16:07:41.091197 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083370 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 28 16:07:41.091197 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083372 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 28 16:07:41.091197 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083375 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 28 16:07:41.091197 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083378 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 28 16:07:41.091197 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083381 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 28 16:07:41.091197 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083384 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 28 16:07:41.091197 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083389 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores May 28 16:07:41.091197 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083393 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 28 16:07:41.091197 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083398 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 28 16:07:41.091197 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083401 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 28 16:07:41.091197 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083404 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 28 16:07:41.091197 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083407 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 28 16:07:41.091197 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083409 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 28 16:07:41.091197 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083413 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 28 16:07:41.091197 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083416 2566 feature_gate.go:328] unrecognized feature gate: Example May 28 16:07:41.091197 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083418 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 28 16:07:41.091704 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083422 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 28 16:07:41.091704 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083424 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 28 16:07:41.091704 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083427 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 28 16:07:41.091704 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083430 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 28 16:07:41.091704 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083432 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 28 16:07:41.091704 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083434 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 28 16:07:41.091704 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.083437 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 28 16:07:41.091704 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.084155 2566 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 28 16:07:41.091704 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.090965 2566 server.go:530] "Kubelet version" kubeletVersion="v1.33.11" May 28 16:07:41.091704 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.090983 2566 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" May 28 16:07:41.091704 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091035 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 28 16:07:41.091704 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091041 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 28 16:07:41.091704 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091045 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 28 16:07:41.091704 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091048 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores May 28 16:07:41.091704 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091052 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 28 16:07:41.092090 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091056 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 28 16:07:41.092090 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091059 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 28 16:07:41.092090 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091062 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 28 16:07:41.092090 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091065 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 28 16:07:41.092090 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091068 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 28 16:07:41.092090 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091070 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 28 16:07:41.092090 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091073 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 28 16:07:41.092090 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091075 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 28 16:07:41.092090 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091078 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 28 16:07:41.092090 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091080 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 28 16:07:41.092090 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091083 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 28 16:07:41.092090 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091085 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 28 16:07:41.092090 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091088 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 28 16:07:41.092090 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091091 2566 feature_gate.go:328] unrecognized feature gate: Example2 May 28 16:07:41.092090 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091093 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 28 16:07:41.092090 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091095 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 28 16:07:41.092090 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091098 2566 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 28 16:07:41.092090 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091100 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 28 16:07:41.092090 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091103 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 28 16:07:41.092558 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091105 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 28 16:07:41.092558 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091108 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 28 16:07:41.092558 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091110 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 28 16:07:41.092558 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091113 2566 feature_gate.go:328] unrecognized feature gate: Example May 28 16:07:41.092558 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091115 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 28 16:07:41.092558 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091118 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 28 16:07:41.092558 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091120 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 28 16:07:41.092558 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091124 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 28 16:07:41.092558 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091127 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 28 16:07:41.092558 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091130 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 28 16:07:41.092558 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091133 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 28 16:07:41.092558 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091135 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 28 16:07:41.092558 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091137 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 28 16:07:41.092558 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091140 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 28 16:07:41.092558 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091142 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 28 16:07:41.092558 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091145 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 28 16:07:41.092558 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091147 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 28 16:07:41.092558 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091150 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 28 16:07:41.092558 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091152 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 28 16:07:41.092558 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091155 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 28 16:07:41.093042 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091157 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 28 16:07:41.093042 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091159 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 28 16:07:41.093042 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091162 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 28 16:07:41.093042 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091164 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 28 16:07:41.093042 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091167 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 28 16:07:41.093042 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091169 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages May 28 16:07:41.093042 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091172 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 28 16:07:41.093042 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091175 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 28 16:07:41.093042 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091177 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 28 16:07:41.093042 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091179 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 28 16:07:41.093042 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091182 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 28 16:07:41.093042 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091185 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 28 16:07:41.093042 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091187 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 28 16:07:41.093042 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091190 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 28 16:07:41.093042 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091192 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 28 16:07:41.093042 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091196 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 28 16:07:41.093042 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091201 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 28 16:07:41.093042 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091204 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability May 28 16:07:41.093042 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091207 2566 feature_gate.go:328] unrecognized feature gate: NewOLM May 28 16:07:41.093516 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091211 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 28 16:07:41.093516 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091215 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 28 16:07:41.093516 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091218 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 28 16:07:41.093516 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091221 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 28 16:07:41.093516 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091224 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 28 16:07:41.093516 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091227 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 28 16:07:41.093516 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091229 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 28 16:07:41.093516 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091232 2566 feature_gate.go:328] unrecognized feature gate: DualReplica May 28 16:07:41.093516 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091235 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 28 16:07:41.093516 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091237 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 28 16:07:41.093516 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091240 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 28 16:07:41.093516 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091242 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 28 16:07:41.093516 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091245 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 28 16:07:41.093516 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091247 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 28 16:07:41.093516 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091250 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 28 16:07:41.093516 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091252 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 28 16:07:41.093516 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091254 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 28 16:07:41.093516 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091257 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 28 16:07:41.093516 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091259 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 28 16:07:41.093516 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091262 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 28 16:07:41.094006 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091264 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 28 16:07:41.094006 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091267 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 28 16:07:41.094006 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091269 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 28 16:07:41.094006 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.091274 2566 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 28 16:07:41.094006 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091416 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 28 16:07:41.094006 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091421 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 28 16:07:41.094006 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091424 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 28 16:07:41.094006 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091427 2566 feature_gate.go:328] unrecognized feature gate: Example May 28 16:07:41.094006 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091430 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 28 16:07:41.094006 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091433 2566 feature_gate.go:328] unrecognized feature gate: DualReplica May 28 16:07:41.094006 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091436 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 28 16:07:41.094006 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091438 2566 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 28 16:07:41.094006 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091442 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 28 16:07:41.094006 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091446 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 28 16:07:41.094006 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091451 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 28 16:07:41.094399 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091454 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 28 16:07:41.094399 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091457 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 28 16:07:41.094399 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091459 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 28 16:07:41.094399 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091462 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 28 16:07:41.094399 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091464 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability May 28 16:07:41.094399 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091468 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 28 16:07:41.094399 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091470 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 28 16:07:41.094399 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091472 2566 feature_gate.go:328] unrecognized feature gate: NewOLM May 28 16:07:41.094399 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091475 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 28 16:07:41.094399 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091478 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 28 16:07:41.094399 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091481 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 28 16:07:41.094399 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091483 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 28 16:07:41.094399 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091486 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 28 16:07:41.094399 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091488 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 28 16:07:41.094399 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091491 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores May 28 16:07:41.094399 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091493 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 28 16:07:41.094399 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091496 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 28 16:07:41.094399 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091498 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 28 16:07:41.094399 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091501 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 28 16:07:41.094399 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091503 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 28 16:07:41.094875 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091505 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 28 16:07:41.094875 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091508 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 28 16:07:41.094875 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091510 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 28 16:07:41.094875 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091512 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 28 16:07:41.094875 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091515 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 28 16:07:41.094875 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091517 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 28 16:07:41.094875 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091520 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 28 16:07:41.094875 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091522 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 28 16:07:41.094875 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091524 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 28 16:07:41.094875 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091527 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 28 16:07:41.094875 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091530 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 28 16:07:41.094875 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091533 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 28 16:07:41.094875 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091536 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 28 16:07:41.094875 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091538 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 28 16:07:41.094875 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091541 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 28 16:07:41.094875 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091543 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 28 16:07:41.094875 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091546 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 28 16:07:41.094875 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091548 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 28 16:07:41.094875 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091550 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 28 16:07:41.095400 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091553 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 28 16:07:41.095400 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091555 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 28 16:07:41.095400 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091557 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 28 16:07:41.095400 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091560 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 28 16:07:41.095400 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091562 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 28 16:07:41.095400 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091565 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 28 16:07:41.095400 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091567 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 28 16:07:41.095400 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091569 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 28 16:07:41.095400 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091572 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 28 16:07:41.095400 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091574 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 28 16:07:41.095400 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091576 2566 feature_gate.go:328] unrecognized feature gate: Example2 May 28 16:07:41.095400 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091579 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 28 16:07:41.095400 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091581 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 28 16:07:41.095400 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091584 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 28 16:07:41.095400 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091586 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 28 16:07:41.095400 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091588 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 28 16:07:41.095400 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091590 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 28 16:07:41.095400 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091594 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 28 16:07:41.095400 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091598 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 28 16:07:41.095400 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091601 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 28 16:07:41.095882 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091604 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 28 16:07:41.095882 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091606 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 28 16:07:41.095882 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091609 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 28 16:07:41.095882 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091612 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 28 16:07:41.095882 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091614 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 28 16:07:41.095882 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091617 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 28 16:07:41.095882 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091619 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 28 16:07:41.095882 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091622 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 28 16:07:41.095882 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091624 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages May 28 16:07:41.095882 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091626 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 28 16:07:41.095882 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091629 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 28 16:07:41.095882 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091632 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 28 16:07:41.095882 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091634 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 28 16:07:41.095882 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091637 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 28 16:07:41.095882 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091639 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 28 16:07:41.095882 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:41.091642 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 28 16:07:41.096322 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.091646 2566 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 28 16:07:41.096322 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.092406 2566 server.go:962] "Client rotation is on, will bootstrap in background" May 28 16:07:41.096322 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.095822 2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" May 28 16:07:41.096699 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.096687 2566 server.go:1019] "Starting client certificate rotation" May 28 16:07:41.096815 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.096791 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" May 28 16:07:41.096871 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.096850 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" May 28 16:07:41.119998 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.119961 2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" May 28 16:07:41.122016 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.121987 2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" May 28 16:07:41.138157 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.138130 2566 log.go:25] "Validated CRI v1 runtime API" May 28 16:07:41.143137 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.143118 2566 log.go:25] "Validated CRI v1 image API" May 28 16:07:41.144407 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.144392 2566 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" May 28 16:07:41.148387 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.148367 2566 fs.go:135] Filesystem UUIDs: map[399a5c2d-9a4d-471d-81f8-0555aef81597:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 c89569fc-ae7b-4286-a0ff-0ceed5302f21:/dev/nvme0n1p4] May 28 16:07:41.148435 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.148388 2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] May 28 16:07:41.152940 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.152827 2566 manager.go:217] Machine: {Timestamp:2026-05-28 16:07:41.152276351 +0000 UTC m=+0.379306898 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3093804 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec25469c22b2008b4d6bbb20ddc18235 SystemUUID:ec25469c-22b2-008b-4d6b-bb20ddc18235 BootID:b0348a72-19c7-44bf-bd50-f17cd9309085 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f9:2c:ae:da:ad Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f9:2c:ae:da:ad Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f6:8c:d7:1c:af:49 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} May 28 16:07:41.153619 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.153607 2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. May 28 16:07:41.153708 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.153696 2566 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} May 28 16:07:41.155441 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.155414 2566 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] May 28 16:07:41.155621 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.155606 2566 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" May 28 16:07:41.155655 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.155455 2566 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-102.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} May 28 16:07:41.155655 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.155648 2566 topology_manager.go:138] "Creating topology manager with none policy" May 28 16:07:41.155722 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.155657 2566 container_manager_linux.go:306] "Creating device plugin manager" May 28 16:07:41.155722 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.155665 2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" May 28 16:07:41.156425 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.156415 2566 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" May 28 16:07:41.158373 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.158360 2566 state_mem.go:36] "Initialized new in-memory state store" May 28 16:07:41.158501 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.158492 2566 server.go:1267] "Using root directory" path="/var/lib/kubelet" May 28 16:07:41.161571 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.161560 2566 kubelet.go:491] "Attempting to sync node with API server" May 28 16:07:41.161606 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.161579 2566 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" May 28 16:07:41.161606 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.161593 2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests" May 28 16:07:41.161606 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.161603 2566 kubelet.go:397] "Adding apiserver pod source" May 28 16:07:41.161758 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.161612 2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods" May 28 16:07:41.162719 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.162705 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" May 28 16:07:41.162756 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.162726 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" May 28 16:07:41.165477 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.165458 2566 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1" May 28 16:07:41.167453 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.167435 2566 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" May 28 16:07:41.168336 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.168322 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" May 28 16:07:41.168428 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.168342 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" May 28 16:07:41.168428 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.168351 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" May 28 16:07:41.168428 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.168360 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" May 28 16:07:41.168428 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.168372 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" May 28 16:07:41.168428 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.168381 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" May 28 16:07:41.168428 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.168398 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" May 28 16:07:41.168428 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.168406 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" May 28 16:07:41.168428 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.168416 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" May 28 16:07:41.168428 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.168424 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" May 28 16:07:41.168694 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.168441 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" May 28 16:07:41.168694 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.168636 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" May 28 16:07:41.170371 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.170357 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" May 28 16:07:41.170449 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.170376 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" May 28 16:07:41.172528 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:41.172491 2566 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" May 28 16:07:41.172712 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:41.172691 2566 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-102.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" May 28 16:07:41.174644 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.174627 2566 watchdog_linux.go:99] "Systemd watchdog is not enabled" May 28 16:07:41.174721 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.174681 2566 server.go:1295] "Started kubelet" May 28 16:07:41.174770 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.174738 2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 May 28 16:07:41.174873 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.174830 2566 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 May 28 16:07:41.174922 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.174897 2566 server_v1.go:47] "podresources" method="list" useActivePods=true May 28 16:07:41.175609 ip-10-0-143-102 systemd[1]: Started Kubernetes Kubelet. May 28 16:07:41.176560 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.176539 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5kt6l" May 28 16:07:41.176674 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.176658 2566 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" May 28 16:07:41.177537 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.177503 2566 server.go:317] "Adding debug handlers to kubelet server" May 28 16:07:41.181736 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.181719 2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" May 28 16:07:41.181817 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.181778 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" May 28 16:07:41.182325 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.181898 2566 volume_manager.go:295] "The desired_state_of_world populator starts" May 28 16:07:41.182325 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.181912 2566 volume_manager.go:297] "Starting Kubelet Volume Manager" May 28 16:07:41.182325 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.182123 2566 desired_state_of_world_populator.go:150] "Desired state populator starts to run" May 28 16:07:41.182325 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.182190 2566 reconstruct.go:97] "Volume reconstruction finished" May 28 16:07:41.182325 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.182199 2566 reconciler.go:26] "Reconciler: start to sync state" May 28 16:07:41.182593 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.182459 2566 factory.go:55] Registering systemd factory May 28 16:07:41.182593 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:41.182463 2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-102.ec2.internal\" not found" May 28 16:07:41.182593 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.182513 2566 factory.go:223] Registration of the systemd container factory successfully May 28 16:07:41.182593 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:41.182535 2566 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" May 28 16:07:41.182818 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.182633 2566 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-102.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope May 28 16:07:41.182818 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.182719 2566 factory.go:153] Registering CRI-O factory May 28 16:07:41.182818 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.182731 2566 factory.go:223] Registration of the crio container factory successfully May 28 16:07:41.182818 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.182780 2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory May 28 16:07:41.182818 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.182801 2566 factory.go:103] Registering Raw factory May 28 16:07:41.182818 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.182816 2566 manager.go:1196] Started watching for new ooms in manager May 28 16:07:41.183227 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.183164 2566 manager.go:319] Starting recovery of all containers May 28 16:07:41.184141 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:41.182672 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-102.ec2.internal.18b3c66fca0798f7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-102.ec2.internal,UID:ip-10-0-143-102.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-102.ec2.internal,},FirstTimestamp:2026-05-28 16:07:41.174642935 +0000 UTC m=+0.401673482,LastTimestamp:2026-05-28 16:07:41.174642935 +0000 UTC m=+0.401673482,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-102.ec2.internal,}" May 28 16:07:41.185477 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.185450 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5kt6l" May 28 16:07:41.186876 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:41.186837 2566 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" May 28 16:07:41.186990 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:41.186962 2566 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-102.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" May 28 16:07:41.193059 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.193042 2566 manager.go:324] Recovery completed May 28 16:07:41.193988 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.193925 2566 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" May 28 16:07:41.194823 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:41.194779 2566 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory May 28 16:07:41.199395 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.199380 2566 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 16:07:41.201272 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.201257 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasSufficientMemory" May 28 16:07:41.201339 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.201304 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasNoDiskPressure" May 28 16:07:41.201339 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.201315 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasSufficientPID" May 28 16:07:41.201889 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.201875 2566 cpu_manager.go:222] "Starting CPU manager" policy="none" May 28 16:07:41.201889 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.201887 2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" May 28 16:07:41.201975 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.201905 2566 state_mem.go:36] "Initialized new in-memory state store" May 28 16:07:41.204351 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.204338 2566 policy_none.go:49] "None policy: Start" May 28 16:07:41.204387 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.204356 2566 memory_manager.go:186] "Starting memorymanager" policy="None" May 28 16:07:41.204387 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.204366 2566 state_mem.go:35] "Initializing new in-memory state store" May 28 16:07:41.245916 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.245901 2566 manager.go:341] "Starting Device Plugin manager" May 28 16:07:41.261894 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:41.245939 2566 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" May 28 16:07:41.261894 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.245949 2566 server.go:85] "Starting device plugin registration server" May 28 16:07:41.261894 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.246236 2566 eviction_manager.go:189] "Eviction manager: starting control loop" May 28 16:07:41.261894 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.246252 2566 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" May 28 16:07:41.261894 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.246357 2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" May 28 16:07:41.261894 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.246436 2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" May 28 16:07:41.261894 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.246447 2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager" May 28 16:07:41.261894 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:41.247245 2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" May 28 16:07:41.261894 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:41.247354 2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-102.ec2.internal\" not found" May 28 16:07:41.332133 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.332050 2566 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" May 28 16:07:41.332133 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.332077 2566 status_manager.go:230] "Starting to sync pod status with apiserver" May 28 16:07:41.332133 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.332097 2566 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." May 28 16:07:41.332133 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.332104 2566 kubelet.go:2451] "Starting kubelet main sync loop" May 28 16:07:41.332410 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:41.332147 2566 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" May 28 16:07:41.338493 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.338468 2566 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" May 28 16:07:41.347396 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.347379 2566 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 16:07:41.348521 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.348504 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasSufficientMemory" May 28 16:07:41.348630 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.348535 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasNoDiskPressure" May 28 16:07:41.348630 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.348551 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasSufficientPID" May 28 16:07:41.348630 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.348579 2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-102.ec2.internal" May 28 16:07:41.354735 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.354719 2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-102.ec2.internal" May 28 16:07:41.354784 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:41.354743 2566 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-102.ec2.internal\": node \"ip-10-0-143-102.ec2.internal\" not found" May 28 16:07:41.428217 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:41.428180 2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-102.ec2.internal\" not found" May 28 16:07:41.433225 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.433192 2566 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-102.ec2.internal"] May 28 16:07:41.433326 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.433296 2566 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 16:07:41.435013 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.434996 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasSufficientMemory" May 28 16:07:41.435101 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.435025 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasNoDiskPressure" May 28 16:07:41.435101 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.435034 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasSufficientPID" May 28 16:07:41.436311 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.436298 2566 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 16:07:41.436515 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.436475 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal" May 28 16:07:41.436515 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.436505 2566 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 16:07:41.436993 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.436977 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasSufficientMemory" May 28 16:07:41.437070 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.437009 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasSufficientMemory" May 28 16:07:41.437070 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.437029 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasNoDiskPressure" May 28 16:07:41.437070 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.437039 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasSufficientPID" May 28 16:07:41.437179 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.437011 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasNoDiskPressure" May 28 16:07:41.437179 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.437092 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasSufficientPID" May 28 16:07:41.438324 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.438306 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-102.ec2.internal" May 28 16:07:41.438423 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.438335 2566 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 16:07:41.438986 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.438971 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasSufficientMemory" May 28 16:07:41.439056 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.438994 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasNoDiskPressure" May 28 16:07:41.439056 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.439007 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeHasSufficientPID" May 28 16:07:41.466997 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:41.466974 2566 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-102.ec2.internal\" not found" node="ip-10-0-143-102.ec2.internal" May 28 16:07:41.471183 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:41.471166 2566 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-102.ec2.internal\" not found" node="ip-10-0-143-102.ec2.internal" May 28 16:07:41.483632 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.483614 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/90fc7a1de7d02404a7ce08e7f99d20e6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal\" (UID: \"90fc7a1de7d02404a7ce08e7f99d20e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal" May 28 16:07:41.483706 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.483640 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90fc7a1de7d02404a7ce08e7f99d20e6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal\" (UID: \"90fc7a1de7d02404a7ce08e7f99d20e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal" May 28 16:07:41.483706 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.483659 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3682fbf2c94d063cca39e2c160569366-config\") pod \"kube-apiserver-proxy-ip-10-0-143-102.ec2.internal\" (UID: \"3682fbf2c94d063cca39e2c160569366\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-102.ec2.internal" May 28 16:07:41.529165 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:41.529132 2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-102.ec2.internal\" not found" May 28 16:07:41.583993 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.583912 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3682fbf2c94d063cca39e2c160569366-config\") pod \"kube-apiserver-proxy-ip-10-0-143-102.ec2.internal\" (UID: \"3682fbf2c94d063cca39e2c160569366\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-102.ec2.internal" May 28 16:07:41.583993 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.583947 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/90fc7a1de7d02404a7ce08e7f99d20e6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal\" (UID: \"90fc7a1de7d02404a7ce08e7f99d20e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal" May 28 16:07:41.583993 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.583967 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90fc7a1de7d02404a7ce08e7f99d20e6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal\" (UID: \"90fc7a1de7d02404a7ce08e7f99d20e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal" May 28 16:07:41.584208 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.584038 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3682fbf2c94d063cca39e2c160569366-config\") pod \"kube-apiserver-proxy-ip-10-0-143-102.ec2.internal\" (UID: \"3682fbf2c94d063cca39e2c160569366\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-102.ec2.internal" May 28 16:07:41.584208 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.584077 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/90fc7a1de7d02404a7ce08e7f99d20e6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal\" (UID: \"90fc7a1de7d02404a7ce08e7f99d20e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal" May 28 16:07:41.584208 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.584109 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90fc7a1de7d02404a7ce08e7f99d20e6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal\" (UID: \"90fc7a1de7d02404a7ce08e7f99d20e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal" May 28 16:07:41.630200 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:41.630168 2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-102.ec2.internal\" not found" May 28 16:07:41.730855 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:41.730811 2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-102.ec2.internal\" not found" May 28 16:07:41.770012 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.769976 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal" May 28 16:07:41.774854 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:41.774832 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-102.ec2.internal" May 28 16:07:41.831136 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:41.831090 2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-102.ec2.internal\" not found" May 28 16:07:41.931647 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:41.931564 2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-102.ec2.internal\" not found" May 28 16:07:42.032050 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:42.032015 2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-102.ec2.internal\" not found" May 28 16:07:42.097515 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:42.097481 2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" May 28 16:07:42.098259 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:42.097622 2566 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" May 28 16:07:42.132850 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:42.132816 2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-102.ec2.internal\" not found" May 28 16:07:42.182342 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:42.182263 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" May 28 16:07:42.187964 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:42.187922 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-27 16:02:41 +0000 UTC" deadline="2027-11-24 15:59:31.38699609 +0000 UTC" May 28 16:07:42.187964 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:42.187963 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13079h51m49.199037309s" May 28 16:07:42.214209 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:42.214182 2566 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" May 28 16:07:42.215085 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:42.215067 2566 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" May 28 16:07:42.219508 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:42.219488 2566 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" May 28 16:07:42.225553 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:42.225523 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90fc7a1de7d02404a7ce08e7f99d20e6.slice/crio-c768d9031fee789bace3629901c2cb076f24009cb974d9d58a259d261d9facaa WatchSource:0}: Error finding container c768d9031fee789bace3629901c2cb076f24009cb974d9d58a259d261d9facaa: Status 404 returned error can't find the container with id c768d9031fee789bace3629901c2cb076f24009cb974d9d58a259d261d9facaa May 28 16:07:42.225775 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:42.225755 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3682fbf2c94d063cca39e2c160569366.slice/crio-1c2632f37df09ee7ee603d7916478847e0e94c410d5b46ccf77c8ffed8a23a15 WatchSource:0}: Error finding container 1c2632f37df09ee7ee603d7916478847e0e94c410d5b46ccf77c8ffed8a23a15: Status 404 returned error can't find the container with id 1c2632f37df09ee7ee603d7916478847e0e94c410d5b46ccf77c8ffed8a23a15 May 28 16:07:42.229101 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:42.229080 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 28 16:07:42.257945 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:42.257917 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mv9vq" May 28 16:07:42.270453 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:42.270432 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mv9vq" May 28 16:07:42.282448 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:42.282423 2566 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal" May 28 16:07:42.301426 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:42.301403 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" May 28 16:07:42.302190 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:42.302177 2566 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-102.ec2.internal" May 28 16:07:42.317660 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:42.317641 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" May 28 16:07:42.335758 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:42.335717 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal" event={"ID":"90fc7a1de7d02404a7ce08e7f99d20e6","Type":"ContainerStarted","Data":"c768d9031fee789bace3629901c2cb076f24009cb974d9d58a259d261d9facaa"} May 28 16:07:42.336725 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:42.336705 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-102.ec2.internal" event={"ID":"3682fbf2c94d063cca39e2c160569366","Type":"ContainerStarted","Data":"1c2632f37df09ee7ee603d7916478847e0e94c410d5b46ccf77c8ffed8a23a15"} May 28 16:07:42.603690 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:42.603662 2566 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" May 28 16:07:43.162528 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.162497 2566 apiserver.go:52] "Watching apiserver" May 28 16:07:43.176627 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.176596 2566 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" May 28 16:07:43.179072 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.179041 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-b56lh","openshift-multus/network-metrics-daemon-6dt52","openshift-ovn-kubernetes/ovnkube-node-db77q","kube-system/konnectivity-agent-jq2b6","kube-system/kube-apiserver-proxy-ip-10-0-143-102.ec2.internal","openshift-cluster-node-tuning-operator/tuned-vxt4n","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal","openshift-network-diagnostics/network-check-target-2nzbn","openshift-network-operator/iptables-alerter-vf6tv","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4swv9","openshift-dns/node-resolver-4k22m","openshift-image-registry/node-ca-z55zs","openshift-multus/multus-additional-cni-plugins-j8ptw"] May 28 16:07:43.181863 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.181834 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4swv9" May 28 16:07:43.183917 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.183834 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dt52" May 28 16:07:43.184034 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:43.183923 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dt52" podUID="3b54fd06-9913-49b9-ac98-d2916e5f0276" May 28 16:07:43.184576 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.184402 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" May 28 16:07:43.184576 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.184409 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-cwgrq\"" May 28 16:07:43.184576 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.184439 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" May 28 16:07:43.184774 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.184589 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" May 28 16:07:43.187622 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.187595 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.189733 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.189709 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jq2b6" May 28 16:07:43.190245 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.190226 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dk4bg\"" May 28 16:07:43.191835 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.191812 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" May 28 16:07:43.191835 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.191824 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" May 28 16:07:43.191835 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.191852 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" May 28 16:07:43.192172 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.191948 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" May 28 16:07:43.192172 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.192050 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" May 28 16:07:43.192172 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.192060 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" May 28 16:07:43.192172 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.192111 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b54fd06-9913-49b9-ac98-d2916e5f0276-metrics-certs\") pod \"network-metrics-daemon-6dt52\" (UID: \"3b54fd06-9913-49b9-ac98-d2916e5f0276\") " pod="openshift-multus/network-metrics-daemon-6dt52" May 28 16:07:43.192172 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.192148 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxmxj\" (UniqueName: \"kubernetes.io/projected/3b54fd06-9913-49b9-ac98-d2916e5f0276-kube-api-access-mxmxj\") pod \"network-metrics-daemon-6dt52\" (UID: \"3b54fd06-9913-49b9-ac98-d2916e5f0276\") " pod="openshift-multus/network-metrics-daemon-6dt52" May 28 16:07:43.192172 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.192162 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hsqpg\"" May 28 16:07:43.192450 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.192177 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7acdfd90-ea35-421a-9249-f372fb4ea69c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4swv9\" (UID: \"7acdfd90-ea35-421a-9249-f372fb4ea69c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4swv9" May 28 16:07:43.192450 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.192203 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7acdfd90-ea35-421a-9249-f372fb4ea69c-socket-dir\") pod \"aws-ebs-csi-driver-node-4swv9\" (UID: \"7acdfd90-ea35-421a-9249-f372fb4ea69c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4swv9" May 28 16:07:43.192450 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.192207 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" May 28 16:07:43.192450 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.192231 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7acdfd90-ea35-421a-9249-f372fb4ea69c-sys-fs\") pod \"aws-ebs-csi-driver-node-4swv9\" (UID: \"7acdfd90-ea35-421a-9249-f372fb4ea69c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4swv9" May 28 16:07:43.192450 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.192258 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vk8m6\" (UniqueName: \"kubernetes.io/projected/7acdfd90-ea35-421a-9249-f372fb4ea69c-kube-api-access-vk8m6\") pod \"aws-ebs-csi-driver-node-4swv9\" (UID: \"7acdfd90-ea35-421a-9249-f372fb4ea69c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4swv9" May 28 16:07:43.192450 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.192306 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7acdfd90-ea35-421a-9249-f372fb4ea69c-registration-dir\") pod \"aws-ebs-csi-driver-node-4swv9\" (UID: \"7acdfd90-ea35-421a-9249-f372fb4ea69c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4swv9" May 28 16:07:43.192450 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.192334 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7acdfd90-ea35-421a-9249-f372fb4ea69c-device-dir\") pod \"aws-ebs-csi-driver-node-4swv9\" (UID: \"7acdfd90-ea35-421a-9249-f372fb4ea69c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4swv9" May 28 16:07:43.192450 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.192357 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7acdfd90-ea35-421a-9249-f372fb4ea69c-etc-selinux\") pod \"aws-ebs-csi-driver-node-4swv9\" (UID: \"7acdfd90-ea35-421a-9249-f372fb4ea69c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4swv9" May 28 16:07:43.192931 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.192914 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" May 28 16:07:43.194432 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.194413 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.194547 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.194532 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nzbn" May 28 16:07:43.194637 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:43.194608 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2nzbn" podUID="f15055af-8047-4a04-b79f-64af8836b946" May 28 16:07:43.196562 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.196542 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" May 28 16:07:43.196802 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.196783 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vf6tv" May 28 16:07:43.197201 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.197180 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" May 28 16:07:43.197332 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.197180 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-m6dcz\"" May 28 16:07:43.198970 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.198946 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-b56lh" May 28 16:07:43.199658 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.199638 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" May 28 16:07:43.199879 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.199860 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qr5xr\"" May 28 16:07:43.199959 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.199878 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" May 28 16:07:43.200144 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.200090 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" May 28 16:07:43.201387 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.201362 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4k22m" May 28 16:07:43.202223 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.201838 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-kfl7p\"" May 28 16:07:43.202223 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.201982 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" May 28 16:07:43.202223 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.202006 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" May 28 16:07:43.202223 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.202006 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" May 28 16:07:43.202223 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.202169 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" May 28 16:07:43.203874 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.203849 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" May 28 16:07:43.203874 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.203869 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" May 28 16:07:43.204011 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.203903 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9tnsc\"" May 28 16:07:43.204824 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.204792 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-z55zs" May 28 16:07:43.209801 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.208558 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" May 28 16:07:43.209801 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.208751 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" May 28 16:07:43.209801 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.208930 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" May 28 16:07:43.210203 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.210182 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j8ptw" May 28 16:07:43.210203 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.210200 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-wz8ms\"" May 28 16:07:43.212455 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.212434 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" May 28 16:07:43.212559 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.212519 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zjpsj\"" May 28 16:07:43.212613 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.212567 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" May 28 16:07:43.271070 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.271037 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 16:02:42 +0000 UTC" deadline="2028-03-11 15:46:48.44761635 +0000 UTC" May 28 16:07:43.271070 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.271067 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15671h39m5.176551785s" May 28 16:07:43.283826 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.283802 2566 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" May 28 16:07:43.292702 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.292663 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-host-run-k8s-cni-cncf-io\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.292867 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.292714 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd5gg\" (UniqueName: \"kubernetes.io/projected/a40ad2f3-e916-4272-9a82-9b0a818900e5-kube-api-access-zd5gg\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.292867 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.292746 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7acdfd90-ea35-421a-9249-f372fb4ea69c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4swv9\" (UID: \"7acdfd90-ea35-421a-9249-f372fb4ea69c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4swv9" May 28 16:07:43.292867 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.292773 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-systemd-units\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.292867 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.292798 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a40ad2f3-e916-4272-9a82-9b0a818900e5-cni-binary-copy\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.292867 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.292820 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dc91efd9-d166-4a72-bcdc-3247cc10fce8-hosts-file\") pod \"node-resolver-4k22m\" (UID: \"dc91efd9-d166-4a72-bcdc-3247cc10fce8\") " pod="openshift-dns/node-resolver-4k22m" May 28 16:07:43.292867 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.292844 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8d262a97-a722-48e6-a264-4fdba30f80e8-etc-sysctl-conf\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.292867 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.292845 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7acdfd90-ea35-421a-9249-f372fb4ea69c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4swv9\" (UID: \"7acdfd90-ea35-421a-9249-f372fb4ea69c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4swv9" May 28 16:07:43.292867 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.292865 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8d262a97-a722-48e6-a264-4fdba30f80e8-sys\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.293226 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.292891 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9cbg\" (UniqueName: \"kubernetes.io/projected/5ae4021e-a738-403c-9143-976b0c75fca8-kube-api-access-v9cbg\") pod \"node-ca-z55zs\" (UID: \"5ae4021e-a738-403c-9143-976b0c75fca8\") " pod="openshift-image-registry/node-ca-z55zs" May 28 16:07:43.293226 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.292914 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-host-kubelet\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.293226 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.292935 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-system-cni-dir\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.293226 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.292953 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8d262a97-a722-48e6-a264-4fdba30f80e8-etc-systemd\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.293226 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.292967 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4gx5\" (UniqueName: \"kubernetes.io/projected/8d262a97-a722-48e6-a264-4fdba30f80e8-kube-api-access-j4gx5\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.293226 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.292992 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mxmxj\" (UniqueName: \"kubernetes.io/projected/3b54fd06-9913-49b9-ac98-d2916e5f0276-kube-api-access-mxmxj\") pod \"network-metrics-daemon-6dt52\" (UID: \"3b54fd06-9913-49b9-ac98-d2916e5f0276\") " pod="openshift-multus/network-metrics-daemon-6dt52" May 28 16:07:43.293226 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.293034 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7acdfd90-ea35-421a-9249-f372fb4ea69c-sys-fs\") pod \"aws-ebs-csi-driver-node-4swv9\" (UID: \"7acdfd90-ea35-421a-9249-f372fb4ea69c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4swv9" May 28 16:07:43.293226 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.293116 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vk8m6\" (UniqueName: \"kubernetes.io/projected/7acdfd90-ea35-421a-9249-f372fb4ea69c-kube-api-access-vk8m6\") pod \"aws-ebs-csi-driver-node-4swv9\" (UID: \"7acdfd90-ea35-421a-9249-f372fb4ea69c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4swv9" May 28 16:07:43.293226 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.293152 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7acdfd90-ea35-421a-9249-f372fb4ea69c-sys-fs\") pod \"aws-ebs-csi-driver-node-4swv9\" (UID: \"7acdfd90-ea35-421a-9249-f372fb4ea69c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4swv9" May 28 16:07:43.293226 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.293191 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-run-openvswitch\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.293226 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.293226 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-node-log\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.293839 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.293257 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8d262a97-a722-48e6-a264-4fdba30f80e8-etc-sysctl-d\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.293839 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.293299 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8d262a97-a722-48e6-a264-4fdba30f80e8-etc-tuned\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.293839 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.293327 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7acdfd90-ea35-421a-9249-f372fb4ea69c-etc-selinux\") pod \"aws-ebs-csi-driver-node-4swv9\" (UID: \"7acdfd90-ea35-421a-9249-f372fb4ea69c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4swv9" May 28 16:07:43.293839 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.293353 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/714f97cf-9a9a-4321-a50d-511802063910-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j8ptw\" (UID: \"714f97cf-9a9a-4321-a50d-511802063910\") " pod="openshift-multus/multus-additional-cni-plugins-j8ptw" May 28 16:07:43.293839 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.293412 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-os-release\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.293839 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.293438 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-multus-socket-dir-parent\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.293839 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.293459 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7acdfd90-ea35-421a-9249-f372fb4ea69c-etc-selinux\") pod \"aws-ebs-csi-driver-node-4swv9\" (UID: \"7acdfd90-ea35-421a-9249-f372fb4ea69c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4swv9" May 28 16:07:43.293839 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.293463 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-host-run-netns\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.293839 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.293511 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-host-run-multus-certs\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.293839 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.293548 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8d262a97-a722-48e6-a264-4fdba30f80e8-etc-modprobe-d\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.293839 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.293582 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8d262a97-a722-48e6-a264-4fdba30f80e8-run\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.293839 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.293612 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-host-run-ovn-kubernetes\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.293839 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.293640 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztdpt\" (UniqueName: \"kubernetes.io/projected/f15055af-8047-4a04-b79f-64af8836b946-kube-api-access-ztdpt\") pod \"network-check-target-2nzbn\" (UID: \"f15055af-8047-4a04-b79f-64af8836b946\") " pod="openshift-network-diagnostics/network-check-target-2nzbn" May 28 16:07:43.293839 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.293678 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/eb400c72-a841-4a4b-be8b-6684abea58b8-iptables-alerter-script\") pod \"iptables-alerter-vf6tv\" (UID: \"eb400c72-a841-4a4b-be8b-6684abea58b8\") " pod="openshift-network-operator/iptables-alerter-vf6tv" May 28 16:07:43.293839 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.293765 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8d262a97-a722-48e6-a264-4fdba30f80e8-lib-modules\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.293839 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.293790 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d262a97-a722-48e6-a264-4fdba30f80e8-var-lib-kubelet\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.294571 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.293816 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/714f97cf-9a9a-4321-a50d-511802063910-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j8ptw\" (UID: \"714f97cf-9a9a-4321-a50d-511802063910\") " pod="openshift-multus/multus-additional-cni-plugins-j8ptw" May 28 16:07:43.294571 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.293839 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-host-run-netns\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.294571 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.293861 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-var-lib-openvswitch\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.294571 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.293908 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d07065a1-27ff-4f50-9b8e-88d35f69bc89-ovn-node-metrics-cert\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.294571 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.293939 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-etc-kubernetes\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.294571 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.293962 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8d262a97-a722-48e6-a264-4fdba30f80e8-etc-sysconfig\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.294571 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.294019 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/714f97cf-9a9a-4321-a50d-511802063910-system-cni-dir\") pod \"multus-additional-cni-plugins-j8ptw\" (UID: \"714f97cf-9a9a-4321-a50d-511802063910\") " pod="openshift-multus/multus-additional-cni-plugins-j8ptw" May 28 16:07:43.294571 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.294066 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-multus-conf-dir\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.294571 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.294091 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c0258ed5-55e5-4a30-a658-3e8d61330e0a-agent-certs\") pod \"konnectivity-agent-jq2b6\" (UID: \"c0258ed5-55e5-4a30-a658-3e8d61330e0a\") " pod="kube-system/konnectivity-agent-jq2b6" May 28 16:07:43.294571 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.294113 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d262a97-a722-48e6-a264-4fdba30f80e8-tmp\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.294571 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.294142 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7acdfd90-ea35-421a-9249-f372fb4ea69c-device-dir\") pod \"aws-ebs-csi-driver-node-4swv9\" (UID: \"7acdfd90-ea35-421a-9249-f372fb4ea69c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4swv9" May 28 16:07:43.294571 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.294202 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7acdfd90-ea35-421a-9249-f372fb4ea69c-device-dir\") pod \"aws-ebs-csi-driver-node-4swv9\" (UID: \"7acdfd90-ea35-421a-9249-f372fb4ea69c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4swv9" May 28 16:07:43.294571 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.294218 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/714f97cf-9a9a-4321-a50d-511802063910-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j8ptw\" (UID: \"714f97cf-9a9a-4321-a50d-511802063910\") " pod="openshift-multus/multus-additional-cni-plugins-j8ptw" May 28 16:07:43.294571 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.294257 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-log-socket\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.294571 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.294299 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-host-cni-netd\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.294571 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.294328 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzpz9\" (UniqueName: \"kubernetes.io/projected/d07065a1-27ff-4f50-9b8e-88d35f69bc89-kube-api-access-dzpz9\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.295273 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.294354 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eb400c72-a841-4a4b-be8b-6684abea58b8-host-slash\") pod \"iptables-alerter-vf6tv\" (UID: \"eb400c72-a841-4a4b-be8b-6684abea58b8\") " pod="openshift-network-operator/iptables-alerter-vf6tv" May 28 16:07:43.295273 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.294393 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7acdfd90-ea35-421a-9249-f372fb4ea69c-socket-dir\") pod \"aws-ebs-csi-driver-node-4swv9\" (UID: \"7acdfd90-ea35-421a-9249-f372fb4ea69c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4swv9" May 28 16:07:43.295273 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.294419 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/714f97cf-9a9a-4321-a50d-511802063910-os-release\") pod \"multus-additional-cni-plugins-j8ptw\" (UID: \"714f97cf-9a9a-4321-a50d-511802063910\") " pod="openshift-multus/multus-additional-cni-plugins-j8ptw" May 28 16:07:43.295273 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.294446 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-host-var-lib-cni-multus\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.295273 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.294478 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c0258ed5-55e5-4a30-a658-3e8d61330e0a-konnectivity-ca\") pod \"konnectivity-agent-jq2b6\" (UID: \"c0258ed5-55e5-4a30-a658-3e8d61330e0a\") " pod="kube-system/konnectivity-agent-jq2b6" May 28 16:07:43.295273 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.294510 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7acdfd90-ea35-421a-9249-f372fb4ea69c-registration-dir\") pod \"aws-ebs-csi-driver-node-4swv9\" (UID: \"7acdfd90-ea35-421a-9249-f372fb4ea69c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4swv9" May 28 16:07:43.295273 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.294530 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d07065a1-27ff-4f50-9b8e-88d35f69bc89-ovnkube-config\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.295273 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.294552 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-hostroot\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.295273 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.294578 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b54fd06-9913-49b9-ac98-d2916e5f0276-metrics-certs\") pod \"network-metrics-daemon-6dt52\" (UID: \"3b54fd06-9913-49b9-ac98-d2916e5f0276\") " pod="openshift-multus/network-metrics-daemon-6dt52" May 28 16:07:43.295273 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.294598 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-host-slash\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.295273 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.294623 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d07065a1-27ff-4f50-9b8e-88d35f69bc89-ovnkube-script-lib\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.295273 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.294632 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7acdfd90-ea35-421a-9249-f372fb4ea69c-socket-dir\") pod \"aws-ebs-csi-driver-node-4swv9\" (UID: \"7acdfd90-ea35-421a-9249-f372fb4ea69c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4swv9" May 28 16:07:43.295273 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.294647 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-multus-cni-dir\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.295273 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.294666 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-etc-openvswitch\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.295273 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:43.294757 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 16:07:43.295273 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.294805 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a40ad2f3-e916-4272-9a82-9b0a818900e5-multus-daemon-config\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.295273 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.294808 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7acdfd90-ea35-421a-9249-f372fb4ea69c-registration-dir\") pod \"aws-ebs-csi-driver-node-4swv9\" (UID: \"7acdfd90-ea35-421a-9249-f372fb4ea69c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4swv9" May 28 16:07:43.295932 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.294829 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmk64\" (UniqueName: \"kubernetes.io/projected/dc91efd9-d166-4a72-bcdc-3247cc10fce8-kube-api-access-zmk64\") pod \"node-resolver-4k22m\" (UID: \"dc91efd9-d166-4a72-bcdc-3247cc10fce8\") " pod="openshift-dns/node-resolver-4k22m" May 28 16:07:43.295932 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.294857 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d262a97-a722-48e6-a264-4fdba30f80e8-etc-kubernetes\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.295932 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:43.294893 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b54fd06-9913-49b9-ac98-d2916e5f0276-metrics-certs podName:3b54fd06-9913-49b9-ac98-d2916e5f0276 nodeName:}" failed. No retries permitted until 2026-05-28 16:07:43.794854127 +0000 UTC m=+3.021884702 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b54fd06-9913-49b9-ac98-d2916e5f0276-metrics-certs") pod "network-metrics-daemon-6dt52" (UID: "3b54fd06-9913-49b9-ac98-d2916e5f0276") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 16:07:43.295932 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.294948 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5ae4021e-a738-403c-9143-976b0c75fca8-serviceca\") pod \"node-ca-z55zs\" (UID: \"5ae4021e-a738-403c-9143-976b0c75fca8\") " pod="openshift-image-registry/node-ca-z55zs" May 28 16:07:43.295932 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.294979 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/714f97cf-9a9a-4321-a50d-511802063910-cni-binary-copy\") pod \"multus-additional-cni-plugins-j8ptw\" (UID: \"714f97cf-9a9a-4321-a50d-511802063910\") " pod="openshift-multus/multus-additional-cni-plugins-j8ptw" May 28 16:07:43.295932 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.295022 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-run-systemd\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.295932 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.295048 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-host-cni-bin\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.295932 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.295085 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qs82\" (UniqueName: \"kubernetes.io/projected/eb400c72-a841-4a4b-be8b-6684abea58b8-kube-api-access-4qs82\") pod \"iptables-alerter-vf6tv\" (UID: \"eb400c72-a841-4a4b-be8b-6684abea58b8\") " pod="openshift-network-operator/iptables-alerter-vf6tv" May 28 16:07:43.295932 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.295122 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-host-var-lib-kubelet\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.295932 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.295149 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc91efd9-d166-4a72-bcdc-3247cc10fce8-tmp-dir\") pod \"node-resolver-4k22m\" (UID: \"dc91efd9-d166-4a72-bcdc-3247cc10fce8\") " pod="openshift-dns/node-resolver-4k22m" May 28 16:07:43.295932 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.295169 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8d262a97-a722-48e6-a264-4fdba30f80e8-host\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.295932 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.295190 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5ae4021e-a738-403c-9143-976b0c75fca8-host\") pod \"node-ca-z55zs\" (UID: \"5ae4021e-a738-403c-9143-976b0c75fca8\") " pod="openshift-image-registry/node-ca-z55zs" May 28 16:07:43.295932 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.295215 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.295932 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.295259 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/714f97cf-9a9a-4321-a50d-511802063910-cnibin\") pod \"multus-additional-cni-plugins-j8ptw\" (UID: \"714f97cf-9a9a-4321-a50d-511802063910\") " pod="openshift-multus/multus-additional-cni-plugins-j8ptw" May 28 16:07:43.295932 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.295306 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46jwd\" (UniqueName: \"kubernetes.io/projected/714f97cf-9a9a-4321-a50d-511802063910-kube-api-access-46jwd\") pod \"multus-additional-cni-plugins-j8ptw\" (UID: \"714f97cf-9a9a-4321-a50d-511802063910\") " pod="openshift-multus/multus-additional-cni-plugins-j8ptw" May 28 16:07:43.295932 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.295335 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-run-ovn\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.296406 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.295358 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d07065a1-27ff-4f50-9b8e-88d35f69bc89-env-overrides\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.296406 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.295410 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-cnibin\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.296406 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.295464 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-host-var-lib-cni-bin\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.298507 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.298477 2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" May 28 16:07:43.301812 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.301790 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxmxj\" (UniqueName: \"kubernetes.io/projected/3b54fd06-9913-49b9-ac98-d2916e5f0276-kube-api-access-mxmxj\") pod \"network-metrics-daemon-6dt52\" (UID: \"3b54fd06-9913-49b9-ac98-d2916e5f0276\") " pod="openshift-multus/network-metrics-daemon-6dt52" May 28 16:07:43.301927 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.301823 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vk8m6\" (UniqueName: \"kubernetes.io/projected/7acdfd90-ea35-421a-9249-f372fb4ea69c-kube-api-access-vk8m6\") pod \"aws-ebs-csi-driver-node-4swv9\" (UID: \"7acdfd90-ea35-421a-9249-f372fb4ea69c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4swv9" May 28 16:07:43.358223 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.358026 2566 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" May 28 16:07:43.396156 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396109 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-systemd-units\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.396156 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396161 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a40ad2f3-e916-4272-9a82-9b0a818900e5-cni-binary-copy\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.396427 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396189 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dc91efd9-d166-4a72-bcdc-3247cc10fce8-hosts-file\") pod \"node-resolver-4k22m\" (UID: \"dc91efd9-d166-4a72-bcdc-3247cc10fce8\") " pod="openshift-dns/node-resolver-4k22m" May 28 16:07:43.396427 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396214 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8d262a97-a722-48e6-a264-4fdba30f80e8-etc-sysctl-conf\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.396427 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396231 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8d262a97-a722-48e6-a264-4fdba30f80e8-sys\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.396427 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396231 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-systemd-units\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.396427 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396247 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v9cbg\" (UniqueName: \"kubernetes.io/projected/5ae4021e-a738-403c-9143-976b0c75fca8-kube-api-access-v9cbg\") pod \"node-ca-z55zs\" (UID: \"5ae4021e-a738-403c-9143-976b0c75fca8\") " pod="openshift-image-registry/node-ca-z55zs" May 28 16:07:43.396427 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396271 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-host-kubelet\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.396427 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396308 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-system-cni-dir\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.396427 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396331 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8d262a97-a722-48e6-a264-4fdba30f80e8-etc-systemd\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.396427 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396334 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dc91efd9-d166-4a72-bcdc-3247cc10fce8-hosts-file\") pod \"node-resolver-4k22m\" (UID: \"dc91efd9-d166-4a72-bcdc-3247cc10fce8\") " pod="openshift-dns/node-resolver-4k22m" May 28 16:07:43.396427 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396350 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j4gx5\" (UniqueName: \"kubernetes.io/projected/8d262a97-a722-48e6-a264-4fdba30f80e8-kube-api-access-j4gx5\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.396427 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396373 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-run-openvswitch\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.396427 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396381 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-host-kubelet\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.396427 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396393 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-node-log\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.396427 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396416 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8d262a97-a722-48e6-a264-4fdba30f80e8-etc-sysctl-d\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.396427 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396421 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8d262a97-a722-48e6-a264-4fdba30f80e8-etc-sysctl-conf\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.396427 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396433 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8d262a97-a722-48e6-a264-4fdba30f80e8-sys\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.396986 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396438 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8d262a97-a722-48e6-a264-4fdba30f80e8-etc-tuned\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.396986 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396482 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-run-openvswitch\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.396986 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396480 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/714f97cf-9a9a-4321-a50d-511802063910-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j8ptw\" (UID: \"714f97cf-9a9a-4321-a50d-511802063910\") " pod="openshift-multus/multus-additional-cni-plugins-j8ptw" May 28 16:07:43.396986 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396519 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-os-release\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.396986 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396541 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-multus-socket-dir-parent\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.396986 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396566 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-host-run-netns\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.396986 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396589 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-host-run-multus-certs\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.396986 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396613 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8d262a97-a722-48e6-a264-4fdba30f80e8-etc-modprobe-d\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.396986 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396636 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8d262a97-a722-48e6-a264-4fdba30f80e8-run\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.396986 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396664 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-host-run-ovn-kubernetes\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.396986 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396695 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ztdpt\" (UniqueName: \"kubernetes.io/projected/f15055af-8047-4a04-b79f-64af8836b946-kube-api-access-ztdpt\") pod \"network-check-target-2nzbn\" (UID: \"f15055af-8047-4a04-b79f-64af8836b946\") " pod="openshift-network-diagnostics/network-check-target-2nzbn" May 28 16:07:43.396986 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396721 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/eb400c72-a841-4a4b-be8b-6684abea58b8-iptables-alerter-script\") pod \"iptables-alerter-vf6tv\" (UID: \"eb400c72-a841-4a4b-be8b-6684abea58b8\") " pod="openshift-network-operator/iptables-alerter-vf6tv" May 28 16:07:43.396986 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396747 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8d262a97-a722-48e6-a264-4fdba30f80e8-lib-modules\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.396986 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396783 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d262a97-a722-48e6-a264-4fdba30f80e8-var-lib-kubelet\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.396986 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396806 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/714f97cf-9a9a-4321-a50d-511802063910-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j8ptw\" (UID: \"714f97cf-9a9a-4321-a50d-511802063910\") " pod="openshift-multus/multus-additional-cni-plugins-j8ptw" May 28 16:07:43.396986 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396820 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a40ad2f3-e916-4272-9a82-9b0a818900e5-cni-binary-copy\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.396986 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396832 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-host-run-netns\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.397932 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396859 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-var-lib-openvswitch\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.397932 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396886 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d07065a1-27ff-4f50-9b8e-88d35f69bc89-ovn-node-metrics-cert\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.397932 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396973 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8d262a97-a722-48e6-a264-4fdba30f80e8-etc-modprobe-d\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.397932 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396982 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/714f97cf-9a9a-4321-a50d-511802063910-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j8ptw\" (UID: \"714f97cf-9a9a-4321-a50d-511802063910\") " pod="openshift-multus/multus-additional-cni-plugins-j8ptw" May 28 16:07:43.397932 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396984 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8d262a97-a722-48e6-a264-4fdba30f80e8-etc-sysctl-d\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.397932 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.396993 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8d262a97-a722-48e6-a264-4fdba30f80e8-etc-systemd\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.397932 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397026 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-etc-kubernetes\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.397932 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397031 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-multus-socket-dir-parent\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.397932 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397068 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-host-run-multus-certs\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.397932 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397071 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-host-run-netns\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.397932 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397088 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-node-log\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.397932 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397093 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-etc-kubernetes\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.397932 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397113 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-system-cni-dir\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.397932 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397121 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8d262a97-a722-48e6-a264-4fdba30f80e8-etc-sysconfig\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.397932 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397161 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d262a97-a722-48e6-a264-4fdba30f80e8-var-lib-kubelet\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.397932 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397173 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8d262a97-a722-48e6-a264-4fdba30f80e8-lib-modules\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.397932 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397193 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/714f97cf-9a9a-4321-a50d-511802063910-system-cni-dir\") pod \"multus-additional-cni-plugins-j8ptw\" (UID: \"714f97cf-9a9a-4321-a50d-511802063910\") " pod="openshift-multus/multus-additional-cni-plugins-j8ptw" May 28 16:07:43.397932 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397201 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8d262a97-a722-48e6-a264-4fdba30f80e8-etc-sysconfig\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.398799 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397159 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/714f97cf-9a9a-4321-a50d-511802063910-system-cni-dir\") pod \"multus-additional-cni-plugins-j8ptw\" (UID: \"714f97cf-9a9a-4321-a50d-511802063910\") " pod="openshift-multus/multus-additional-cni-plugins-j8ptw" May 28 16:07:43.398799 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397223 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8d262a97-a722-48e6-a264-4fdba30f80e8-run\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.398799 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397234 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-multus-conf-dir\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.398799 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397244 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-host-run-netns\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.398799 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397261 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c0258ed5-55e5-4a30-a658-3e8d61330e0a-agent-certs\") pod \"konnectivity-agent-jq2b6\" (UID: \"c0258ed5-55e5-4a30-a658-3e8d61330e0a\") " pod="kube-system/konnectivity-agent-jq2b6" May 28 16:07:43.398799 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397267 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-host-run-ovn-kubernetes\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.398799 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397304 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-var-lib-openvswitch\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.398799 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397306 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d262a97-a722-48e6-a264-4fdba30f80e8-tmp\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.398799 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397368 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-os-release\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.398799 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397345 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/714f97cf-9a9a-4321-a50d-511802063910-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j8ptw\" (UID: \"714f97cf-9a9a-4321-a50d-511802063910\") " pod="openshift-multus/multus-additional-cni-plugins-j8ptw" May 28 16:07:43.398799 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397428 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-log-socket\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.398799 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397479 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-log-socket\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.398799 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397493 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-host-cni-netd\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.398799 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397617 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/714f97cf-9a9a-4321-a50d-511802063910-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j8ptw\" (UID: \"714f97cf-9a9a-4321-a50d-511802063910\") " pod="openshift-multus/multus-additional-cni-plugins-j8ptw" May 28 16:07:43.398799 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397675 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-multus-conf-dir\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.398799 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397693 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dzpz9\" (UniqueName: \"kubernetes.io/projected/d07065a1-27ff-4f50-9b8e-88d35f69bc89-kube-api-access-dzpz9\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.398799 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397716 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-host-cni-netd\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.398799 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397723 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eb400c72-a841-4a4b-be8b-6684abea58b8-host-slash\") pod \"iptables-alerter-vf6tv\" (UID: \"eb400c72-a841-4a4b-be8b-6684abea58b8\") " pod="openshift-network-operator/iptables-alerter-vf6tv" May 28 16:07:43.399662 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397749 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/714f97cf-9a9a-4321-a50d-511802063910-os-release\") pod \"multus-additional-cni-plugins-j8ptw\" (UID: \"714f97cf-9a9a-4321-a50d-511802063910\") " pod="openshift-multus/multus-additional-cni-plugins-j8ptw" May 28 16:07:43.399662 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397775 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-host-var-lib-cni-multus\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.399662 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397783 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/eb400c72-a841-4a4b-be8b-6684abea58b8-iptables-alerter-script\") pod \"iptables-alerter-vf6tv\" (UID: \"eb400c72-a841-4a4b-be8b-6684abea58b8\") " pod="openshift-network-operator/iptables-alerter-vf6tv" May 28 16:07:43.399662 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397802 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c0258ed5-55e5-4a30-a658-3e8d61330e0a-konnectivity-ca\") pod \"konnectivity-agent-jq2b6\" (UID: \"c0258ed5-55e5-4a30-a658-3e8d61330e0a\") " pod="kube-system/konnectivity-agent-jq2b6" May 28 16:07:43.399662 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397830 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d07065a1-27ff-4f50-9b8e-88d35f69bc89-ovnkube-config\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.399662 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397855 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-hostroot\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.399662 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397864 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/714f97cf-9a9a-4321-a50d-511802063910-os-release\") pod \"multus-additional-cni-plugins-j8ptw\" (UID: \"714f97cf-9a9a-4321-a50d-511802063910\") " pod="openshift-multus/multus-additional-cni-plugins-j8ptw" May 28 16:07:43.399662 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397896 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-host-slash\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.399662 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397921 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d07065a1-27ff-4f50-9b8e-88d35f69bc89-ovnkube-script-lib\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.399662 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397950 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-multus-cni-dir\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.399662 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397977 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-etc-openvswitch\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.399662 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397978 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-hostroot\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.399662 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.398014 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-host-var-lib-cni-multus\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.399662 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.397921 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eb400c72-a841-4a4b-be8b-6684abea58b8-host-slash\") pod \"iptables-alerter-vf6tv\" (UID: \"eb400c72-a841-4a4b-be8b-6684abea58b8\") " pod="openshift-network-operator/iptables-alerter-vf6tv" May 28 16:07:43.399662 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.398048 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a40ad2f3-e916-4272-9a82-9b0a818900e5-multus-daemon-config\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.399662 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.398079 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zmk64\" (UniqueName: \"kubernetes.io/projected/dc91efd9-d166-4a72-bcdc-3247cc10fce8-kube-api-access-zmk64\") pod \"node-resolver-4k22m\" (UID: \"dc91efd9-d166-4a72-bcdc-3247cc10fce8\") " pod="openshift-dns/node-resolver-4k22m" May 28 16:07:43.399662 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.398098 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-multus-cni-dir\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.399662 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.398105 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d262a97-a722-48e6-a264-4fdba30f80e8-etc-kubernetes\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.400411 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.398141 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-etc-openvswitch\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.400411 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.398151 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d262a97-a722-48e6-a264-4fdba30f80e8-etc-kubernetes\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.400411 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.398168 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5ae4021e-a738-403c-9143-976b0c75fca8-serviceca\") pod \"node-ca-z55zs\" (UID: \"5ae4021e-a738-403c-9143-976b0c75fca8\") " pod="openshift-image-registry/node-ca-z55zs" May 28 16:07:43.400411 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.398195 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/714f97cf-9a9a-4321-a50d-511802063910-cni-binary-copy\") pod \"multus-additional-cni-plugins-j8ptw\" (UID: \"714f97cf-9a9a-4321-a50d-511802063910\") " pod="openshift-multus/multus-additional-cni-plugins-j8ptw" May 28 16:07:43.400411 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.398221 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-run-systemd\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.400411 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.398246 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-host-cni-bin\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.400411 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.398272 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4qs82\" (UniqueName: \"kubernetes.io/projected/eb400c72-a841-4a4b-be8b-6684abea58b8-kube-api-access-4qs82\") pod \"iptables-alerter-vf6tv\" (UID: \"eb400c72-a841-4a4b-be8b-6684abea58b8\") " pod="openshift-network-operator/iptables-alerter-vf6tv" May 28 16:07:43.400411 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.398319 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-host-var-lib-kubelet\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.400411 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.398345 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc91efd9-d166-4a72-bcdc-3247cc10fce8-tmp-dir\") pod \"node-resolver-4k22m\" (UID: \"dc91efd9-d166-4a72-bcdc-3247cc10fce8\") " pod="openshift-dns/node-resolver-4k22m" May 28 16:07:43.400411 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.398358 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/714f97cf-9a9a-4321-a50d-511802063910-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j8ptw\" (UID: \"714f97cf-9a9a-4321-a50d-511802063910\") " pod="openshift-multus/multus-additional-cni-plugins-j8ptw" May 28 16:07:43.400411 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.398368 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8d262a97-a722-48e6-a264-4fdba30f80e8-host\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.400411 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.398405 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5ae4021e-a738-403c-9143-976b0c75fca8-host\") pod \"node-ca-z55zs\" (UID: \"5ae4021e-a738-403c-9143-976b0c75fca8\") " pod="openshift-image-registry/node-ca-z55zs" May 28 16:07:43.400411 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.398418 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8d262a97-a722-48e6-a264-4fdba30f80e8-host\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.400411 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.398434 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.400411 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.398464 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/714f97cf-9a9a-4321-a50d-511802063910-cnibin\") pod \"multus-additional-cni-plugins-j8ptw\" (UID: \"714f97cf-9a9a-4321-a50d-511802063910\") " pod="openshift-multus/multus-additional-cni-plugins-j8ptw" May 28 16:07:43.400411 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.398489 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-46jwd\" (UniqueName: \"kubernetes.io/projected/714f97cf-9a9a-4321-a50d-511802063910-kube-api-access-46jwd\") pod \"multus-additional-cni-plugins-j8ptw\" (UID: \"714f97cf-9a9a-4321-a50d-511802063910\") " pod="openshift-multus/multus-additional-cni-plugins-j8ptw" May 28 16:07:43.400411 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.398518 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-run-ovn\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.400895 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.398544 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d07065a1-27ff-4f50-9b8e-88d35f69bc89-env-overrides\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.400895 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.398555 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-host-slash\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.400895 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.398566 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-cnibin\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.400895 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.398591 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-host-var-lib-cni-bin\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.400895 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.398617 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-host-run-k8s-cni-cncf-io\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.400895 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.398645 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zd5gg\" (UniqueName: \"kubernetes.io/projected/a40ad2f3-e916-4272-9a82-9b0a818900e5-kube-api-access-zd5gg\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.400895 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.398650 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d07065a1-27ff-4f50-9b8e-88d35f69bc89-ovnkube-script-lib\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.400895 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.398517 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d07065a1-27ff-4f50-9b8e-88d35f69bc89-ovnkube-config\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.400895 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.398719 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/714f97cf-9a9a-4321-a50d-511802063910-cnibin\") pod \"multus-additional-cni-plugins-j8ptw\" (UID: \"714f97cf-9a9a-4321-a50d-511802063910\") " pod="openshift-multus/multus-additional-cni-plugins-j8ptw" May 28 16:07:43.400895 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.398894 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5ae4021e-a738-403c-9143-976b0c75fca8-serviceca\") pod \"node-ca-z55zs\" (UID: \"5ae4021e-a738-403c-9143-976b0c75fca8\") " pod="openshift-image-registry/node-ca-z55zs" May 28 16:07:43.400895 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.398946 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-run-ovn\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.400895 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.399048 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c0258ed5-55e5-4a30-a658-3e8d61330e0a-konnectivity-ca\") pod \"konnectivity-agent-jq2b6\" (UID: \"c0258ed5-55e5-4a30-a658-3e8d61330e0a\") " pod="kube-system/konnectivity-agent-jq2b6" May 28 16:07:43.400895 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.399058 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d07065a1-27ff-4f50-9b8e-88d35f69bc89-env-overrides\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.400895 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.399117 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-host-run-k8s-cni-cncf-io\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.400895 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.399158 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-run-systemd\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.400895 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.399193 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-host-cni-bin\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.400895 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.399253 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8d262a97-a722-48e6-a264-4fdba30f80e8-etc-tuned\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.400895 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.399350 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/714f97cf-9a9a-4321-a50d-511802063910-cni-binary-copy\") pod \"multus-additional-cni-plugins-j8ptw\" (UID: \"714f97cf-9a9a-4321-a50d-511802063910\") " pod="openshift-multus/multus-additional-cni-plugins-j8ptw" May 28 16:07:43.401382 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.399356 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-cnibin\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.401382 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.399412 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-host-var-lib-kubelet\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.401382 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.399450 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a40ad2f3-e916-4272-9a82-9b0a818900e5-host-var-lib-cni-bin\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.401382 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.399493 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d07065a1-27ff-4f50-9b8e-88d35f69bc89-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.401382 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.399547 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5ae4021e-a738-403c-9143-976b0c75fca8-host\") pod \"node-ca-z55zs\" (UID: \"5ae4021e-a738-403c-9143-976b0c75fca8\") " pod="openshift-image-registry/node-ca-z55zs" May 28 16:07:43.401382 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.399564 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d07065a1-27ff-4f50-9b8e-88d35f69bc89-ovn-node-metrics-cert\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.401382 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.399634 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc91efd9-d166-4a72-bcdc-3247cc10fce8-tmp-dir\") pod \"node-resolver-4k22m\" (UID: \"dc91efd9-d166-4a72-bcdc-3247cc10fce8\") " pod="openshift-dns/node-resolver-4k22m" May 28 16:07:43.401382 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.399809 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a40ad2f3-e916-4272-9a82-9b0a818900e5-multus-daemon-config\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.401382 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.401215 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d262a97-a722-48e6-a264-4fdba30f80e8-tmp\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.401669 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.401444 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c0258ed5-55e5-4a30-a658-3e8d61330e0a-agent-certs\") pod \"konnectivity-agent-jq2b6\" (UID: \"c0258ed5-55e5-4a30-a658-3e8d61330e0a\") " pod="kube-system/konnectivity-agent-jq2b6" May 28 16:07:43.403986 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:43.403961 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 16:07:43.403986 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:43.403991 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 16:07:43.405162 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:43.404005 2566 projected.go:194] Error preparing data for projected volume kube-api-access-ztdpt for pod openshift-network-diagnostics/network-check-target-2nzbn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 16:07:43.405162 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:43.404082 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f15055af-8047-4a04-b79f-64af8836b946-kube-api-access-ztdpt podName:f15055af-8047-4a04-b79f-64af8836b946 nodeName:}" failed. No retries permitted until 2026-05-28 16:07:43.904062224 +0000 UTC m=+3.131092778 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-ztdpt" (UniqueName: "kubernetes.io/projected/f15055af-8047-4a04-b79f-64af8836b946-kube-api-access-ztdpt") pod "network-check-target-2nzbn" (UID: "f15055af-8047-4a04-b79f-64af8836b946") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 16:07:43.406947 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.406860 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzpz9\" (UniqueName: \"kubernetes.io/projected/d07065a1-27ff-4f50-9b8e-88d35f69bc89-kube-api-access-dzpz9\") pod \"ovnkube-node-db77q\" (UID: \"d07065a1-27ff-4f50-9b8e-88d35f69bc89\") " pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.407491 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.407329 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4gx5\" (UniqueName: \"kubernetes.io/projected/8d262a97-a722-48e6-a264-4fdba30f80e8-kube-api-access-j4gx5\") pod \"tuned-vxt4n\" (UID: \"8d262a97-a722-48e6-a264-4fdba30f80e8\") " pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.407940 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.407895 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmk64\" (UniqueName: \"kubernetes.io/projected/dc91efd9-d166-4a72-bcdc-3247cc10fce8-kube-api-access-zmk64\") pod \"node-resolver-4k22m\" (UID: \"dc91efd9-d166-4a72-bcdc-3247cc10fce8\") " pod="openshift-dns/node-resolver-4k22m" May 28 16:07:43.408106 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.408085 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qs82\" (UniqueName: \"kubernetes.io/projected/eb400c72-a841-4a4b-be8b-6684abea58b8-kube-api-access-4qs82\") pod \"iptables-alerter-vf6tv\" (UID: \"eb400c72-a841-4a4b-be8b-6684abea58b8\") " pod="openshift-network-operator/iptables-alerter-vf6tv" May 28 16:07:43.408591 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.408540 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd5gg\" (UniqueName: \"kubernetes.io/projected/a40ad2f3-e916-4272-9a82-9b0a818900e5-kube-api-access-zd5gg\") pod \"multus-b56lh\" (UID: \"a40ad2f3-e916-4272-9a82-9b0a818900e5\") " pod="openshift-multus/multus-b56lh" May 28 16:07:43.408591 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.408577 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-46jwd\" (UniqueName: \"kubernetes.io/projected/714f97cf-9a9a-4321-a50d-511802063910-kube-api-access-46jwd\") pod \"multus-additional-cni-plugins-j8ptw\" (UID: \"714f97cf-9a9a-4321-a50d-511802063910\") " pod="openshift-multus/multus-additional-cni-plugins-j8ptw" May 28 16:07:43.409910 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.409885 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9cbg\" (UniqueName: \"kubernetes.io/projected/5ae4021e-a738-403c-9143-976b0c75fca8-kube-api-access-v9cbg\") pod \"node-ca-z55zs\" (UID: \"5ae4021e-a738-403c-9143-976b0c75fca8\") " pod="openshift-image-registry/node-ca-z55zs" May 28 16:07:43.493065 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.493022 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4swv9" May 28 16:07:43.502937 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.502903 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:07:43.513754 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.513723 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jq2b6" May 28 16:07:43.517479 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.517448 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" May 28 16:07:43.526160 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.526124 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vf6tv" May 28 16:07:43.533904 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.533876 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-b56lh" May 28 16:07:43.540603 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.540577 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4k22m" May 28 16:07:43.547348 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.547316 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-z55zs" May 28 16:07:43.552139 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.552110 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j8ptw" May 28 16:07:43.801311 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:43.801210 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b54fd06-9913-49b9-ac98-d2916e5f0276-metrics-certs\") pod \"network-metrics-daemon-6dt52\" (UID: \"3b54fd06-9913-49b9-ac98-d2916e5f0276\") " pod="openshift-multus/network-metrics-daemon-6dt52" May 28 16:07:43.801451 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:43.801345 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 16:07:43.801451 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:43.801402 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b54fd06-9913-49b9-ac98-d2916e5f0276-metrics-certs podName:3b54fd06-9913-49b9-ac98-d2916e5f0276 nodeName:}" failed. No retries permitted until 2026-05-28 16:07:44.801388936 +0000 UTC m=+4.028419470 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b54fd06-9913-49b9-ac98-d2916e5f0276-metrics-certs") pod "network-metrics-daemon-6dt52" (UID: "3b54fd06-9913-49b9-ac98-d2916e5f0276") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 16:07:43.867112 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:43.866886 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb400c72_a841_4a4b_be8b_6684abea58b8.slice/crio-a2f553e7ac3703e4ef8818bb43ff43ad2e9e54632ff1fc41ca904b28fba49557 WatchSource:0}: Error finding container a2f553e7ac3703e4ef8818bb43ff43ad2e9e54632ff1fc41ca904b28fba49557: Status 404 returned error can't find the container with id a2f553e7ac3703e4ef8818bb43ff43ad2e9e54632ff1fc41ca904b28fba49557 May 28 16:07:43.868244 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:43.868217 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0258ed5_55e5_4a30_a658_3e8d61330e0a.slice/crio-f56362ef6d8b362c4ef5745838f8d7c0fea29aae524c91c53a54a72d8847b753 WatchSource:0}: Error finding container f56362ef6d8b362c4ef5745838f8d7c0fea29aae524c91c53a54a72d8847b753: Status 404 returned error can't find the container with id f56362ef6d8b362c4ef5745838f8d7c0fea29aae524c91c53a54a72d8847b753 May 28 16:07:43.869362 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:43.869123 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d262a97_a722_48e6_a264_4fdba30f80e8.slice/crio-0e251f1ce6b7d3d8724bfcf7e4ab1a94ac5f08991a046ffa48e2142a45e38d26 WatchSource:0}: Error finding container 0e251f1ce6b7d3d8724bfcf7e4ab1a94ac5f08991a046ffa48e2142a45e38d26: Status 404 returned error can't find the container with id 0e251f1ce6b7d3d8724bfcf7e4ab1a94ac5f08991a046ffa48e2142a45e38d26 May 28 16:07:43.874320 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:43.873433 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc91efd9_d166_4a72_bcdc_3247cc10fce8.slice/crio-f0700d4c68461c18ac34417e3c17254466bc71aa87f6d28636d0ad6759784a70 WatchSource:0}: Error finding container f0700d4c68461c18ac34417e3c17254466bc71aa87f6d28636d0ad6759784a70: Status 404 returned error can't find the container with id f0700d4c68461c18ac34417e3c17254466bc71aa87f6d28636d0ad6759784a70 May 28 16:07:43.874320 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:43.873738 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7acdfd90_ea35_421a_9249_f372fb4ea69c.slice/crio-7488b52cb530b9f0a9ad5c352a71b5d80b200ca810131896833516cbf9af058f WatchSource:0}: Error finding container 7488b52cb530b9f0a9ad5c352a71b5d80b200ca810131896833516cbf9af058f: Status 404 returned error can't find the container with id 7488b52cb530b9f0a9ad5c352a71b5d80b200ca810131896833516cbf9af058f May 28 16:07:43.874539 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:43.874344 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda40ad2f3_e916_4272_9a82_9b0a818900e5.slice/crio-94460b0e2f42d4faf7e9d3a79c9cc20993fb783cff0e25c896c0d6a2fb3cb4b7 WatchSource:0}: Error finding container 94460b0e2f42d4faf7e9d3a79c9cc20993fb783cff0e25c896c0d6a2fb3cb4b7: Status 404 returned error can't find the container with id 94460b0e2f42d4faf7e9d3a79c9cc20993fb783cff0e25c896c0d6a2fb3cb4b7 May 28 16:07:43.876245 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:43.876039 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod714f97cf_9a9a_4321_a50d_511802063910.slice/crio-0fd00220bab0a0e02dae4d0626e3c35cf91ca619ce6f9195504e0c9fa0267c97 WatchSource:0}: Error finding container 0fd00220bab0a0e02dae4d0626e3c35cf91ca619ce6f9195504e0c9fa0267c97: Status 404 returned error can't find the container with id 0fd00220bab0a0e02dae4d0626e3c35cf91ca619ce6f9195504e0c9fa0267c97 May 28 16:07:43.878370 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:43.877827 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ae4021e_a738_403c_9143_976b0c75fca8.slice/crio-e549b0a7439a08672a9178dc9fb3c8c8a1d85b9b1f8694c86690deba02894c01 WatchSource:0}: Error finding container e549b0a7439a08672a9178dc9fb3c8c8a1d85b9b1f8694c86690deba02894c01: Status 404 returned error can't find the container with id e549b0a7439a08672a9178dc9fb3c8c8a1d85b9b1f8694c86690deba02894c01 May 28 16:07:43.880129 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:07:43.880108 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd07065a1_27ff_4f50_9b8e_88d35f69bc89.slice/crio-78b31880a065bfed2af11f4a631f8355edc35a3cc58be7d77e0937dbdd9e9f49 WatchSource:0}: Error finding container 78b31880a065bfed2af11f4a631f8355edc35a3cc58be7d77e0937dbdd9e9f49: Status 404 returned error can't find the container with id 78b31880a065bfed2af11f4a631f8355edc35a3cc58be7d77e0937dbdd9e9f49 May 28 16:07:44.003106 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:44.003071 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ztdpt\" (UniqueName: \"kubernetes.io/projected/f15055af-8047-4a04-b79f-64af8836b946-kube-api-access-ztdpt\") pod \"network-check-target-2nzbn\" (UID: \"f15055af-8047-4a04-b79f-64af8836b946\") " pod="openshift-network-diagnostics/network-check-target-2nzbn" May 28 16:07:44.003267 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:44.003197 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 16:07:44.003267 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:44.003212 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 16:07:44.003267 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:44.003221 2566 projected.go:194] Error preparing data for projected volume kube-api-access-ztdpt for pod openshift-network-diagnostics/network-check-target-2nzbn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 16:07:44.003400 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:44.003271 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f15055af-8047-4a04-b79f-64af8836b946-kube-api-access-ztdpt podName:f15055af-8047-4a04-b79f-64af8836b946 nodeName:}" failed. No retries permitted until 2026-05-28 16:07:45.003257843 +0000 UTC m=+4.230288381 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-ztdpt" (UniqueName: "kubernetes.io/projected/f15055af-8047-4a04-b79f-64af8836b946-kube-api-access-ztdpt") pod "network-check-target-2nzbn" (UID: "f15055af-8047-4a04-b79f-64af8836b946") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 16:07:44.272058 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:44.272023 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 16:02:42 +0000 UTC" deadline="2028-03-03 04:51:43.112812802 +0000 UTC" May 28 16:07:44.272058 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:44.272053 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15468h43m58.840762802s" May 28 16:07:44.342956 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:44.342916 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4k22m" event={"ID":"dc91efd9-d166-4a72-bcdc-3247cc10fce8","Type":"ContainerStarted","Data":"f0700d4c68461c18ac34417e3c17254466bc71aa87f6d28636d0ad6759784a70"} May 28 16:07:44.344277 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:44.344248 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jq2b6" event={"ID":"c0258ed5-55e5-4a30-a658-3e8d61330e0a","Type":"ContainerStarted","Data":"f56362ef6d8b362c4ef5745838f8d7c0fea29aae524c91c53a54a72d8847b753"} May 28 16:07:44.345656 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:44.345618 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vf6tv" event={"ID":"eb400c72-a841-4a4b-be8b-6684abea58b8","Type":"ContainerStarted","Data":"a2f553e7ac3703e4ef8818bb43ff43ad2e9e54632ff1fc41ca904b28fba49557"} May 28 16:07:44.348734 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:44.348673 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-z55zs" event={"ID":"5ae4021e-a738-403c-9143-976b0c75fca8","Type":"ContainerStarted","Data":"e549b0a7439a08672a9178dc9fb3c8c8a1d85b9b1f8694c86690deba02894c01"} May 28 16:07:44.350521 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:44.350460 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4swv9" event={"ID":"7acdfd90-ea35-421a-9249-f372fb4ea69c","Type":"ContainerStarted","Data":"7488b52cb530b9f0a9ad5c352a71b5d80b200ca810131896833516cbf9af058f"} May 28 16:07:44.355259 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:44.355201 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" event={"ID":"8d262a97-a722-48e6-a264-4fdba30f80e8","Type":"ContainerStarted","Data":"0e251f1ce6b7d3d8724bfcf7e4ab1a94ac5f08991a046ffa48e2142a45e38d26"} May 28 16:07:44.360543 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:44.359703 2566 generic.go:358] "Generic (PLEG): container finished" podID="90fc7a1de7d02404a7ce08e7f99d20e6" containerID="a65baca1a702c3539b005f0d61f64e70c8212838856467dd92bb0736fe5aa5d1" exitCode=0 May 28 16:07:44.360543 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:44.359774 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal" event={"ID":"90fc7a1de7d02404a7ce08e7f99d20e6","Type":"ContainerDied","Data":"a65baca1a702c3539b005f0d61f64e70c8212838856467dd92bb0736fe5aa5d1"} May 28 16:07:44.368380 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:44.368345 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-102.ec2.internal" event={"ID":"3682fbf2c94d063cca39e2c160569366","Type":"ContainerStarted","Data":"b5cea632490a79c1288d6492ef5aae73194115a406c5ccb49b61f18404bd2569"} May 28 16:07:44.371256 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:44.371206 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-db77q" event={"ID":"d07065a1-27ff-4f50-9b8e-88d35f69bc89","Type":"ContainerStarted","Data":"78b31880a065bfed2af11f4a631f8355edc35a3cc58be7d77e0937dbdd9e9f49"} May 28 16:07:44.385248 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:44.385196 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j8ptw" event={"ID":"714f97cf-9a9a-4321-a50d-511802063910","Type":"ContainerStarted","Data":"0fd00220bab0a0e02dae4d0626e3c35cf91ca619ce6f9195504e0c9fa0267c97"} May 28 16:07:44.410624 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:44.410462 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-b56lh" event={"ID":"a40ad2f3-e916-4272-9a82-9b0a818900e5","Type":"ContainerStarted","Data":"94460b0e2f42d4faf7e9d3a79c9cc20993fb783cff0e25c896c0d6a2fb3cb4b7"} May 28 16:07:44.812686 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:44.812581 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b54fd06-9913-49b9-ac98-d2916e5f0276-metrics-certs\") pod \"network-metrics-daemon-6dt52\" (UID: \"3b54fd06-9913-49b9-ac98-d2916e5f0276\") " pod="openshift-multus/network-metrics-daemon-6dt52" May 28 16:07:44.812877 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:44.812741 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 16:07:44.812877 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:44.812824 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b54fd06-9913-49b9-ac98-d2916e5f0276-metrics-certs podName:3b54fd06-9913-49b9-ac98-d2916e5f0276 nodeName:}" failed. No retries permitted until 2026-05-28 16:07:46.812800234 +0000 UTC m=+6.039830780 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b54fd06-9913-49b9-ac98-d2916e5f0276-metrics-certs") pod "network-metrics-daemon-6dt52" (UID: "3b54fd06-9913-49b9-ac98-d2916e5f0276") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 16:07:45.014300 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:45.014248 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ztdpt\" (UniqueName: \"kubernetes.io/projected/f15055af-8047-4a04-b79f-64af8836b946-kube-api-access-ztdpt\") pod \"network-check-target-2nzbn\" (UID: \"f15055af-8047-4a04-b79f-64af8836b946\") " pod="openshift-network-diagnostics/network-check-target-2nzbn" May 28 16:07:45.014516 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:45.014473 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 16:07:45.014516 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:45.014498 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 16:07:45.014516 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:45.014511 2566 projected.go:194] Error preparing data for projected volume kube-api-access-ztdpt for pod openshift-network-diagnostics/network-check-target-2nzbn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 16:07:45.014705 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:45.014578 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f15055af-8047-4a04-b79f-64af8836b946-kube-api-access-ztdpt podName:f15055af-8047-4a04-b79f-64af8836b946 nodeName:}" failed. No retries permitted until 2026-05-28 16:07:47.014559366 +0000 UTC m=+6.241589916 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-ztdpt" (UniqueName: "kubernetes.io/projected/f15055af-8047-4a04-b79f-64af8836b946-kube-api-access-ztdpt") pod "network-check-target-2nzbn" (UID: "f15055af-8047-4a04-b79f-64af8836b946") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 16:07:45.333571 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:45.333451 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dt52" May 28 16:07:45.334124 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:45.333590 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dt52" podUID="3b54fd06-9913-49b9-ac98-d2916e5f0276" May 28 16:07:45.334124 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:45.333943 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nzbn" May 28 16:07:45.334124 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:45.334031 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2nzbn" podUID="f15055af-8047-4a04-b79f-64af8836b946" May 28 16:07:45.424474 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:45.424339 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal" event={"ID":"90fc7a1de7d02404a7ce08e7f99d20e6","Type":"ContainerStarted","Data":"d38ce09a1eb3fe81b9f82cb26a909a4e6832efc2f6e99a0a5a537565b2ab3a5e"} May 28 16:07:45.444017 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:45.440681 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-102.ec2.internal" podStartSLOduration=3.440660053 podStartE2EDuration="3.440660053s" podCreationTimestamp="2026-05-28 16:07:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 16:07:44.395123642 +0000 UTC m=+3.622154206" watchObservedRunningTime="2026-05-28 16:07:45.440660053 +0000 UTC m=+4.667690609" May 28 16:07:45.444017 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:45.440939 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-102.ec2.internal" podStartSLOduration=3.440928786 podStartE2EDuration="3.440928786s" podCreationTimestamp="2026-05-28 16:07:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 16:07:45.439864549 +0000 UTC m=+4.666895107" watchObservedRunningTime="2026-05-28 16:07:45.440928786 +0000 UTC m=+4.667959346" May 28 16:07:46.830319 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:46.830258 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b54fd06-9913-49b9-ac98-d2916e5f0276-metrics-certs\") pod \"network-metrics-daemon-6dt52\" (UID: \"3b54fd06-9913-49b9-ac98-d2916e5f0276\") " pod="openshift-multus/network-metrics-daemon-6dt52" May 28 16:07:46.830760 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:46.830430 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 16:07:46.830760 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:46.830492 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b54fd06-9913-49b9-ac98-d2916e5f0276-metrics-certs podName:3b54fd06-9913-49b9-ac98-d2916e5f0276 nodeName:}" failed. No retries permitted until 2026-05-28 16:07:50.830474701 +0000 UTC m=+10.057505254 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b54fd06-9913-49b9-ac98-d2916e5f0276-metrics-certs") pod "network-metrics-daemon-6dt52" (UID: "3b54fd06-9913-49b9-ac98-d2916e5f0276") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 16:07:47.033249 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:47.032593 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ztdpt\" (UniqueName: \"kubernetes.io/projected/f15055af-8047-4a04-b79f-64af8836b946-kube-api-access-ztdpt\") pod \"network-check-target-2nzbn\" (UID: \"f15055af-8047-4a04-b79f-64af8836b946\") " pod="openshift-network-diagnostics/network-check-target-2nzbn" May 28 16:07:47.033249 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:47.032799 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 16:07:47.033249 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:47.032820 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 16:07:47.033249 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:47.032833 2566 projected.go:194] Error preparing data for projected volume kube-api-access-ztdpt for pod openshift-network-diagnostics/network-check-target-2nzbn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 16:07:47.033249 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:47.032894 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f15055af-8047-4a04-b79f-64af8836b946-kube-api-access-ztdpt podName:f15055af-8047-4a04-b79f-64af8836b946 nodeName:}" failed. No retries permitted until 2026-05-28 16:07:51.032873624 +0000 UTC m=+10.259904159 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-ztdpt" (UniqueName: "kubernetes.io/projected/f15055af-8047-4a04-b79f-64af8836b946-kube-api-access-ztdpt") pod "network-check-target-2nzbn" (UID: "f15055af-8047-4a04-b79f-64af8836b946") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 16:07:47.335780 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:47.334870 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dt52" May 28 16:07:47.335780 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:47.334997 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dt52" podUID="3b54fd06-9913-49b9-ac98-d2916e5f0276" May 28 16:07:47.335780 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:47.335381 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nzbn" May 28 16:07:47.335780 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:47.335468 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2nzbn" podUID="f15055af-8047-4a04-b79f-64af8836b946" May 28 16:07:49.333455 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:49.333139 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dt52" May 28 16:07:49.333455 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:49.333302 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dt52" podUID="3b54fd06-9913-49b9-ac98-d2916e5f0276" May 28 16:07:49.334031 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:49.333697 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nzbn" May 28 16:07:49.334031 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:49.333787 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2nzbn" podUID="f15055af-8047-4a04-b79f-64af8836b946" May 28 16:07:50.865583 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:50.865543 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b54fd06-9913-49b9-ac98-d2916e5f0276-metrics-certs\") pod \"network-metrics-daemon-6dt52\" (UID: \"3b54fd06-9913-49b9-ac98-d2916e5f0276\") " pod="openshift-multus/network-metrics-daemon-6dt52" May 28 16:07:50.866000 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:50.865704 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 16:07:50.866000 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:50.865776 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b54fd06-9913-49b9-ac98-d2916e5f0276-metrics-certs podName:3b54fd06-9913-49b9-ac98-d2916e5f0276 nodeName:}" failed. No retries permitted until 2026-05-28 16:07:58.865757733 +0000 UTC m=+18.092788272 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b54fd06-9913-49b9-ac98-d2916e5f0276-metrics-certs") pod "network-metrics-daemon-6dt52" (UID: "3b54fd06-9913-49b9-ac98-d2916e5f0276") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 16:07:51.066997 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:51.066951 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ztdpt\" (UniqueName: \"kubernetes.io/projected/f15055af-8047-4a04-b79f-64af8836b946-kube-api-access-ztdpt\") pod \"network-check-target-2nzbn\" (UID: \"f15055af-8047-4a04-b79f-64af8836b946\") " pod="openshift-network-diagnostics/network-check-target-2nzbn" May 28 16:07:51.067183 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:51.067154 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 16:07:51.067183 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:51.067175 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 16:07:51.067183 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:51.067186 2566 projected.go:194] Error preparing data for projected volume kube-api-access-ztdpt for pod openshift-network-diagnostics/network-check-target-2nzbn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 16:07:51.067313 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:51.067230 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f15055af-8047-4a04-b79f-64af8836b946-kube-api-access-ztdpt podName:f15055af-8047-4a04-b79f-64af8836b946 nodeName:}" failed. No retries permitted until 2026-05-28 16:07:59.067216171 +0000 UTC m=+18.294246709 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-ztdpt" (UniqueName: "kubernetes.io/projected/f15055af-8047-4a04-b79f-64af8836b946-kube-api-access-ztdpt") pod "network-check-target-2nzbn" (UID: "f15055af-8047-4a04-b79f-64af8836b946") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 16:07:51.333794 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:51.333309 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dt52" May 28 16:07:51.333794 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:51.333346 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nzbn" May 28 16:07:51.333794 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:51.333443 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dt52" podUID="3b54fd06-9913-49b9-ac98-d2916e5f0276" May 28 16:07:51.333794 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:51.333575 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2nzbn" podUID="f15055af-8047-4a04-b79f-64af8836b946" May 28 16:07:53.335045 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:53.333213 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dt52" May 28 16:07:53.335045 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:53.333250 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nzbn" May 28 16:07:53.335045 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:53.333380 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dt52" podUID="3b54fd06-9913-49b9-ac98-d2916e5f0276" May 28 16:07:53.335045 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:53.333808 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2nzbn" podUID="f15055af-8047-4a04-b79f-64af8836b946" May 28 16:07:55.332478 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:55.332440 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nzbn" May 28 16:07:55.332478 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:55.332472 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dt52" May 28 16:07:55.332953 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:55.332563 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2nzbn" podUID="f15055af-8047-4a04-b79f-64af8836b946" May 28 16:07:55.332953 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:55.332719 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dt52" podUID="3b54fd06-9913-49b9-ac98-d2916e5f0276" May 28 16:07:57.333237 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:57.333196 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dt52" May 28 16:07:57.333827 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:57.333203 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nzbn" May 28 16:07:57.333827 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:57.333362 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dt52" podUID="3b54fd06-9913-49b9-ac98-d2916e5f0276" May 28 16:07:57.333827 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:57.333380 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2nzbn" podUID="f15055af-8047-4a04-b79f-64af8836b946" May 28 16:07:58.922676 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:58.922635 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b54fd06-9913-49b9-ac98-d2916e5f0276-metrics-certs\") pod \"network-metrics-daemon-6dt52\" (UID: \"3b54fd06-9913-49b9-ac98-d2916e5f0276\") " pod="openshift-multus/network-metrics-daemon-6dt52" May 28 16:07:58.923092 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:58.922789 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 16:07:58.923092 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:58.922882 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b54fd06-9913-49b9-ac98-d2916e5f0276-metrics-certs podName:3b54fd06-9913-49b9-ac98-d2916e5f0276 nodeName:}" failed. No retries permitted until 2026-05-28 16:08:14.922860652 +0000 UTC m=+34.149891190 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b54fd06-9913-49b9-ac98-d2916e5f0276-metrics-certs") pod "network-metrics-daemon-6dt52" (UID: "3b54fd06-9913-49b9-ac98-d2916e5f0276") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 16:07:59.123998 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:59.123959 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ztdpt\" (UniqueName: \"kubernetes.io/projected/f15055af-8047-4a04-b79f-64af8836b946-kube-api-access-ztdpt\") pod \"network-check-target-2nzbn\" (UID: \"f15055af-8047-4a04-b79f-64af8836b946\") " pod="openshift-network-diagnostics/network-check-target-2nzbn" May 28 16:07:59.124202 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:59.124143 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 16:07:59.124202 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:59.124168 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 16:07:59.124202 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:59.124178 2566 projected.go:194] Error preparing data for projected volume kube-api-access-ztdpt for pod openshift-network-diagnostics/network-check-target-2nzbn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 16:07:59.124385 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:59.124245 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f15055af-8047-4a04-b79f-64af8836b946-kube-api-access-ztdpt podName:f15055af-8047-4a04-b79f-64af8836b946 nodeName:}" failed. No retries permitted until 2026-05-28 16:08:15.124227566 +0000 UTC m=+34.351258123 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-ztdpt" (UniqueName: "kubernetes.io/projected/f15055af-8047-4a04-b79f-64af8836b946-kube-api-access-ztdpt") pod "network-check-target-2nzbn" (UID: "f15055af-8047-4a04-b79f-64af8836b946") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 16:07:59.333051 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:59.333009 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nzbn" May 28 16:07:59.333223 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:07:59.333009 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dt52" May 28 16:07:59.333223 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:59.333144 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2nzbn" podUID="f15055af-8047-4a04-b79f-64af8836b946" May 28 16:07:59.333368 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:07:59.333260 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dt52" podUID="3b54fd06-9913-49b9-ac98-d2916e5f0276" May 28 16:08:00.909900 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:00.909866 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-pb7mn"] May 28 16:08:01.053211 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:01.053173 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pb7mn" May 28 16:08:01.053411 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:01.053274 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pb7mn" podUID="b113a9e6-a1e9-43bf-a08c-b7e6e466640f" May 28 16:08:01.140118 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:01.140069 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b113a9e6-a1e9-43bf-a08c-b7e6e466640f-original-pull-secret\") pod \"global-pull-secret-syncer-pb7mn\" (UID: \"b113a9e6-a1e9-43bf-a08c-b7e6e466640f\") " pod="kube-system/global-pull-secret-syncer-pb7mn" May 28 16:08:01.140118 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:01.140116 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b113a9e6-a1e9-43bf-a08c-b7e6e466640f-dbus\") pod \"global-pull-secret-syncer-pb7mn\" (UID: \"b113a9e6-a1e9-43bf-a08c-b7e6e466640f\") " pod="kube-system/global-pull-secret-syncer-pb7mn" May 28 16:08:01.140354 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:01.140169 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b113a9e6-a1e9-43bf-a08c-b7e6e466640f-kubelet-config\") pod \"global-pull-secret-syncer-pb7mn\" (UID: \"b113a9e6-a1e9-43bf-a08c-b7e6e466640f\") " pod="kube-system/global-pull-secret-syncer-pb7mn" May 28 16:08:01.240729 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:01.240685 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b113a9e6-a1e9-43bf-a08c-b7e6e466640f-original-pull-secret\") pod \"global-pull-secret-syncer-pb7mn\" (UID: \"b113a9e6-a1e9-43bf-a08c-b7e6e466640f\") " pod="kube-system/global-pull-secret-syncer-pb7mn" May 28 16:08:01.240908 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:01.240737 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b113a9e6-a1e9-43bf-a08c-b7e6e466640f-dbus\") pod \"global-pull-secret-syncer-pb7mn\" (UID: \"b113a9e6-a1e9-43bf-a08c-b7e6e466640f\") " pod="kube-system/global-pull-secret-syncer-pb7mn" May 28 16:08:01.240908 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:01.240792 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b113a9e6-a1e9-43bf-a08c-b7e6e466640f-kubelet-config\") pod \"global-pull-secret-syncer-pb7mn\" (UID: \"b113a9e6-a1e9-43bf-a08c-b7e6e466640f\") " pod="kube-system/global-pull-secret-syncer-pb7mn" May 28 16:08:01.240908 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:01.240873 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b113a9e6-a1e9-43bf-a08c-b7e6e466640f-kubelet-config\") pod \"global-pull-secret-syncer-pb7mn\" (UID: \"b113a9e6-a1e9-43bf-a08c-b7e6e466640f\") " pod="kube-system/global-pull-secret-syncer-pb7mn" May 28 16:08:01.240908 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:01.240875 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 28 16:08:01.241091 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:01.240943 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b113a9e6-a1e9-43bf-a08c-b7e6e466640f-original-pull-secret podName:b113a9e6-a1e9-43bf-a08c-b7e6e466640f nodeName:}" failed. No retries permitted until 2026-05-28 16:08:01.740925639 +0000 UTC m=+20.967956177 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b113a9e6-a1e9-43bf-a08c-b7e6e466640f-original-pull-secret") pod "global-pull-secret-syncer-pb7mn" (UID: "b113a9e6-a1e9-43bf-a08c-b7e6e466640f") : object "kube-system"/"original-pull-secret" not registered May 28 16:08:01.241091 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:01.241021 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b113a9e6-a1e9-43bf-a08c-b7e6e466640f-dbus\") pod \"global-pull-secret-syncer-pb7mn\" (UID: \"b113a9e6-a1e9-43bf-a08c-b7e6e466640f\") " pod="kube-system/global-pull-secret-syncer-pb7mn" May 28 16:08:01.333354 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:01.333309 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nzbn" May 28 16:08:01.333536 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:01.333354 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dt52" May 28 16:08:01.333536 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:01.333416 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2nzbn" podUID="f15055af-8047-4a04-b79f-64af8836b946" May 28 16:08:01.333536 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:01.333473 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dt52" podUID="3b54fd06-9913-49b9-ac98-d2916e5f0276" May 28 16:08:01.744501 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:01.744464 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b113a9e6-a1e9-43bf-a08c-b7e6e466640f-original-pull-secret\") pod \"global-pull-secret-syncer-pb7mn\" (UID: \"b113a9e6-a1e9-43bf-a08c-b7e6e466640f\") " pod="kube-system/global-pull-secret-syncer-pb7mn" May 28 16:08:01.744706 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:01.744631 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 28 16:08:01.744770 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:01.744723 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b113a9e6-a1e9-43bf-a08c-b7e6e466640f-original-pull-secret podName:b113a9e6-a1e9-43bf-a08c-b7e6e466640f nodeName:}" failed. No retries permitted until 2026-05-28 16:08:02.744702163 +0000 UTC m=+21.971732719 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b113a9e6-a1e9-43bf-a08c-b7e6e466640f-original-pull-secret") pod "global-pull-secret-syncer-pb7mn" (UID: "b113a9e6-a1e9-43bf-a08c-b7e6e466640f") : object "kube-system"/"original-pull-secret" not registered May 28 16:08:02.457169 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:02.456916 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j8ptw" event={"ID":"714f97cf-9a9a-4321-a50d-511802063910","Type":"ContainerStarted","Data":"5fd36fccb7358229c74c4ee08706bc50d1585ef3b3779bd672fee158e352f381"} May 28 16:08:02.459333 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:02.459301 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-b56lh" event={"ID":"a40ad2f3-e916-4272-9a82-9b0a818900e5","Type":"ContainerStarted","Data":"df434ee3ed87c8c6510b5be704c2ea40b0f5b539d950ca8a1892b2f90a303b26"} May 28 16:08:02.460791 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:02.460770 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-z55zs" event={"ID":"5ae4021e-a738-403c-9143-976b0c75fca8","Type":"ContainerStarted","Data":"c0255af3639ba556f4f112cc80bb669fcc6eb2ad7d65c9261aac74e9db2ed972"} May 28 16:08:02.496138 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:02.496083 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-b56lh" podStartSLOduration=3.173721853 podStartE2EDuration="21.496066618s" podCreationTimestamp="2026-05-28 16:07:41 +0000 UTC" firstStartedPulling="2026-05-28 16:07:43.877034382 +0000 UTC m=+3.104064920" lastFinishedPulling="2026-05-28 16:08:02.199379141 +0000 UTC m=+21.426409685" observedRunningTime="2026-05-28 16:08:02.495482307 +0000 UTC m=+21.722512866" watchObservedRunningTime="2026-05-28 16:08:02.496066618 +0000 UTC m=+21.723097175" May 28 16:08:02.510376 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:02.510330 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-z55zs" podStartSLOduration=7.895709899 podStartE2EDuration="21.510315671s" podCreationTimestamp="2026-05-28 16:07:41 +0000 UTC" firstStartedPulling="2026-05-28 16:07:43.8799576 +0000 UTC m=+3.106988148" lastFinishedPulling="2026-05-28 16:07:57.494563366 +0000 UTC m=+16.721593920" observedRunningTime="2026-05-28 16:08:02.510009196 +0000 UTC m=+21.737039751" watchObservedRunningTime="2026-05-28 16:08:02.510315671 +0000 UTC m=+21.737346224" May 28 16:08:02.752708 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:02.752452 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b113a9e6-a1e9-43bf-a08c-b7e6e466640f-original-pull-secret\") pod \"global-pull-secret-syncer-pb7mn\" (UID: \"b113a9e6-a1e9-43bf-a08c-b7e6e466640f\") " pod="kube-system/global-pull-secret-syncer-pb7mn" May 28 16:08:02.752828 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:02.752597 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 28 16:08:02.752863 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:02.752843 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b113a9e6-a1e9-43bf-a08c-b7e6e466640f-original-pull-secret podName:b113a9e6-a1e9-43bf-a08c-b7e6e466640f nodeName:}" failed. No retries permitted until 2026-05-28 16:08:04.752822022 +0000 UTC m=+23.979852572 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b113a9e6-a1e9-43bf-a08c-b7e6e466640f-original-pull-secret") pod "global-pull-secret-syncer-pb7mn" (UID: "b113a9e6-a1e9-43bf-a08c-b7e6e466640f") : object "kube-system"/"original-pull-secret" not registered May 28 16:08:03.333240 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:03.333199 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nzbn" May 28 16:08:03.333425 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:03.333200 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dt52" May 28 16:08:03.333425 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:03.333329 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2nzbn" podUID="f15055af-8047-4a04-b79f-64af8836b946" May 28 16:08:03.333552 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:03.333421 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dt52" podUID="3b54fd06-9913-49b9-ac98-d2916e5f0276" May 28 16:08:03.333552 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:03.333199 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pb7mn" May 28 16:08:03.333552 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:03.333511 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pb7mn" podUID="b113a9e6-a1e9-43bf-a08c-b7e6e466640f" May 28 16:08:03.465390 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:03.465318 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-db77q_d07065a1-27ff-4f50-9b8e-88d35f69bc89/ovn-acl-logging/0.log" May 28 16:08:03.465999 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:03.465610 2566 generic.go:358] "Generic (PLEG): container finished" podID="d07065a1-27ff-4f50-9b8e-88d35f69bc89" containerID="53d45860562985c9dc75a2dd1fc88795049bb8b7a7b4e5bba828282b2d1333a3" exitCode=1 May 28 16:08:03.465999 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:03.465666 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-db77q" event={"ID":"d07065a1-27ff-4f50-9b8e-88d35f69bc89","Type":"ContainerStarted","Data":"f9cade561551a98093df44f011aa09b3d992f036e2608a9094088115581c4c37"} May 28 16:08:03.465999 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:03.465701 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-db77q" event={"ID":"d07065a1-27ff-4f50-9b8e-88d35f69bc89","Type":"ContainerStarted","Data":"7cd36b5e7b02a5ce3fb28c1e99535ecddbb47917fdabb7c4a968c969258c25f5"} May 28 16:08:03.465999 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:03.465715 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-db77q" event={"ID":"d07065a1-27ff-4f50-9b8e-88d35f69bc89","Type":"ContainerStarted","Data":"14ae8a0d674a569be092a1c0a4828a93761ef419337c8c58a48da2ba3a188c34"} May 28 16:08:03.465999 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:03.465727 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-db77q" event={"ID":"d07065a1-27ff-4f50-9b8e-88d35f69bc89","Type":"ContainerStarted","Data":"3ebd08339ae3e0e8092fe77b26c53155a955313b1a68b82b6c81001694139589"} May 28 16:08:03.465999 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:03.465739 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-db77q" event={"ID":"d07065a1-27ff-4f50-9b8e-88d35f69bc89","Type":"ContainerDied","Data":"53d45860562985c9dc75a2dd1fc88795049bb8b7a7b4e5bba828282b2d1333a3"} May 28 16:08:03.465999 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:03.465755 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-db77q" event={"ID":"d07065a1-27ff-4f50-9b8e-88d35f69bc89","Type":"ContainerStarted","Data":"7987885c63634fdc9c733fabd0575d9635cfa5fedcfe1d374b1ac0c57fc0cd20"} May 28 16:08:03.466937 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:03.466910 2566 generic.go:358] "Generic (PLEG): container finished" podID="714f97cf-9a9a-4321-a50d-511802063910" containerID="5fd36fccb7358229c74c4ee08706bc50d1585ef3b3779bd672fee158e352f381" exitCode=0 May 28 16:08:03.467036 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:03.466995 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j8ptw" event={"ID":"714f97cf-9a9a-4321-a50d-511802063910","Type":"ContainerDied","Data":"5fd36fccb7358229c74c4ee08706bc50d1585ef3b3779bd672fee158e352f381"} May 28 16:08:03.468319 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:03.468271 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4k22m" event={"ID":"dc91efd9-d166-4a72-bcdc-3247cc10fce8","Type":"ContainerStarted","Data":"afa70dc60065207c1618888eb473871806bb415a7530b0e13fc79de3e10922fa"} May 28 16:08:03.469462 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:03.469443 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jq2b6" event={"ID":"c0258ed5-55e5-4a30-a658-3e8d61330e0a","Type":"ContainerStarted","Data":"d85b96674386147db106399647ddcd65a17cd9e57af8f0baa4f9543943058ae9"} May 28 16:08:03.470614 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:03.470595 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4swv9" event={"ID":"7acdfd90-ea35-421a-9249-f372fb4ea69c","Type":"ContainerStarted","Data":"875ca595967f2c66c5fc6f280c3b9ff27a91e325750f9dd76022c7a2d40cb96f"} May 28 16:08:03.471708 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:03.471687 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" event={"ID":"8d262a97-a722-48e6-a264-4fdba30f80e8","Type":"ContainerStarted","Data":"1b65e4cc88fd55e58ddbc992011d705d2a4c684306d926b6674570c6638629d7"} May 28 16:08:03.506040 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:03.505979 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-4k22m" podStartSLOduration=4.187768194 podStartE2EDuration="22.505964577s" podCreationTimestamp="2026-05-28 16:07:41 +0000 UTC" firstStartedPulling="2026-05-28 16:07:43.875581702 +0000 UTC m=+3.102612235" lastFinishedPulling="2026-05-28 16:08:02.193778084 +0000 UTC m=+21.420808618" observedRunningTime="2026-05-28 16:08:03.505760246 +0000 UTC m=+22.732790801" watchObservedRunningTime="2026-05-28 16:08:03.505964577 +0000 UTC m=+22.732995132" May 28 16:08:03.520690 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:03.520650 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-jq2b6" podStartSLOduration=4.198459229 podStartE2EDuration="22.52063634s" podCreationTimestamp="2026-05-28 16:07:41 +0000 UTC" firstStartedPulling="2026-05-28 16:07:43.870577131 +0000 UTC m=+3.097607680" lastFinishedPulling="2026-05-28 16:08:02.192754244 +0000 UTC m=+21.419784791" observedRunningTime="2026-05-28 16:08:03.520469422 +0000 UTC m=+22.747499977" watchObservedRunningTime="2026-05-28 16:08:03.52063634 +0000 UTC m=+22.747666896" May 28 16:08:03.535566 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:03.535518 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-vxt4n" podStartSLOduration=4.243448187 podStartE2EDuration="22.535504608s" podCreationTimestamp="2026-05-28 16:07:41 +0000 UTC" firstStartedPulling="2026-05-28 16:07:43.871374138 +0000 UTC m=+3.098404686" lastFinishedPulling="2026-05-28 16:08:02.163430563 +0000 UTC m=+21.390461107" observedRunningTime="2026-05-28 16:08:03.535052312 +0000 UTC m=+22.762082870" watchObservedRunningTime="2026-05-28 16:08:03.535504608 +0000 UTC m=+22.762535163" May 28 16:08:03.659813 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:03.659787 2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" May 28 16:08:04.258637 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:04.258497 2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-28T16:08:03.659805483Z","UUID":"9f006d87-ea4d-4af9-ad6f-bb49ccfddd25","Handler":null,"Name":"","Endpoint":""} May 28 16:08:04.261875 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:04.261849 2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 May 28 16:08:04.262010 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:04.261884 2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock May 28 16:08:04.475420 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:04.475377 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vf6tv" event={"ID":"eb400c72-a841-4a4b-be8b-6684abea58b8","Type":"ContainerStarted","Data":"d5c69187443db2f404311aa663e4fcb1e172a912f042732ea718fde07f8785bd"} May 28 16:08:04.477327 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:04.477294 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4swv9" event={"ID":"7acdfd90-ea35-421a-9249-f372fb4ea69c","Type":"ContainerStarted","Data":"f244f06e71d5c05a9e8d7188f2db52cd7ed9aae855966c02763098261b7a5d33"} May 28 16:08:04.489949 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:04.489902 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vf6tv" podStartSLOduration=9.864881578 podStartE2EDuration="23.489888918s" podCreationTimestamp="2026-05-28 16:07:41 +0000 UTC" firstStartedPulling="2026-05-28 16:07:43.869557091 +0000 UTC m=+3.096587641" lastFinishedPulling="2026-05-28 16:07:57.494564433 +0000 UTC m=+16.721594981" observedRunningTime="2026-05-28 16:08:04.489479282 +0000 UTC m=+23.716509840" watchObservedRunningTime="2026-05-28 16:08:04.489888918 +0000 UTC m=+23.716919474" May 28 16:08:04.767179 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:04.767020 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b113a9e6-a1e9-43bf-a08c-b7e6e466640f-original-pull-secret\") pod \"global-pull-secret-syncer-pb7mn\" (UID: \"b113a9e6-a1e9-43bf-a08c-b7e6e466640f\") " pod="kube-system/global-pull-secret-syncer-pb7mn" May 28 16:08:04.767276 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:04.767155 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 28 16:08:04.767276 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:04.767260 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b113a9e6-a1e9-43bf-a08c-b7e6e466640f-original-pull-secret podName:b113a9e6-a1e9-43bf-a08c-b7e6e466640f nodeName:}" failed. No retries permitted until 2026-05-28 16:08:08.767223432 +0000 UTC m=+27.994253965 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b113a9e6-a1e9-43bf-a08c-b7e6e466640f-original-pull-secret") pod "global-pull-secret-syncer-pb7mn" (UID: "b113a9e6-a1e9-43bf-a08c-b7e6e466640f") : object "kube-system"/"original-pull-secret" not registered May 28 16:08:05.332651 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:05.332611 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pb7mn" May 28 16:08:05.332794 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:05.332663 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dt52" May 28 16:08:05.332794 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:05.332734 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pb7mn" podUID="b113a9e6-a1e9-43bf-a08c-b7e6e466640f" May 28 16:08:05.332879 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:05.332855 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dt52" podUID="3b54fd06-9913-49b9-ac98-d2916e5f0276" May 28 16:08:05.333022 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:05.332892 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nzbn" May 28 16:08:05.333022 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:05.332953 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2nzbn" podUID="f15055af-8047-4a04-b79f-64af8836b946" May 28 16:08:05.480626 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:05.480589 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4swv9" event={"ID":"7acdfd90-ea35-421a-9249-f372fb4ea69c","Type":"ContainerStarted","Data":"d66d9dcb72a6917c1611fe5af20807530d8fb706f8c016724cfa28b8d4a9928d"} May 28 16:08:05.483345 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:05.483326 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-db77q_d07065a1-27ff-4f50-9b8e-88d35f69bc89/ovn-acl-logging/0.log" May 28 16:08:05.483702 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:05.483672 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-db77q" event={"ID":"d07065a1-27ff-4f50-9b8e-88d35f69bc89","Type":"ContainerStarted","Data":"86011b1bad69fdabd09bb5210c5bcc69c535bc076c13dddc1771fb2b5548dcf6"} May 28 16:08:05.496638 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:05.496583 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4swv9" podStartSLOduration=3.659674247 podStartE2EDuration="24.496564573s" podCreationTimestamp="2026-05-28 16:07:41 +0000 UTC" firstStartedPulling="2026-05-28 16:07:43.876799238 +0000 UTC m=+3.103829785" lastFinishedPulling="2026-05-28 16:08:04.713689563 +0000 UTC m=+23.940720111" observedRunningTime="2026-05-28 16:08:05.496251685 +0000 UTC m=+24.723282242" watchObservedRunningTime="2026-05-28 16:08:05.496564573 +0000 UTC m=+24.723595131" May 28 16:08:05.644084 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:05.644035 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-jq2b6" May 28 16:08:05.644790 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:05.644762 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-jq2b6" May 28 16:08:07.332725 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:07.332693 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pb7mn" May 28 16:08:07.332725 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:07.332729 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nzbn" May 28 16:08:07.333362 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:07.332806 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pb7mn" podUID="b113a9e6-a1e9-43bf-a08c-b7e6e466640f" May 28 16:08:07.333362 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:07.332698 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dt52" May 28 16:08:07.333362 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:07.332919 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2nzbn" podUID="f15055af-8047-4a04-b79f-64af8836b946" May 28 16:08:07.333362 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:07.333001 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dt52" podUID="3b54fd06-9913-49b9-ac98-d2916e5f0276" May 28 16:08:08.798226 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:08.797963 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b113a9e6-a1e9-43bf-a08c-b7e6e466640f-original-pull-secret\") pod \"global-pull-secret-syncer-pb7mn\" (UID: \"b113a9e6-a1e9-43bf-a08c-b7e6e466640f\") " pod="kube-system/global-pull-secret-syncer-pb7mn" May 28 16:08:08.798226 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:08.798109 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 28 16:08:08.798226 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:08.798218 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b113a9e6-a1e9-43bf-a08c-b7e6e466640f-original-pull-secret podName:b113a9e6-a1e9-43bf-a08c-b7e6e466640f nodeName:}" failed. No retries permitted until 2026-05-28 16:08:16.798203732 +0000 UTC m=+36.025234266 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b113a9e6-a1e9-43bf-a08c-b7e6e466640f-original-pull-secret") pod "global-pull-secret-syncer-pb7mn" (UID: "b113a9e6-a1e9-43bf-a08c-b7e6e466640f") : object "kube-system"/"original-pull-secret" not registered May 28 16:08:09.333163 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:09.333117 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pb7mn" May 28 16:08:09.333331 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:09.333219 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pb7mn" podUID="b113a9e6-a1e9-43bf-a08c-b7e6e466640f" May 28 16:08:09.333331 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:09.333250 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dt52" May 28 16:08:09.333424 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:09.333263 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nzbn" May 28 16:08:09.333457 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:09.333413 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dt52" podUID="3b54fd06-9913-49b9-ac98-d2916e5f0276" May 28 16:08:09.333524 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:09.333504 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2nzbn" podUID="f15055af-8047-4a04-b79f-64af8836b946" May 28 16:08:09.493219 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:09.493196 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-db77q_d07065a1-27ff-4f50-9b8e-88d35f69bc89/ovn-acl-logging/0.log" May 28 16:08:09.493551 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:09.493526 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-db77q" event={"ID":"d07065a1-27ff-4f50-9b8e-88d35f69bc89","Type":"ContainerStarted","Data":"fcab79f1f59647e7976037d1c5601c0616744e5827780df41a4c63a64996d95d"} May 28 16:08:09.493850 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:09.493814 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:08:09.493850 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:09.493840 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:08:09.494007 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:09.493984 2566 scope.go:117] "RemoveContainer" containerID="53d45860562985c9dc75a2dd1fc88795049bb8b7a7b4e5bba828282b2d1333a3" May 28 16:08:09.495257 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:09.495227 2566 generic.go:358] "Generic (PLEG): container finished" podID="714f97cf-9a9a-4321-a50d-511802063910" containerID="0b7e05d05ae41ee33f4500e7d2abf2ba449cc8bfb49ef221459496dc30e7e97d" exitCode=0 May 28 16:08:09.495349 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:09.495271 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j8ptw" event={"ID":"714f97cf-9a9a-4321-a50d-511802063910","Type":"ContainerDied","Data":"0b7e05d05ae41ee33f4500e7d2abf2ba449cc8bfb49ef221459496dc30e7e97d"} May 28 16:08:09.511407 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:09.511384 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:08:10.500563 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:10.500478 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-db77q_d07065a1-27ff-4f50-9b8e-88d35f69bc89/ovn-acl-logging/0.log" May 28 16:08:10.500982 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:10.500817 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-db77q" event={"ID":"d07065a1-27ff-4f50-9b8e-88d35f69bc89","Type":"ContainerStarted","Data":"10fc5349408cb9c3cab51866cb2fc48af5f9cf41de824603f2f92c686fd06e96"} May 28 16:08:10.501083 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:10.501063 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:08:10.502697 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:10.502661 2566 generic.go:358] "Generic (PLEG): container finished" podID="714f97cf-9a9a-4321-a50d-511802063910" containerID="63c3aec0eddf8ce4a09bb86ddaa5a122f86fa7c88e226c0af2f8122d10fe93c9" exitCode=0 May 28 16:08:10.502803 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:10.502714 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j8ptw" event={"ID":"714f97cf-9a9a-4321-a50d-511802063910","Type":"ContainerDied","Data":"63c3aec0eddf8ce4a09bb86ddaa5a122f86fa7c88e226c0af2f8122d10fe93c9"} May 28 16:08:10.518275 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:10.518249 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:08:10.528347 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:10.528225 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-db77q" podStartSLOduration=11.172366782 podStartE2EDuration="29.528208544s" podCreationTimestamp="2026-05-28 16:07:41 +0000 UTC" firstStartedPulling="2026-05-28 16:07:43.883138706 +0000 UTC m=+3.110169244" lastFinishedPulling="2026-05-28 16:08:02.238980468 +0000 UTC m=+21.466011006" observedRunningTime="2026-05-28 16:08:10.52768714 +0000 UTC m=+29.754717709" watchObservedRunningTime="2026-05-28 16:08:10.528208544 +0000 UTC m=+29.755239102" May 28 16:08:10.538835 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:10.538807 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pb7mn"] May 28 16:08:10.538957 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:10.538945 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pb7mn" May 28 16:08:10.539088 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:10.539060 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pb7mn" podUID="b113a9e6-a1e9-43bf-a08c-b7e6e466640f" May 28 16:08:10.541857 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:10.541837 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6dt52"] May 28 16:08:10.541951 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:10.541940 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dt52" May 28 16:08:10.542037 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:10.542022 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dt52" podUID="3b54fd06-9913-49b9-ac98-d2916e5f0276" May 28 16:08:10.542315 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:10.542275 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2nzbn"] May 28 16:08:10.542412 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:10.542381 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nzbn" May 28 16:08:10.542462 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:10.542448 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2nzbn" podUID="f15055af-8047-4a04-b79f-64af8836b946" May 28 16:08:11.507622 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:11.507587 2566 generic.go:358] "Generic (PLEG): container finished" podID="714f97cf-9a9a-4321-a50d-511802063910" containerID="1511334ae54774f2e83808ff7dbc0ea584dac8091dce0f4012fb49f74b66eb78" exitCode=0 May 28 16:08:11.508108 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:11.507670 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j8ptw" event={"ID":"714f97cf-9a9a-4321-a50d-511802063910","Type":"ContainerDied","Data":"1511334ae54774f2e83808ff7dbc0ea584dac8091dce0f4012fb49f74b66eb78"} May 28 16:08:12.333163 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:12.333126 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nzbn" May 28 16:08:12.333391 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:12.333166 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pb7mn" May 28 16:08:12.333391 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:12.333225 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2nzbn" podUID="f15055af-8047-4a04-b79f-64af8836b946" May 28 16:08:12.333391 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:12.333315 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dt52" May 28 16:08:12.333555 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:12.333417 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dt52" podUID="3b54fd06-9913-49b9-ac98-d2916e5f0276" May 28 16:08:12.333555 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:12.333523 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pb7mn" podUID="b113a9e6-a1e9-43bf-a08c-b7e6e466640f" May 28 16:08:12.700473 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:12.700432 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-jq2b6" May 28 16:08:12.701010 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:12.700583 2566 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" May 28 16:08:12.701095 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:12.701070 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jq2b6" May 28 16:08:14.333018 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:14.332981 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nzbn" May 28 16:08:14.333018 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:14.333013 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dt52" May 28 16:08:14.333516 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:14.332981 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pb7mn" May 28 16:08:14.333516 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:14.333113 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2nzbn" podUID="f15055af-8047-4a04-b79f-64af8836b946" May 28 16:08:14.333516 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:14.333188 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pb7mn" podUID="b113a9e6-a1e9-43bf-a08c-b7e6e466640f" May 28 16:08:14.333516 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:14.333268 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dt52" podUID="3b54fd06-9913-49b9-ac98-d2916e5f0276" May 28 16:08:14.944514 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:14.944481 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b54fd06-9913-49b9-ac98-d2916e5f0276-metrics-certs\") pod \"network-metrics-daemon-6dt52\" (UID: \"3b54fd06-9913-49b9-ac98-d2916e5f0276\") " pod="openshift-multus/network-metrics-daemon-6dt52" May 28 16:08:14.944685 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:14.944654 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 16:08:14.944732 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:14.944725 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b54fd06-9913-49b9-ac98-d2916e5f0276-metrics-certs podName:3b54fd06-9913-49b9-ac98-d2916e5f0276 nodeName:}" failed. No retries permitted until 2026-05-28 16:08:46.944708301 +0000 UTC m=+66.171738840 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b54fd06-9913-49b9-ac98-d2916e5f0276-metrics-certs") pod "network-metrics-daemon-6dt52" (UID: "3b54fd06-9913-49b9-ac98-d2916e5f0276") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 16:08:15.136572 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:15.136491 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-102.ec2.internal" event="NodeReady" May 28 16:08:15.136718 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:15.136650 2566 kubelet_node_status.go:550] "Fast updating node status as it just became ready" May 28 16:08:15.146736 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:15.146707 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ztdpt\" (UniqueName: \"kubernetes.io/projected/f15055af-8047-4a04-b79f-64af8836b946-kube-api-access-ztdpt\") pod \"network-check-target-2nzbn\" (UID: \"f15055af-8047-4a04-b79f-64af8836b946\") " pod="openshift-network-diagnostics/network-check-target-2nzbn" May 28 16:08:15.146920 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:15.146873 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 16:08:15.146920 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:15.146889 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 16:08:15.146920 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:15.146899 2566 projected.go:194] Error preparing data for projected volume kube-api-access-ztdpt for pod openshift-network-diagnostics/network-check-target-2nzbn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 16:08:15.147048 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:15.146951 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f15055af-8047-4a04-b79f-64af8836b946-kube-api-access-ztdpt podName:f15055af-8047-4a04-b79f-64af8836b946 nodeName:}" failed. No retries permitted until 2026-05-28 16:08:47.14693521 +0000 UTC m=+66.373965745 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-ztdpt" (UniqueName: "kubernetes.io/projected/f15055af-8047-4a04-b79f-64af8836b946-kube-api-access-ztdpt") pod "network-check-target-2nzbn" (UID: "f15055af-8047-4a04-b79f-64af8836b946") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 16:08:15.179357 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:15.179327 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-cwtzt"] May 28 16:08:15.181238 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:15.181223 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cwtzt" May 28 16:08:15.182139 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:15.182115 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-zg8g7"] May 28 16:08:15.183603 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:15.183584 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zg8g7" May 28 16:08:15.184043 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:15.184022 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" May 28 16:08:15.184043 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:15.184035 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rfj6w\"" May 28 16:08:15.184140 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:15.184060 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" May 28 16:08:15.185921 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:15.185894 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" May 28 16:08:15.186179 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:15.186146 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" May 28 16:08:15.186336 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:15.186321 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" May 28 16:08:15.186631 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:15.186616 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ckq8r\"" May 28 16:08:15.191038 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:15.191019 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cwtzt"] May 28 16:08:15.202848 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:15.202818 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zg8g7"] May 28 16:08:15.248047 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:15.248010 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57j5m\" (UniqueName: \"kubernetes.io/projected/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-kube-api-access-57j5m\") pod \"dns-default-cwtzt\" (UID: \"8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5\") " pod="openshift-dns/dns-default-cwtzt" May 28 16:08:15.248197 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:15.248068 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b104af3f-b526-4a24-956d-4d85f22a39c1-cert\") pod \"ingress-canary-zg8g7\" (UID: \"b104af3f-b526-4a24-956d-4d85f22a39c1\") " pod="openshift-ingress-canary/ingress-canary-zg8g7" May 28 16:08:15.248197 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:15.248153 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrkzh\" (UniqueName: \"kubernetes.io/projected/b104af3f-b526-4a24-956d-4d85f22a39c1-kube-api-access-rrkzh\") pod \"ingress-canary-zg8g7\" (UID: \"b104af3f-b526-4a24-956d-4d85f22a39c1\") " pod="openshift-ingress-canary/ingress-canary-zg8g7" May 28 16:08:15.248197 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:15.248178 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-config-volume\") pod \"dns-default-cwtzt\" (UID: \"8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5\") " pod="openshift-dns/dns-default-cwtzt" May 28 16:08:15.248197 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:15.248198 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-metrics-tls\") pod \"dns-default-cwtzt\" (UID: \"8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5\") " pod="openshift-dns/dns-default-cwtzt" May 28 16:08:15.248384 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:15.248212 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-tmp-dir\") pod \"dns-default-cwtzt\" (UID: \"8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5\") " pod="openshift-dns/dns-default-cwtzt" May 28 16:08:15.348982 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:15.348948 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b104af3f-b526-4a24-956d-4d85f22a39c1-cert\") pod \"ingress-canary-zg8g7\" (UID: \"b104af3f-b526-4a24-956d-4d85f22a39c1\") " pod="openshift-ingress-canary/ingress-canary-zg8g7" May 28 16:08:15.349604 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:15.349028 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rrkzh\" (UniqueName: \"kubernetes.io/projected/b104af3f-b526-4a24-956d-4d85f22a39c1-kube-api-access-rrkzh\") pod \"ingress-canary-zg8g7\" (UID: \"b104af3f-b526-4a24-956d-4d85f22a39c1\") " pod="openshift-ingress-canary/ingress-canary-zg8g7" May 28 16:08:15.349604 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:15.349055 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-config-volume\") pod \"dns-default-cwtzt\" (UID: \"8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5\") " pod="openshift-dns/dns-default-cwtzt" May 28 16:08:15.349604 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:15.349078 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-metrics-tls\") pod \"dns-default-cwtzt\" (UID: \"8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5\") " pod="openshift-dns/dns-default-cwtzt" May 28 16:08:15.349604 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:15.349103 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-tmp-dir\") pod \"dns-default-cwtzt\" (UID: \"8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5\") " pod="openshift-dns/dns-default-cwtzt" May 28 16:08:15.349604 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:15.349128 2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 16:08:15.349604 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:15.349155 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-57j5m\" (UniqueName: \"kubernetes.io/projected/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-kube-api-access-57j5m\") pod \"dns-default-cwtzt\" (UID: \"8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5\") " pod="openshift-dns/dns-default-cwtzt" May 28 16:08:15.349604 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:15.349210 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b104af3f-b526-4a24-956d-4d85f22a39c1-cert podName:b104af3f-b526-4a24-956d-4d85f22a39c1 nodeName:}" failed. No retries permitted until 2026-05-28 16:08:15.849187479 +0000 UTC m=+35.076218017 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b104af3f-b526-4a24-956d-4d85f22a39c1-cert") pod "ingress-canary-zg8g7" (UID: "b104af3f-b526-4a24-956d-4d85f22a39c1") : secret "canary-serving-cert" not found May 28 16:08:15.349604 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:15.349409 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 16:08:15.349604 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:15.349443 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-metrics-tls podName:8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5 nodeName:}" failed. No retries permitted until 2026-05-28 16:08:15.849433689 +0000 UTC m=+35.076464224 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-metrics-tls") pod "dns-default-cwtzt" (UID: "8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5") : secret "dns-default-metrics-tls" not found May 28 16:08:15.349951 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:15.349647 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-tmp-dir\") pod \"dns-default-cwtzt\" (UID: \"8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5\") " pod="openshift-dns/dns-default-cwtzt" May 28 16:08:15.349951 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:15.349733 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-config-volume\") pod \"dns-default-cwtzt\" (UID: \"8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5\") " pod="openshift-dns/dns-default-cwtzt" May 28 16:08:15.362087 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:15.362061 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-57j5m\" (UniqueName: \"kubernetes.io/projected/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-kube-api-access-57j5m\") pod \"dns-default-cwtzt\" (UID: \"8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5\") " pod="openshift-dns/dns-default-cwtzt" May 28 16:08:15.362200 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:15.362181 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrkzh\" (UniqueName: \"kubernetes.io/projected/b104af3f-b526-4a24-956d-4d85f22a39c1-kube-api-access-rrkzh\") pod \"ingress-canary-zg8g7\" (UID: \"b104af3f-b526-4a24-956d-4d85f22a39c1\") " pod="openshift-ingress-canary/ingress-canary-zg8g7" May 28 16:08:15.852793 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:15.852755 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b104af3f-b526-4a24-956d-4d85f22a39c1-cert\") pod \"ingress-canary-zg8g7\" (UID: \"b104af3f-b526-4a24-956d-4d85f22a39c1\") " pod="openshift-ingress-canary/ingress-canary-zg8g7" May 28 16:08:15.852954 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:15.852820 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-metrics-tls\") pod \"dns-default-cwtzt\" (UID: \"8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5\") " pod="openshift-dns/dns-default-cwtzt" May 28 16:08:15.852954 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:15.852903 2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 16:08:15.852954 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:15.852915 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 16:08:15.853067 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:15.852970 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b104af3f-b526-4a24-956d-4d85f22a39c1-cert podName:b104af3f-b526-4a24-956d-4d85f22a39c1 nodeName:}" failed. No retries permitted until 2026-05-28 16:08:16.852952872 +0000 UTC m=+36.079983415 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b104af3f-b526-4a24-956d-4d85f22a39c1-cert") pod "ingress-canary-zg8g7" (UID: "b104af3f-b526-4a24-956d-4d85f22a39c1") : secret "canary-serving-cert" not found May 28 16:08:15.853067 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:15.852985 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-metrics-tls podName:8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5 nodeName:}" failed. No retries permitted until 2026-05-28 16:08:16.852979131 +0000 UTC m=+36.080009668 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-metrics-tls") pod "dns-default-cwtzt" (UID: "8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5") : secret "dns-default-metrics-tls" not found May 28 16:08:16.333034 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:16.333000 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pb7mn" May 28 16:08:16.333234 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:16.333000 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nzbn" May 28 16:08:16.333316 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:16.333002 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dt52" May 28 16:08:16.336535 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:16.336502 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" May 28 16:08:16.336627 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:16.336547 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xz8bh\"" May 28 16:08:16.337687 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:16.337662 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" May 28 16:08:16.337687 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:16.337668 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" May 28 16:08:16.337824 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:16.337715 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" May 28 16:08:16.337824 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:16.337683 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qjc5m\"" May 28 16:08:16.860720 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:16.860659 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b113a9e6-a1e9-43bf-a08c-b7e6e466640f-original-pull-secret\") pod \"global-pull-secret-syncer-pb7mn\" (UID: \"b113a9e6-a1e9-43bf-a08c-b7e6e466640f\") " pod="kube-system/global-pull-secret-syncer-pb7mn" May 28 16:08:16.861176 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:16.860726 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-metrics-tls\") pod \"dns-default-cwtzt\" (UID: \"8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5\") " pod="openshift-dns/dns-default-cwtzt" May 28 16:08:16.861176 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:16.860867 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 16:08:16.861176 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:16.860933 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-metrics-tls podName:8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5 nodeName:}" failed. No retries permitted until 2026-05-28 16:08:18.860913343 +0000 UTC m=+38.087943899 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-metrics-tls") pod "dns-default-cwtzt" (UID: "8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5") : secret "dns-default-metrics-tls" not found May 28 16:08:16.861176 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:16.860929 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b104af3f-b526-4a24-956d-4d85f22a39c1-cert\") pod \"ingress-canary-zg8g7\" (UID: \"b104af3f-b526-4a24-956d-4d85f22a39c1\") " pod="openshift-ingress-canary/ingress-canary-zg8g7" May 28 16:08:16.861176 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:16.861057 2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 16:08:16.861176 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:16.861111 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b104af3f-b526-4a24-956d-4d85f22a39c1-cert podName:b104af3f-b526-4a24-956d-4d85f22a39c1 nodeName:}" failed. No retries permitted until 2026-05-28 16:08:18.861096969 +0000 UTC m=+38.088127514 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b104af3f-b526-4a24-956d-4d85f22a39c1-cert") pod "ingress-canary-zg8g7" (UID: "b104af3f-b526-4a24-956d-4d85f22a39c1") : secret "canary-serving-cert" not found May 28 16:08:16.863820 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:16.863794 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b113a9e6-a1e9-43bf-a08c-b7e6e466640f-original-pull-secret\") pod \"global-pull-secret-syncer-pb7mn\" (UID: \"b113a9e6-a1e9-43bf-a08c-b7e6e466640f\") " pod="kube-system/global-pull-secret-syncer-pb7mn" May 28 16:08:16.944120 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:16.944083 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pb7mn" May 28 16:08:18.312413 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:18.312231 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pb7mn"] May 28 16:08:18.317234 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:08:18.317201 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb113a9e6_a1e9_43bf_a08c_b7e6e466640f.slice/crio-b043a38382cda34faf5ac8b9bc442280301c85c1c8a4305ae5d140b7ee9c66ee WatchSource:0}: Error finding container b043a38382cda34faf5ac8b9bc442280301c85c1c8a4305ae5d140b7ee9c66ee: Status 404 returned error can't find the container with id b043a38382cda34faf5ac8b9bc442280301c85c1c8a4305ae5d140b7ee9c66ee May 28 16:08:18.523643 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:18.523601 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j8ptw" event={"ID":"714f97cf-9a9a-4321-a50d-511802063910","Type":"ContainerStarted","Data":"5ba4444e62bd28b425ea700aff9d1b9ecef3014b2f08ddbb8c591630cbfa4dba"} May 28 16:08:18.524715 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:18.524677 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pb7mn" event={"ID":"b113a9e6-a1e9-43bf-a08c-b7e6e466640f","Type":"ContainerStarted","Data":"b043a38382cda34faf5ac8b9bc442280301c85c1c8a4305ae5d140b7ee9c66ee"} May 28 16:08:18.877929 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:18.877849 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b104af3f-b526-4a24-956d-4d85f22a39c1-cert\") pod \"ingress-canary-zg8g7\" (UID: \"b104af3f-b526-4a24-956d-4d85f22a39c1\") " pod="openshift-ingress-canary/ingress-canary-zg8g7" May 28 16:08:18.877929 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:18.877909 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-metrics-tls\") pod \"dns-default-cwtzt\" (UID: \"8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5\") " pod="openshift-dns/dns-default-cwtzt" May 28 16:08:18.878093 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:18.878003 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 16:08:18.878093 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:18.878000 2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 16:08:18.878093 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:18.878067 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-metrics-tls podName:8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5 nodeName:}" failed. No retries permitted until 2026-05-28 16:08:22.878053516 +0000 UTC m=+42.105084056 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-metrics-tls") pod "dns-default-cwtzt" (UID: "8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5") : secret "dns-default-metrics-tls" not found May 28 16:08:18.878186 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:18.878106 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b104af3f-b526-4a24-956d-4d85f22a39c1-cert podName:b104af3f-b526-4a24-956d-4d85f22a39c1 nodeName:}" failed. No retries permitted until 2026-05-28 16:08:22.87809202 +0000 UTC m=+42.105122555 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b104af3f-b526-4a24-956d-4d85f22a39c1-cert") pod "ingress-canary-zg8g7" (UID: "b104af3f-b526-4a24-956d-4d85f22a39c1") : secret "canary-serving-cert" not found May 28 16:08:19.537044 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:19.537004 2566 generic.go:358] "Generic (PLEG): container finished" podID="714f97cf-9a9a-4321-a50d-511802063910" containerID="5ba4444e62bd28b425ea700aff9d1b9ecef3014b2f08ddbb8c591630cbfa4dba" exitCode=0 May 28 16:08:19.537620 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:19.537075 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j8ptw" event={"ID":"714f97cf-9a9a-4321-a50d-511802063910","Type":"ContainerDied","Data":"5ba4444e62bd28b425ea700aff9d1b9ecef3014b2f08ddbb8c591630cbfa4dba"} May 28 16:08:20.542572 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:20.542378 2566 generic.go:358] "Generic (PLEG): container finished" podID="714f97cf-9a9a-4321-a50d-511802063910" containerID="9a68388986860ab89c61b4f159df79a68032b9e8b3c17fe17fed7a492fe12e08" exitCode=0 May 28 16:08:20.542971 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:20.542450 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j8ptw" event={"ID":"714f97cf-9a9a-4321-a50d-511802063910","Type":"ContainerDied","Data":"9a68388986860ab89c61b4f159df79a68032b9e8b3c17fe17fed7a492fe12e08"} May 28 16:08:21.547731 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:21.547695 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j8ptw" event={"ID":"714f97cf-9a9a-4321-a50d-511802063910","Type":"ContainerStarted","Data":"1c56574116c43ac306ae40e019217051b225818c81e60c21b12b925ea854b463"} May 28 16:08:21.572150 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:21.572091 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-j8ptw" podStartSLOduration=6.169775592 podStartE2EDuration="40.572076238s" podCreationTimestamp="2026-05-28 16:07:41 +0000 UTC" firstStartedPulling="2026-05-28 16:07:43.87814738 +0000 UTC m=+3.105177933" lastFinishedPulling="2026-05-28 16:08:18.280448038 +0000 UTC m=+37.507478579" observedRunningTime="2026-05-28 16:08:21.571585806 +0000 UTC m=+40.798616365" watchObservedRunningTime="2026-05-28 16:08:21.572076238 +0000 UTC m=+40.799106793" May 28 16:08:22.909384 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:22.909349 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-metrics-tls\") pod \"dns-default-cwtzt\" (UID: \"8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5\") " pod="openshift-dns/dns-default-cwtzt" May 28 16:08:22.909775 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:22.909412 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b104af3f-b526-4a24-956d-4d85f22a39c1-cert\") pod \"ingress-canary-zg8g7\" (UID: \"b104af3f-b526-4a24-956d-4d85f22a39c1\") " pod="openshift-ingress-canary/ingress-canary-zg8g7" May 28 16:08:22.909775 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:22.909505 2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 16:08:22.909775 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:22.909508 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 16:08:22.909775 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:22.909558 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b104af3f-b526-4a24-956d-4d85f22a39c1-cert podName:b104af3f-b526-4a24-956d-4d85f22a39c1 nodeName:}" failed. No retries permitted until 2026-05-28 16:08:30.909543261 +0000 UTC m=+50.136573795 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b104af3f-b526-4a24-956d-4d85f22a39c1-cert") pod "ingress-canary-zg8g7" (UID: "b104af3f-b526-4a24-956d-4d85f22a39c1") : secret "canary-serving-cert" not found May 28 16:08:22.909775 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:22.909571 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-metrics-tls podName:8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5 nodeName:}" failed. No retries permitted until 2026-05-28 16:08:30.909565512 +0000 UTC m=+50.136596046 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-metrics-tls") pod "dns-default-cwtzt" (UID: "8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5") : secret "dns-default-metrics-tls" not found May 28 16:08:23.552556 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:23.552516 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pb7mn" event={"ID":"b113a9e6-a1e9-43bf-a08c-b7e6e466640f","Type":"ContainerStarted","Data":"bac4cd7dc11825c116b39a5e9a437494516fec0be7598421d492e1f1a685beff"} May 28 16:08:23.566850 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:23.566803 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-pb7mn" podStartSLOduration=18.760472767 podStartE2EDuration="23.566790412s" podCreationTimestamp="2026-05-28 16:08:00 +0000 UTC" firstStartedPulling="2026-05-28 16:08:18.319260677 +0000 UTC m=+37.546291213" lastFinishedPulling="2026-05-28 16:08:23.125578312 +0000 UTC m=+42.352608858" observedRunningTime="2026-05-28 16:08:23.566421737 +0000 UTC m=+42.793452295" watchObservedRunningTime="2026-05-28 16:08:23.566790412 +0000 UTC m=+42.793820967" May 28 16:08:30.968488 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:30.968447 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-metrics-tls\") pod \"dns-default-cwtzt\" (UID: \"8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5\") " pod="openshift-dns/dns-default-cwtzt" May 28 16:08:30.968889 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:30.968517 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b104af3f-b526-4a24-956d-4d85f22a39c1-cert\") pod \"ingress-canary-zg8g7\" (UID: \"b104af3f-b526-4a24-956d-4d85f22a39c1\") " pod="openshift-ingress-canary/ingress-canary-zg8g7" May 28 16:08:30.968889 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:30.968639 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 16:08:30.968889 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:30.968719 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-metrics-tls podName:8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5 nodeName:}" failed. No retries permitted until 2026-05-28 16:08:46.968701371 +0000 UTC m=+66.195731905 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-metrics-tls") pod "dns-default-cwtzt" (UID: "8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5") : secret "dns-default-metrics-tls" not found May 28 16:08:30.968889 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:30.968639 2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 16:08:30.968889 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:30.968774 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b104af3f-b526-4a24-956d-4d85f22a39c1-cert podName:b104af3f-b526-4a24-956d-4d85f22a39c1 nodeName:}" failed. No retries permitted until 2026-05-28 16:08:46.968761969 +0000 UTC m=+66.195792503 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b104af3f-b526-4a24-956d-4d85f22a39c1-cert") pod "ingress-canary-zg8g7" (UID: "b104af3f-b526-4a24-956d-4d85f22a39c1") : secret "canary-serving-cert" not found May 28 16:08:38.400883 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:38.400845 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj"] May 28 16:08:38.404565 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:38.404549 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj" May 28 16:08:38.407360 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:38.407333 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" May 28 16:08:38.407472 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:38.407404 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" May 28 16:08:38.407631 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:38.407614 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" May 28 16:08:38.408817 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:38.408799 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" May 28 16:08:38.408933 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:38.408917 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" May 28 16:08:38.409040 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:38.409025 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" May 28 16:08:38.409104 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:38.409028 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" May 28 16:08:38.414090 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:38.414067 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj"] May 28 16:08:38.523502 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:38.523467 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7l85\" (UniqueName: \"kubernetes.io/projected/9f194b55-2d06-4c01-9d01-65d0587862be-kube-api-access-b7l85\") pod \"cluster-proxy-proxy-agent-558455b789-7s7hj\" (UID: \"9f194b55-2d06-4c01-9d01-65d0587862be\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj" May 28 16:08:38.523502 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:38.523503 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/9f194b55-2d06-4c01-9d01-65d0587862be-hub\") pod \"cluster-proxy-proxy-agent-558455b789-7s7hj\" (UID: \"9f194b55-2d06-4c01-9d01-65d0587862be\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj" May 28 16:08:38.523720 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:38.523526 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9f194b55-2d06-4c01-9d01-65d0587862be-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-558455b789-7s7hj\" (UID: \"9f194b55-2d06-4c01-9d01-65d0587862be\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj" May 28 16:08:38.523720 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:38.523544 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/9f194b55-2d06-4c01-9d01-65d0587862be-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-558455b789-7s7hj\" (UID: \"9f194b55-2d06-4c01-9d01-65d0587862be\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj" May 28 16:08:38.523720 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:38.523566 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/9f194b55-2d06-4c01-9d01-65d0587862be-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-558455b789-7s7hj\" (UID: \"9f194b55-2d06-4c01-9d01-65d0587862be\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj" May 28 16:08:38.523720 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:38.523615 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/9f194b55-2d06-4c01-9d01-65d0587862be-ca\") pod \"cluster-proxy-proxy-agent-558455b789-7s7hj\" (UID: \"9f194b55-2d06-4c01-9d01-65d0587862be\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj" May 28 16:08:38.624819 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:38.624787 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b7l85\" (UniqueName: \"kubernetes.io/projected/9f194b55-2d06-4c01-9d01-65d0587862be-kube-api-access-b7l85\") pod \"cluster-proxy-proxy-agent-558455b789-7s7hj\" (UID: \"9f194b55-2d06-4c01-9d01-65d0587862be\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj" May 28 16:08:38.624819 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:38.624827 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/9f194b55-2d06-4c01-9d01-65d0587862be-hub\") pod \"cluster-proxy-proxy-agent-558455b789-7s7hj\" (UID: \"9f194b55-2d06-4c01-9d01-65d0587862be\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj" May 28 16:08:38.625086 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:38.624849 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9f194b55-2d06-4c01-9d01-65d0587862be-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-558455b789-7s7hj\" (UID: \"9f194b55-2d06-4c01-9d01-65d0587862be\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj" May 28 16:08:38.625086 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:38.624865 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/9f194b55-2d06-4c01-9d01-65d0587862be-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-558455b789-7s7hj\" (UID: \"9f194b55-2d06-4c01-9d01-65d0587862be\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj" May 28 16:08:38.625086 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:38.624884 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/9f194b55-2d06-4c01-9d01-65d0587862be-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-558455b789-7s7hj\" (UID: \"9f194b55-2d06-4c01-9d01-65d0587862be\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj" May 28 16:08:38.625086 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:38.624929 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/9f194b55-2d06-4c01-9d01-65d0587862be-ca\") pod \"cluster-proxy-proxy-agent-558455b789-7s7hj\" (UID: \"9f194b55-2d06-4c01-9d01-65d0587862be\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj" May 28 16:08:38.625700 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:38.625675 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/9f194b55-2d06-4c01-9d01-65d0587862be-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-558455b789-7s7hj\" (UID: \"9f194b55-2d06-4c01-9d01-65d0587862be\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj" May 28 16:08:38.629047 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:38.629016 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/9f194b55-2d06-4c01-9d01-65d0587862be-ca\") pod \"cluster-proxy-proxy-agent-558455b789-7s7hj\" (UID: \"9f194b55-2d06-4c01-9d01-65d0587862be\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj" May 28 16:08:38.629170 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:38.629101 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/9f194b55-2d06-4c01-9d01-65d0587862be-hub\") pod \"cluster-proxy-proxy-agent-558455b789-7s7hj\" (UID: \"9f194b55-2d06-4c01-9d01-65d0587862be\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj" May 28 16:08:38.629170 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:38.629131 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9f194b55-2d06-4c01-9d01-65d0587862be-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-558455b789-7s7hj\" (UID: \"9f194b55-2d06-4c01-9d01-65d0587862be\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj" May 28 16:08:38.629246 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:38.629190 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/9f194b55-2d06-4c01-9d01-65d0587862be-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-558455b789-7s7hj\" (UID: \"9f194b55-2d06-4c01-9d01-65d0587862be\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj" May 28 16:08:38.633173 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:38.633151 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7l85\" (UniqueName: \"kubernetes.io/projected/9f194b55-2d06-4c01-9d01-65d0587862be-kube-api-access-b7l85\") pod \"cluster-proxy-proxy-agent-558455b789-7s7hj\" (UID: \"9f194b55-2d06-4c01-9d01-65d0587862be\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj" May 28 16:08:38.721580 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:38.721494 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj" May 28 16:08:38.855428 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:38.855397 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj"] May 28 16:08:38.858573 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:08:38.858546 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f194b55_2d06_4c01_9d01_65d0587862be.slice/crio-488b590484d4c2c6677edc4f23eeef9226f75fb075e0db6b32833723713c65bd WatchSource:0}: Error finding container 488b590484d4c2c6677edc4f23eeef9226f75fb075e0db6b32833723713c65bd: Status 404 returned error can't find the container with id 488b590484d4c2c6677edc4f23eeef9226f75fb075e0db6b32833723713c65bd May 28 16:08:39.588079 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:39.588037 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj" event={"ID":"9f194b55-2d06-4c01-9d01-65d0587862be","Type":"ContainerStarted","Data":"488b590484d4c2c6677edc4f23eeef9226f75fb075e0db6b32833723713c65bd"} May 28 16:08:42.520645 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:42.520618 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-db77q" May 28 16:08:45.601094 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:45.601052 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj" event={"ID":"9f194b55-2d06-4c01-9d01-65d0587862be","Type":"ContainerStarted","Data":"a26abd575e5c79fd3e19fbace0fbc551ef4ceab3122ca637c598571bdfad50c9"} May 28 16:08:46.985826 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:46.985782 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-metrics-tls\") pod \"dns-default-cwtzt\" (UID: \"8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5\") " pod="openshift-dns/dns-default-cwtzt" May 28 16:08:46.985826 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:46.985831 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b54fd06-9913-49b9-ac98-d2916e5f0276-metrics-certs\") pod \"network-metrics-daemon-6dt52\" (UID: \"3b54fd06-9913-49b9-ac98-d2916e5f0276\") " pod="openshift-multus/network-metrics-daemon-6dt52" May 28 16:08:46.986344 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:46.985885 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b104af3f-b526-4a24-956d-4d85f22a39c1-cert\") pod \"ingress-canary-zg8g7\" (UID: \"b104af3f-b526-4a24-956d-4d85f22a39c1\") " pod="openshift-ingress-canary/ingress-canary-zg8g7" May 28 16:08:46.986344 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:46.985925 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 16:08:46.986344 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:46.985989 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-metrics-tls podName:8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5 nodeName:}" failed. No retries permitted until 2026-05-28 16:09:18.985972842 +0000 UTC m=+98.213003379 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-metrics-tls") pod "dns-default-cwtzt" (UID: "8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5") : secret "dns-default-metrics-tls" not found May 28 16:08:46.986344 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:46.985993 2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 16:08:46.986344 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:46.986045 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b104af3f-b526-4a24-956d-4d85f22a39c1-cert podName:b104af3f-b526-4a24-956d-4d85f22a39c1 nodeName:}" failed. No retries permitted until 2026-05-28 16:09:18.986028674 +0000 UTC m=+98.213059209 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b104af3f-b526-4a24-956d-4d85f22a39c1-cert") pod "ingress-canary-zg8g7" (UID: "b104af3f-b526-4a24-956d-4d85f22a39c1") : secret "canary-serving-cert" not found May 28 16:08:46.988523 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:46.988502 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" May 28 16:08:46.996615 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:46.996597 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found May 28 16:08:46.996674 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:08:46.996658 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b54fd06-9913-49b9-ac98-d2916e5f0276-metrics-certs podName:3b54fd06-9913-49b9-ac98-d2916e5f0276 nodeName:}" failed. No retries permitted until 2026-05-28 16:09:50.996638393 +0000 UTC m=+130.223668932 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b54fd06-9913-49b9-ac98-d2916e5f0276-metrics-certs") pod "network-metrics-daemon-6dt52" (UID: "3b54fd06-9913-49b9-ac98-d2916e5f0276") : secret "metrics-daemon-secret" not found May 28 16:08:47.187397 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:47.187349 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ztdpt\" (UniqueName: \"kubernetes.io/projected/f15055af-8047-4a04-b79f-64af8836b946-kube-api-access-ztdpt\") pod \"network-check-target-2nzbn\" (UID: \"f15055af-8047-4a04-b79f-64af8836b946\") " pod="openshift-network-diagnostics/network-check-target-2nzbn" May 28 16:08:47.190227 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:47.190195 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" May 28 16:08:47.200423 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:47.200391 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" May 28 16:08:47.211463 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:47.211434 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztdpt\" (UniqueName: \"kubernetes.io/projected/f15055af-8047-4a04-b79f-64af8836b946-kube-api-access-ztdpt\") pod \"network-check-target-2nzbn\" (UID: \"f15055af-8047-4a04-b79f-64af8836b946\") " pod="openshift-network-diagnostics/network-check-target-2nzbn" May 28 16:08:47.253042 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:47.252967 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qjc5m\"" May 28 16:08:47.260260 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:47.260224 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nzbn" May 28 16:08:47.405003 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:47.404930 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2nzbn"] May 28 16:08:47.409132 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:08:47.409097 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf15055af_8047_4a04_b79f_64af8836b946.slice/crio-4f3d700926ac10813a6bddcafc67f2c3e844b232ede1663b7b62991dfe2e7ad3 WatchSource:0}: Error finding container 4f3d700926ac10813a6bddcafc67f2c3e844b232ede1663b7b62991dfe2e7ad3: Status 404 returned error can't find the container with id 4f3d700926ac10813a6bddcafc67f2c3e844b232ede1663b7b62991dfe2e7ad3 May 28 16:08:47.607641 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:47.607549 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2nzbn" event={"ID":"f15055af-8047-4a04-b79f-64af8836b946","Type":"ContainerStarted","Data":"4f3d700926ac10813a6bddcafc67f2c3e844b232ede1663b7b62991dfe2e7ad3"} May 28 16:08:49.614337 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:49.614270 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj" event={"ID":"9f194b55-2d06-4c01-9d01-65d0587862be","Type":"ContainerStarted","Data":"8fa59934f8ae209a0a182a5d4c676269edbcff9c1e04f71be051efdcc663e78b"} May 28 16:08:49.614337 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:49.614343 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj" event={"ID":"9f194b55-2d06-4c01-9d01-65d0587862be","Type":"ContainerStarted","Data":"10c49389272a90e0d790250c0045cb18410a01c392d53824eca44ff8f7249090"} May 28 16:08:49.631733 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:49.631604 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj" podStartSLOduration=1.807030455 podStartE2EDuration="11.631588033s" podCreationTimestamp="2026-05-28 16:08:38 +0000 UTC" firstStartedPulling="2026-05-28 16:08:38.860355146 +0000 UTC m=+58.087385681" lastFinishedPulling="2026-05-28 16:08:48.684912716 +0000 UTC m=+67.911943259" observedRunningTime="2026-05-28 16:08:49.631442878 +0000 UTC m=+68.858473433" watchObservedRunningTime="2026-05-28 16:08:49.631588033 +0000 UTC m=+68.858618621" May 28 16:08:51.619238 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:51.619145 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2nzbn" event={"ID":"f15055af-8047-4a04-b79f-64af8836b946","Type":"ContainerStarted","Data":"b8a31ad4b11dd612f5df2d7f9116fb2fb1e9ac30f23faee3a902d29cc45a0d2b"} May 28 16:08:51.619679 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:08:51.619348 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2nzbn" May 28 16:09:12.906021 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:12.905866 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2nzbn" podStartSLOduration=88.038532841 podStartE2EDuration="1m31.905851953s" podCreationTimestamp="2026-05-28 16:07:41 +0000 UTC" firstStartedPulling="2026-05-28 16:08:47.411317585 +0000 UTC m=+66.638348127" lastFinishedPulling="2026-05-28 16:08:51.278636692 +0000 UTC m=+70.505667239" observedRunningTime="2026-05-28 16:08:51.633601819 +0000 UTC m=+70.860632374" watchObservedRunningTime="2026-05-28 16:09:12.905851953 +0000 UTC m=+92.132882525" May 28 16:09:12.906498 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:12.906470 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-qc7zn"] May 28 16:09:12.914929 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:12.914758 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-qc7zn" May 28 16:09:12.915913 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:12.915882 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-qc7zn"] May 28 16:09:12.917380 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:12.917357 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" May 28 16:09:12.917483 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:12.917361 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" May 28 16:09:12.917483 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:12.917442 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" May 28 16:09:12.918472 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:12.918453 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-6q44c\"" May 28 16:09:12.918472 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:12.918468 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" May 28 16:09:12.970426 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:12.970384 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3e7f8fbd-4455-4478-8f61-8ded41ddd032-serving-cert\") pod \"service-ca-operator-57b5b68b8-qc7zn\" (UID: \"3e7f8fbd-4455-4478-8f61-8ded41ddd032\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-qc7zn" May 28 16:09:12.970611 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:12.970452 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3e7f8fbd-4455-4478-8f61-8ded41ddd032-config\") pod \"service-ca-operator-57b5b68b8-qc7zn\" (UID: \"3e7f8fbd-4455-4478-8f61-8ded41ddd032\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-qc7zn" May 28 16:09:12.970611 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:12.970540 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgwhc\" (UniqueName: \"kubernetes.io/projected/3e7f8fbd-4455-4478-8f61-8ded41ddd032-kube-api-access-xgwhc\") pod \"service-ca-operator-57b5b68b8-qc7zn\" (UID: \"3e7f8fbd-4455-4478-8f61-8ded41ddd032\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-qc7zn" May 28 16:09:13.071834 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:13.071796 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3e7f8fbd-4455-4478-8f61-8ded41ddd032-config\") pod \"service-ca-operator-57b5b68b8-qc7zn\" (UID: \"3e7f8fbd-4455-4478-8f61-8ded41ddd032\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-qc7zn" May 28 16:09:13.071952 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:13.071854 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xgwhc\" (UniqueName: \"kubernetes.io/projected/3e7f8fbd-4455-4478-8f61-8ded41ddd032-kube-api-access-xgwhc\") pod \"service-ca-operator-57b5b68b8-qc7zn\" (UID: \"3e7f8fbd-4455-4478-8f61-8ded41ddd032\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-qc7zn" May 28 16:09:13.072042 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:13.072020 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3e7f8fbd-4455-4478-8f61-8ded41ddd032-serving-cert\") pod \"service-ca-operator-57b5b68b8-qc7zn\" (UID: \"3e7f8fbd-4455-4478-8f61-8ded41ddd032\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-qc7zn" May 28 16:09:13.072421 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:13.072399 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3e7f8fbd-4455-4478-8f61-8ded41ddd032-config\") pod \"service-ca-operator-57b5b68b8-qc7zn\" (UID: \"3e7f8fbd-4455-4478-8f61-8ded41ddd032\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-qc7zn" May 28 16:09:13.074393 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:13.074376 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3e7f8fbd-4455-4478-8f61-8ded41ddd032-serving-cert\") pod \"service-ca-operator-57b5b68b8-qc7zn\" (UID: \"3e7f8fbd-4455-4478-8f61-8ded41ddd032\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-qc7zn" May 28 16:09:13.080006 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:13.079984 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgwhc\" (UniqueName: \"kubernetes.io/projected/3e7f8fbd-4455-4478-8f61-8ded41ddd032-kube-api-access-xgwhc\") pod \"service-ca-operator-57b5b68b8-qc7zn\" (UID: \"3e7f8fbd-4455-4478-8f61-8ded41ddd032\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-qc7zn" May 28 16:09:13.224346 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:13.224315 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-qc7zn" May 28 16:09:13.353249 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:13.353218 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-qc7zn"] May 28 16:09:13.356702 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:09:13.356666 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e7f8fbd_4455_4478_8f61_8ded41ddd032.slice/crio-12953f3b38142bfb20e3f95599e22338b8990f61ccaf3bc3a6e6f6a3cabefe90 WatchSource:0}: Error finding container 12953f3b38142bfb20e3f95599e22338b8990f61ccaf3bc3a6e6f6a3cabefe90: Status 404 returned error can't find the container with id 12953f3b38142bfb20e3f95599e22338b8990f61ccaf3bc3a6e6f6a3cabefe90 May 28 16:09:13.662321 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:13.662215 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-qc7zn" event={"ID":"3e7f8fbd-4455-4478-8f61-8ded41ddd032","Type":"ContainerStarted","Data":"12953f3b38142bfb20e3f95599e22338b8990f61ccaf3bc3a6e6f6a3cabefe90"} May 28 16:09:15.844998 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:15.844971 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4k22m_dc91efd9-d166-4a72-bcdc-3247cc10fce8/dns-node-resolver/0.log" May 28 16:09:16.644202 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:16.644172 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-z55zs_5ae4021e-a738-403c-9143-976b0c75fca8/node-ca/0.log" May 28 16:09:16.676409 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:16.676369 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-qc7zn" event={"ID":"3e7f8fbd-4455-4478-8f61-8ded41ddd032","Type":"ContainerStarted","Data":"d69603f8367c0871108a1f991f86cd27c631bf4eda0803d9596bd8d999fd5e33"} May 28 16:09:16.692017 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:16.691973 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-qc7zn" podStartSLOduration=1.9975324030000001 podStartE2EDuration="4.691960631s" podCreationTimestamp="2026-05-28 16:09:12 +0000 UTC" firstStartedPulling="2026-05-28 16:09:13.358468169 +0000 UTC m=+92.585498703" lastFinishedPulling="2026-05-28 16:09:16.052896397 +0000 UTC m=+95.279926931" observedRunningTime="2026-05-28 16:09:16.69083103 +0000 UTC m=+95.917861586" watchObservedRunningTime="2026-05-28 16:09:16.691960631 +0000 UTC m=+95.918991186" May 28 16:09:19.019390 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:19.019344 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b104af3f-b526-4a24-956d-4d85f22a39c1-cert\") pod \"ingress-canary-zg8g7\" (UID: \"b104af3f-b526-4a24-956d-4d85f22a39c1\") " pod="openshift-ingress-canary/ingress-canary-zg8g7" May 28 16:09:19.019849 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:19.019403 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-metrics-tls\") pod \"dns-default-cwtzt\" (UID: \"8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5\") " pod="openshift-dns/dns-default-cwtzt" May 28 16:09:19.019849 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:09:19.019510 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 16:09:19.019849 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:09:19.019516 2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 16:09:19.019849 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:09:19.019564 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-metrics-tls podName:8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5 nodeName:}" failed. No retries permitted until 2026-05-28 16:10:23.01955105 +0000 UTC m=+162.246581584 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-metrics-tls") pod "dns-default-cwtzt" (UID: "8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5") : secret "dns-default-metrics-tls" not found May 28 16:09:19.019849 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:09:19.019579 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b104af3f-b526-4a24-956d-4d85f22a39c1-cert podName:b104af3f-b526-4a24-956d-4d85f22a39c1 nodeName:}" failed. No retries permitted until 2026-05-28 16:10:23.019571973 +0000 UTC m=+162.246602507 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b104af3f-b526-4a24-956d-4d85f22a39c1-cert") pod "ingress-canary-zg8g7" (UID: "b104af3f-b526-4a24-956d-4d85f22a39c1") : secret "canary-serving-cert" not found May 28 16:09:19.253531 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:19.253497 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7cd68cf8cd-h6sdf"] May 28 16:09:19.302020 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:19.301941 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7cd68cf8cd-h6sdf"] May 28 16:09:19.302157 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:19.302058 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" May 28 16:09:19.304488 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:19.304466 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" May 28 16:09:19.304627 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:19.304494 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" May 28 16:09:19.304627 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:19.304556 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" May 28 16:09:19.304706 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:19.304699 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gp2cl\"" May 28 16:09:19.319599 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:19.319575 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" May 28 16:09:19.421896 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:19.421864 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4c87c5ee-321e-4fa5-a36a-550f442768f0-ca-trust-extracted\") pod \"image-registry-7cd68cf8cd-h6sdf\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" May 28 16:09:19.422072 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:19.421916 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4c87c5ee-321e-4fa5-a36a-550f442768f0-installation-pull-secrets\") pod \"image-registry-7cd68cf8cd-h6sdf\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" May 28 16:09:19.422072 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:19.421946 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4c87c5ee-321e-4fa5-a36a-550f442768f0-registry-tls\") pod \"image-registry-7cd68cf8cd-h6sdf\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" May 28 16:09:19.422072 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:19.421996 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4c87c5ee-321e-4fa5-a36a-550f442768f0-registry-certificates\") pod \"image-registry-7cd68cf8cd-h6sdf\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" May 28 16:09:19.422072 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:19.422027 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4c87c5ee-321e-4fa5-a36a-550f442768f0-bound-sa-token\") pod \"image-registry-7cd68cf8cd-h6sdf\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" May 28 16:09:19.422072 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:19.422045 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5tg5\" (UniqueName: \"kubernetes.io/projected/4c87c5ee-321e-4fa5-a36a-550f442768f0-kube-api-access-k5tg5\") pod \"image-registry-7cd68cf8cd-h6sdf\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" May 28 16:09:19.422231 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:19.422116 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4c87c5ee-321e-4fa5-a36a-550f442768f0-image-registry-private-configuration\") pod \"image-registry-7cd68cf8cd-h6sdf\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" May 28 16:09:19.422231 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:19.422143 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4c87c5ee-321e-4fa5-a36a-550f442768f0-trusted-ca\") pod \"image-registry-7cd68cf8cd-h6sdf\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" May 28 16:09:19.522949 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:19.522915 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4c87c5ee-321e-4fa5-a36a-550f442768f0-bound-sa-token\") pod \"image-registry-7cd68cf8cd-h6sdf\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" May 28 16:09:19.522949 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:19.522951 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k5tg5\" (UniqueName: \"kubernetes.io/projected/4c87c5ee-321e-4fa5-a36a-550f442768f0-kube-api-access-k5tg5\") pod \"image-registry-7cd68cf8cd-h6sdf\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" May 28 16:09:19.523168 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:19.523004 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4c87c5ee-321e-4fa5-a36a-550f442768f0-image-registry-private-configuration\") pod \"image-registry-7cd68cf8cd-h6sdf\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" May 28 16:09:19.523168 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:19.523026 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4c87c5ee-321e-4fa5-a36a-550f442768f0-trusted-ca\") pod \"image-registry-7cd68cf8cd-h6sdf\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" May 28 16:09:19.523168 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:19.523045 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4c87c5ee-321e-4fa5-a36a-550f442768f0-ca-trust-extracted\") pod \"image-registry-7cd68cf8cd-h6sdf\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" May 28 16:09:19.523168 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:19.523068 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4c87c5ee-321e-4fa5-a36a-550f442768f0-installation-pull-secrets\") pod \"image-registry-7cd68cf8cd-h6sdf\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" May 28 16:09:19.523168 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:19.523092 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4c87c5ee-321e-4fa5-a36a-550f442768f0-registry-tls\") pod \"image-registry-7cd68cf8cd-h6sdf\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" May 28 16:09:19.523168 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:19.523128 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4c87c5ee-321e-4fa5-a36a-550f442768f0-registry-certificates\") pod \"image-registry-7cd68cf8cd-h6sdf\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" May 28 16:09:19.523470 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:09:19.523268 2566 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 28 16:09:19.523470 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:09:19.523308 2566 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cd68cf8cd-h6sdf: secret "image-registry-tls" not found May 28 16:09:19.524396 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:09:19.523788 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c87c5ee-321e-4fa5-a36a-550f442768f0-registry-tls podName:4c87c5ee-321e-4fa5-a36a-550f442768f0 nodeName:}" failed. No retries permitted until 2026-05-28 16:09:20.02375976 +0000 UTC m=+99.250790296 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4c87c5ee-321e-4fa5-a36a-550f442768f0-registry-tls") pod "image-registry-7cd68cf8cd-h6sdf" (UID: "4c87c5ee-321e-4fa5-a36a-550f442768f0") : secret "image-registry-tls" not found May 28 16:09:19.524396 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:19.523823 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4c87c5ee-321e-4fa5-a36a-550f442768f0-ca-trust-extracted\") pod \"image-registry-7cd68cf8cd-h6sdf\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" May 28 16:09:19.524396 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:19.524335 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4c87c5ee-321e-4fa5-a36a-550f442768f0-registry-certificates\") pod \"image-registry-7cd68cf8cd-h6sdf\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" May 28 16:09:19.524749 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:19.524723 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4c87c5ee-321e-4fa5-a36a-550f442768f0-trusted-ca\") pod \"image-registry-7cd68cf8cd-h6sdf\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" May 28 16:09:19.527804 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:19.527739 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4c87c5ee-321e-4fa5-a36a-550f442768f0-image-registry-private-configuration\") pod \"image-registry-7cd68cf8cd-h6sdf\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" May 28 16:09:19.528556 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:19.528535 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4c87c5ee-321e-4fa5-a36a-550f442768f0-installation-pull-secrets\") pod \"image-registry-7cd68cf8cd-h6sdf\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" May 28 16:09:19.534592 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:19.534563 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4c87c5ee-321e-4fa5-a36a-550f442768f0-bound-sa-token\") pod \"image-registry-7cd68cf8cd-h6sdf\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" May 28 16:09:19.534742 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:19.534724 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5tg5\" (UniqueName: \"kubernetes.io/projected/4c87c5ee-321e-4fa5-a36a-550f442768f0-kube-api-access-k5tg5\") pod \"image-registry-7cd68cf8cd-h6sdf\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" May 28 16:09:20.027199 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:20.027155 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4c87c5ee-321e-4fa5-a36a-550f442768f0-registry-tls\") pod \"image-registry-7cd68cf8cd-h6sdf\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" May 28 16:09:20.027609 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:09:20.027305 2566 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 28 16:09:20.027609 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:09:20.027323 2566 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cd68cf8cd-h6sdf: secret "image-registry-tls" not found May 28 16:09:20.027609 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:09:20.027388 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c87c5ee-321e-4fa5-a36a-550f442768f0-registry-tls podName:4c87c5ee-321e-4fa5-a36a-550f442768f0 nodeName:}" failed. No retries permitted until 2026-05-28 16:09:21.02737318 +0000 UTC m=+100.254403714 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4c87c5ee-321e-4fa5-a36a-550f442768f0-registry-tls") pod "image-registry-7cd68cf8cd-h6sdf" (UID: "4c87c5ee-321e-4fa5-a36a-550f442768f0") : secret "image-registry-tls" not found May 28 16:09:21.033827 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:21.033778 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4c87c5ee-321e-4fa5-a36a-550f442768f0-registry-tls\") pod \"image-registry-7cd68cf8cd-h6sdf\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" May 28 16:09:21.034206 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:09:21.033945 2566 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 28 16:09:21.034206 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:09:21.033959 2566 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cd68cf8cd-h6sdf: secret "image-registry-tls" not found May 28 16:09:21.034206 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:09:21.034026 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c87c5ee-321e-4fa5-a36a-550f442768f0-registry-tls podName:4c87c5ee-321e-4fa5-a36a-550f442768f0 nodeName:}" failed. No retries permitted until 2026-05-28 16:09:23.034009372 +0000 UTC m=+102.261039907 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4c87c5ee-321e-4fa5-a36a-550f442768f0-registry-tls") pod "image-registry-7cd68cf8cd-h6sdf" (UID: "4c87c5ee-321e-4fa5-a36a-550f442768f0") : secret "image-registry-tls" not found May 28 16:09:22.623797 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:22.623763 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2nzbn" May 28 16:09:23.049180 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:23.049136 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4c87c5ee-321e-4fa5-a36a-550f442768f0-registry-tls\") pod \"image-registry-7cd68cf8cd-h6sdf\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" May 28 16:09:23.049392 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:09:23.049265 2566 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 28 16:09:23.049392 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:09:23.049277 2566 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cd68cf8cd-h6sdf: secret "image-registry-tls" not found May 28 16:09:23.049392 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:09:23.049361 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c87c5ee-321e-4fa5-a36a-550f442768f0-registry-tls podName:4c87c5ee-321e-4fa5-a36a-550f442768f0 nodeName:}" failed. No retries permitted until 2026-05-28 16:09:27.049345912 +0000 UTC m=+106.276376446 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4c87c5ee-321e-4fa5-a36a-550f442768f0-registry-tls") pod "image-registry-7cd68cf8cd-h6sdf" (UID: "4c87c5ee-321e-4fa5-a36a-550f442768f0") : secret "image-registry-tls" not found May 28 16:09:27.080371 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:27.080333 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4c87c5ee-321e-4fa5-a36a-550f442768f0-registry-tls\") pod \"image-registry-7cd68cf8cd-h6sdf\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" May 28 16:09:27.082889 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:27.082865 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4c87c5ee-321e-4fa5-a36a-550f442768f0-registry-tls\") pod \"image-registry-7cd68cf8cd-h6sdf\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" May 28 16:09:27.111733 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:27.111683 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" May 28 16:09:27.241826 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:27.241793 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7cd68cf8cd-h6sdf"] May 28 16:09:27.245328 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:09:27.245298 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c87c5ee_321e_4fa5_a36a_550f442768f0.slice/crio-1706b0267d2d220933333ba3d92ce634299786b8eec4761d0aa3cc0c75f2f9c7 WatchSource:0}: Error finding container 1706b0267d2d220933333ba3d92ce634299786b8eec4761d0aa3cc0c75f2f9c7: Status 404 returned error can't find the container with id 1706b0267d2d220933333ba3d92ce634299786b8eec4761d0aa3cc0c75f2f9c7 May 28 16:09:27.706214 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:27.706173 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" event={"ID":"4c87c5ee-321e-4fa5-a36a-550f442768f0","Type":"ContainerStarted","Data":"e2c0a2db067a812cb077ec217ea4c5e383e9e9aa7b30f833f5a72fee2a257f68"} May 28 16:09:27.706214 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:27.706219 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" event={"ID":"4c87c5ee-321e-4fa5-a36a-550f442768f0","Type":"ContainerStarted","Data":"1706b0267d2d220933333ba3d92ce634299786b8eec4761d0aa3cc0c75f2f9c7"} May 28 16:09:27.706445 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:27.706309 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" May 28 16:09:27.728324 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:27.728254 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" podStartSLOduration=8.728239211 podStartE2EDuration="8.728239211s" podCreationTimestamp="2026-05-28 16:09:19 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 16:09:27.726604899 +0000 UTC m=+106.953635457" watchObservedRunningTime="2026-05-28 16:09:27.728239211 +0000 UTC m=+106.955269766" May 28 16:09:39.979630 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:39.979594 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7cd68cf8cd-h6sdf"] May 28 16:09:40.017198 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.017157 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-658d7dd74-s7f52"] May 28 16:09:40.022097 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.022074 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-658d7dd74-s7f52" May 28 16:09:40.027016 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.026991 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7x5pb"] May 28 16:09:40.030150 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.030129 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7x5pb" May 28 16:09:40.033338 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.033311 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" May 28 16:09:40.033473 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.033456 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" May 28 16:09:40.033541 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.033474 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" May 28 16:09:40.033541 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.033496 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4lxdz\"" May 28 16:09:40.033809 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.033791 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" May 28 16:09:40.044219 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.044197 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-658d7dd74-s7f52"] May 28 16:09:40.045099 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.045076 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7x5pb"] May 28 16:09:40.176586 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.176548 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zvzx\" (UniqueName: \"kubernetes.io/projected/277f3545-b472-4ea1-be1a-42144d81f7ed-kube-api-access-7zvzx\") pod \"image-registry-658d7dd74-s7f52\" (UID: \"277f3545-b472-4ea1-be1a-42144d81f7ed\") " pod="openshift-image-registry/image-registry-658d7dd74-s7f52" May 28 16:09:40.176586 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.176590 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/acb19e01-6353-4282-b75d-095f260c2083-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7x5pb\" (UID: \"acb19e01-6353-4282-b75d-095f260c2083\") " pod="openshift-insights/insights-runtime-extractor-7x5pb" May 28 16:09:40.176828 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.176649 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/acb19e01-6353-4282-b75d-095f260c2083-data-volume\") pod \"insights-runtime-extractor-7x5pb\" (UID: \"acb19e01-6353-4282-b75d-095f260c2083\") " pod="openshift-insights/insights-runtime-extractor-7x5pb" May 28 16:09:40.176828 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.176697 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/277f3545-b472-4ea1-be1a-42144d81f7ed-installation-pull-secrets\") pod \"image-registry-658d7dd74-s7f52\" (UID: \"277f3545-b472-4ea1-be1a-42144d81f7ed\") " pod="openshift-image-registry/image-registry-658d7dd74-s7f52" May 28 16:09:40.176828 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.176722 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/277f3545-b472-4ea1-be1a-42144d81f7ed-registry-certificates\") pod \"image-registry-658d7dd74-s7f52\" (UID: \"277f3545-b472-4ea1-be1a-42144d81f7ed\") " pod="openshift-image-registry/image-registry-658d7dd74-s7f52" May 28 16:09:40.176828 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.176744 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/277f3545-b472-4ea1-be1a-42144d81f7ed-trusted-ca\") pod \"image-registry-658d7dd74-s7f52\" (UID: \"277f3545-b472-4ea1-be1a-42144d81f7ed\") " pod="openshift-image-registry/image-registry-658d7dd74-s7f52" May 28 16:09:40.177030 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.176866 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/277f3545-b472-4ea1-be1a-42144d81f7ed-registry-tls\") pod \"image-registry-658d7dd74-s7f52\" (UID: \"277f3545-b472-4ea1-be1a-42144d81f7ed\") " pod="openshift-image-registry/image-registry-658d7dd74-s7f52" May 28 16:09:40.177030 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.176895 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/277f3545-b472-4ea1-be1a-42144d81f7ed-bound-sa-token\") pod \"image-registry-658d7dd74-s7f52\" (UID: \"277f3545-b472-4ea1-be1a-42144d81f7ed\") " pod="openshift-image-registry/image-registry-658d7dd74-s7f52" May 28 16:09:40.177030 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.176928 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zfqz\" (UniqueName: \"kubernetes.io/projected/acb19e01-6353-4282-b75d-095f260c2083-kube-api-access-2zfqz\") pod \"insights-runtime-extractor-7x5pb\" (UID: \"acb19e01-6353-4282-b75d-095f260c2083\") " pod="openshift-insights/insights-runtime-extractor-7x5pb" May 28 16:09:40.177030 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.176963 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/277f3545-b472-4ea1-be1a-42144d81f7ed-image-registry-private-configuration\") pod \"image-registry-658d7dd74-s7f52\" (UID: \"277f3545-b472-4ea1-be1a-42144d81f7ed\") " pod="openshift-image-registry/image-registry-658d7dd74-s7f52" May 28 16:09:40.177030 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.177013 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/acb19e01-6353-4282-b75d-095f260c2083-crio-socket\") pod \"insights-runtime-extractor-7x5pb\" (UID: \"acb19e01-6353-4282-b75d-095f260c2083\") " pod="openshift-insights/insights-runtime-extractor-7x5pb" May 28 16:09:40.177229 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.177037 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/277f3545-b472-4ea1-be1a-42144d81f7ed-ca-trust-extracted\") pod \"image-registry-658d7dd74-s7f52\" (UID: \"277f3545-b472-4ea1-be1a-42144d81f7ed\") " pod="openshift-image-registry/image-registry-658d7dd74-s7f52" May 28 16:09:40.177229 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.177108 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/acb19e01-6353-4282-b75d-095f260c2083-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7x5pb\" (UID: \"acb19e01-6353-4282-b75d-095f260c2083\") " pod="openshift-insights/insights-runtime-extractor-7x5pb" May 28 16:09:40.278249 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.278156 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/277f3545-b472-4ea1-be1a-42144d81f7ed-trusted-ca\") pod \"image-registry-658d7dd74-s7f52\" (UID: \"277f3545-b472-4ea1-be1a-42144d81f7ed\") " pod="openshift-image-registry/image-registry-658d7dd74-s7f52" May 28 16:09:40.278249 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.278202 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/277f3545-b472-4ea1-be1a-42144d81f7ed-registry-tls\") pod \"image-registry-658d7dd74-s7f52\" (UID: \"277f3545-b472-4ea1-be1a-42144d81f7ed\") " pod="openshift-image-registry/image-registry-658d7dd74-s7f52" May 28 16:09:40.278249 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.278221 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/277f3545-b472-4ea1-be1a-42144d81f7ed-bound-sa-token\") pod \"image-registry-658d7dd74-s7f52\" (UID: \"277f3545-b472-4ea1-be1a-42144d81f7ed\") " pod="openshift-image-registry/image-registry-658d7dd74-s7f52" May 28 16:09:40.278532 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.278243 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2zfqz\" (UniqueName: \"kubernetes.io/projected/acb19e01-6353-4282-b75d-095f260c2083-kube-api-access-2zfqz\") pod \"insights-runtime-extractor-7x5pb\" (UID: \"acb19e01-6353-4282-b75d-095f260c2083\") " pod="openshift-insights/insights-runtime-extractor-7x5pb" May 28 16:09:40.278532 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.278314 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/277f3545-b472-4ea1-be1a-42144d81f7ed-image-registry-private-configuration\") pod \"image-registry-658d7dd74-s7f52\" (UID: \"277f3545-b472-4ea1-be1a-42144d81f7ed\") " pod="openshift-image-registry/image-registry-658d7dd74-s7f52" May 28 16:09:40.278532 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.278359 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/acb19e01-6353-4282-b75d-095f260c2083-crio-socket\") pod \"insights-runtime-extractor-7x5pb\" (UID: \"acb19e01-6353-4282-b75d-095f260c2083\") " pod="openshift-insights/insights-runtime-extractor-7x5pb" May 28 16:09:40.278532 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.278378 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/277f3545-b472-4ea1-be1a-42144d81f7ed-ca-trust-extracted\") pod \"image-registry-658d7dd74-s7f52\" (UID: \"277f3545-b472-4ea1-be1a-42144d81f7ed\") " pod="openshift-image-registry/image-registry-658d7dd74-s7f52" May 28 16:09:40.278532 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.278412 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/acb19e01-6353-4282-b75d-095f260c2083-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7x5pb\" (UID: \"acb19e01-6353-4282-b75d-095f260c2083\") " pod="openshift-insights/insights-runtime-extractor-7x5pb" May 28 16:09:40.278532 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.278497 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/acb19e01-6353-4282-b75d-095f260c2083-crio-socket\") pod \"insights-runtime-extractor-7x5pb\" (UID: \"acb19e01-6353-4282-b75d-095f260c2083\") " pod="openshift-insights/insights-runtime-extractor-7x5pb" May 28 16:09:40.278813 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.278555 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7zvzx\" (UniqueName: \"kubernetes.io/projected/277f3545-b472-4ea1-be1a-42144d81f7ed-kube-api-access-7zvzx\") pod \"image-registry-658d7dd74-s7f52\" (UID: \"277f3545-b472-4ea1-be1a-42144d81f7ed\") " pod="openshift-image-registry/image-registry-658d7dd74-s7f52" May 28 16:09:40.278813 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.278590 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/acb19e01-6353-4282-b75d-095f260c2083-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7x5pb\" (UID: \"acb19e01-6353-4282-b75d-095f260c2083\") " pod="openshift-insights/insights-runtime-extractor-7x5pb" May 28 16:09:40.278813 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.278622 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/acb19e01-6353-4282-b75d-095f260c2083-data-volume\") pod \"insights-runtime-extractor-7x5pb\" (UID: \"acb19e01-6353-4282-b75d-095f260c2083\") " pod="openshift-insights/insights-runtime-extractor-7x5pb" May 28 16:09:40.278813 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.278669 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/277f3545-b472-4ea1-be1a-42144d81f7ed-installation-pull-secrets\") pod \"image-registry-658d7dd74-s7f52\" (UID: \"277f3545-b472-4ea1-be1a-42144d81f7ed\") " pod="openshift-image-registry/image-registry-658d7dd74-s7f52" May 28 16:09:40.278813 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.278703 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/277f3545-b472-4ea1-be1a-42144d81f7ed-registry-certificates\") pod \"image-registry-658d7dd74-s7f52\" (UID: \"277f3545-b472-4ea1-be1a-42144d81f7ed\") " pod="openshift-image-registry/image-registry-658d7dd74-s7f52" May 28 16:09:40.279081 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.279059 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/acb19e01-6353-4282-b75d-095f260c2083-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7x5pb\" (UID: \"acb19e01-6353-4282-b75d-095f260c2083\") " pod="openshift-insights/insights-runtime-extractor-7x5pb" May 28 16:09:40.279147 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.279069 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/acb19e01-6353-4282-b75d-095f260c2083-data-volume\") pod \"insights-runtime-extractor-7x5pb\" (UID: \"acb19e01-6353-4282-b75d-095f260c2083\") " pod="openshift-insights/insights-runtime-extractor-7x5pb" May 28 16:09:40.279832 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.279426 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/277f3545-b472-4ea1-be1a-42144d81f7ed-ca-trust-extracted\") pod \"image-registry-658d7dd74-s7f52\" (UID: \"277f3545-b472-4ea1-be1a-42144d81f7ed\") " pod="openshift-image-registry/image-registry-658d7dd74-s7f52" May 28 16:09:40.280015 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.279990 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/277f3545-b472-4ea1-be1a-42144d81f7ed-registry-certificates\") pod \"image-registry-658d7dd74-s7f52\" (UID: \"277f3545-b472-4ea1-be1a-42144d81f7ed\") " pod="openshift-image-registry/image-registry-658d7dd74-s7f52" May 28 16:09:40.280426 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.280401 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/277f3545-b472-4ea1-be1a-42144d81f7ed-trusted-ca\") pod \"image-registry-658d7dd74-s7f52\" (UID: \"277f3545-b472-4ea1-be1a-42144d81f7ed\") " pod="openshift-image-registry/image-registry-658d7dd74-s7f52" May 28 16:09:40.281135 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.281117 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/277f3545-b472-4ea1-be1a-42144d81f7ed-registry-tls\") pod \"image-registry-658d7dd74-s7f52\" (UID: \"277f3545-b472-4ea1-be1a-42144d81f7ed\") " pod="openshift-image-registry/image-registry-658d7dd74-s7f52" May 28 16:09:40.281338 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.281314 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/277f3545-b472-4ea1-be1a-42144d81f7ed-installation-pull-secrets\") pod \"image-registry-658d7dd74-s7f52\" (UID: \"277f3545-b472-4ea1-be1a-42144d81f7ed\") " pod="openshift-image-registry/image-registry-658d7dd74-s7f52" May 28 16:09:40.281463 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.281445 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/acb19e01-6353-4282-b75d-095f260c2083-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7x5pb\" (UID: \"acb19e01-6353-4282-b75d-095f260c2083\") " pod="openshift-insights/insights-runtime-extractor-7x5pb" May 28 16:09:40.281516 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.281458 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/277f3545-b472-4ea1-be1a-42144d81f7ed-image-registry-private-configuration\") pod \"image-registry-658d7dd74-s7f52\" (UID: \"277f3545-b472-4ea1-be1a-42144d81f7ed\") " pod="openshift-image-registry/image-registry-658d7dd74-s7f52" May 28 16:09:40.289677 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.289646 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/277f3545-b472-4ea1-be1a-42144d81f7ed-bound-sa-token\") pod \"image-registry-658d7dd74-s7f52\" (UID: \"277f3545-b472-4ea1-be1a-42144d81f7ed\") " pod="openshift-image-registry/image-registry-658d7dd74-s7f52" May 28 16:09:40.289941 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.289921 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zvzx\" (UniqueName: \"kubernetes.io/projected/277f3545-b472-4ea1-be1a-42144d81f7ed-kube-api-access-7zvzx\") pod \"image-registry-658d7dd74-s7f52\" (UID: \"277f3545-b472-4ea1-be1a-42144d81f7ed\") " pod="openshift-image-registry/image-registry-658d7dd74-s7f52" May 28 16:09:40.289987 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.289931 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zfqz\" (UniqueName: \"kubernetes.io/projected/acb19e01-6353-4282-b75d-095f260c2083-kube-api-access-2zfqz\") pod \"insights-runtime-extractor-7x5pb\" (UID: \"acb19e01-6353-4282-b75d-095f260c2083\") " pod="openshift-insights/insights-runtime-extractor-7x5pb" May 28 16:09:40.331713 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.331658 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-658d7dd74-s7f52" May 28 16:09:40.338613 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.338570 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7x5pb" May 28 16:09:40.472104 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.472072 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-658d7dd74-s7f52"] May 28 16:09:40.475958 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:09:40.475925 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod277f3545_b472_4ea1_be1a_42144d81f7ed.slice/crio-16bd7f9f36458c657e92fe13ce8f96f97515c706693be0933cb95f97ec5b261b WatchSource:0}: Error finding container 16bd7f9f36458c657e92fe13ce8f96f97515c706693be0933cb95f97ec5b261b: Status 404 returned error can't find the container with id 16bd7f9f36458c657e92fe13ce8f96f97515c706693be0933cb95f97ec5b261b May 28 16:09:40.485669 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.485645 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7x5pb"] May 28 16:09:40.489332 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:09:40.489306 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacb19e01_6353_4282_b75d_095f260c2083.slice/crio-9def3f2d145b83aed9159566bccd164aa4a1837caf287aa41c2fd5cdc1bc40af WatchSource:0}: Error finding container 9def3f2d145b83aed9159566bccd164aa4a1837caf287aa41c2fd5cdc1bc40af: Status 404 returned error can't find the container with id 9def3f2d145b83aed9159566bccd164aa4a1837caf287aa41c2fd5cdc1bc40af May 28 16:09:40.737504 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.737469 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7x5pb" event={"ID":"acb19e01-6353-4282-b75d-095f260c2083","Type":"ContainerStarted","Data":"5f9c868629fac68e3cc5114da4136aaab4d58962be3d100adb1544bed53022b5"} May 28 16:09:40.737504 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.737508 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7x5pb" event={"ID":"acb19e01-6353-4282-b75d-095f260c2083","Type":"ContainerStarted","Data":"9def3f2d145b83aed9159566bccd164aa4a1837caf287aa41c2fd5cdc1bc40af"} May 28 16:09:40.738844 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.738811 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-658d7dd74-s7f52" event={"ID":"277f3545-b472-4ea1-be1a-42144d81f7ed","Type":"ContainerStarted","Data":"b6b74ceaebe1669e0437fb17b71d7eedeb74d4c726723f563e535e7dc9255d0d"} May 28 16:09:40.738971 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.738853 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-658d7dd74-s7f52" event={"ID":"277f3545-b472-4ea1-be1a-42144d81f7ed","Type":"ContainerStarted","Data":"16bd7f9f36458c657e92fe13ce8f96f97515c706693be0933cb95f97ec5b261b"} May 28 16:09:40.739062 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.739038 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-658d7dd74-s7f52" May 28 16:09:40.759804 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:40.759756 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-658d7dd74-s7f52" podStartSLOduration=1.7597413130000001 podStartE2EDuration="1.759741313s" podCreationTimestamp="2026-05-28 16:09:39 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 16:09:40.758888517 +0000 UTC m=+119.985919075" watchObservedRunningTime="2026-05-28 16:09:40.759741313 +0000 UTC m=+119.986771863" May 28 16:09:41.743036 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:41.743002 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7x5pb" event={"ID":"acb19e01-6353-4282-b75d-095f260c2083","Type":"ContainerStarted","Data":"fba67d6d564acf334c8cc387fe6f580923869d28e6f2d5ad263f83b0a0ed0938"} May 28 16:09:42.747606 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:42.747566 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7x5pb" event={"ID":"acb19e01-6353-4282-b75d-095f260c2083","Type":"ContainerStarted","Data":"fd1fcf0ef51a79946a1d1414f325f6405d899fa3f169d74dce3ffe90269441be"} May 28 16:09:42.774635 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:42.774577 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7x5pb" podStartSLOduration=0.772560499 podStartE2EDuration="2.774558383s" podCreationTimestamp="2026-05-28 16:09:40 +0000 UTC" firstStartedPulling="2026-05-28 16:09:40.541908738 +0000 UTC m=+119.768939273" lastFinishedPulling="2026-05-28 16:09:42.543906604 +0000 UTC m=+121.770937157" observedRunningTime="2026-05-28 16:09:42.774236723 +0000 UTC m=+122.001267269" watchObservedRunningTime="2026-05-28 16:09:42.774558383 +0000 UTC m=+122.001588941" May 28 16:09:49.984971 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:49.984928 2566 patch_prober.go:28] interesting pod/image-registry-7cd68cf8cd-h6sdf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} May 28 16:09:49.985460 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:49.984989 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" podUID="4c87c5ee-321e-4fa5-a36a-550f442768f0" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" May 28 16:09:51.062021 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.061962 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b54fd06-9913-49b9-ac98-d2916e5f0276-metrics-certs\") pod \"network-metrics-daemon-6dt52\" (UID: \"3b54fd06-9913-49b9-ac98-d2916e5f0276\") " pod="openshift-multus/network-metrics-daemon-6dt52" May 28 16:09:51.064525 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.064503 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b54fd06-9913-49b9-ac98-d2916e5f0276-metrics-certs\") pod \"network-metrics-daemon-6dt52\" (UID: \"3b54fd06-9913-49b9-ac98-d2916e5f0276\") " pod="openshift-multus/network-metrics-daemon-6dt52" May 28 16:09:51.159369 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.159335 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xz8bh\"" May 28 16:09:51.166789 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.166767 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dt52" May 28 16:09:51.297164 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.297133 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6dt52"] May 28 16:09:51.300377 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:09:51.300349 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b54fd06_9913_49b9_ac98_d2916e5f0276.slice/crio-1dadeb6f3061be0ef185ddc403e045210578b957603bf625913a059ab6a3c4c2 WatchSource:0}: Error finding container 1dadeb6f3061be0ef185ddc403e045210578b957603bf625913a059ab6a3c4c2: Status 404 returned error can't find the container with id 1dadeb6f3061be0ef185ddc403e045210578b957603bf625913a059ab6a3c4c2 May 28 16:09:51.310154 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.308595 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rwqpn"] May 28 16:09:51.315292 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.315229 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rwqpn" May 28 16:09:51.317853 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.317823 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" May 28 16:09:51.317936 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.317893 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-w2gpp\"" May 28 16:09:51.318868 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.318841 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" May 28 16:09:51.318868 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.318863 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" May 28 16:09:51.319010 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.318841 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" May 28 16:09:51.319010 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.318911 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" May 28 16:09:51.319010 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.318943 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" May 28 16:09:51.464624 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.464590 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2c6458fc-1987-48f7-abc9-289365a79370-node-exporter-accelerators-collector-config\") pod \"node-exporter-rwqpn\" (UID: \"2c6458fc-1987-48f7-abc9-289365a79370\") " pod="openshift-monitoring/node-exporter-rwqpn" May 28 16:09:51.464624 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.464629 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2c6458fc-1987-48f7-abc9-289365a79370-root\") pod \"node-exporter-rwqpn\" (UID: \"2c6458fc-1987-48f7-abc9-289365a79370\") " pod="openshift-monitoring/node-exporter-rwqpn" May 28 16:09:51.464846 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.464653 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c6458fc-1987-48f7-abc9-289365a79370-metrics-client-ca\") pod \"node-exporter-rwqpn\" (UID: \"2c6458fc-1987-48f7-abc9-289365a79370\") " pod="openshift-monitoring/node-exporter-rwqpn" May 28 16:09:51.464846 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.464701 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2c6458fc-1987-48f7-abc9-289365a79370-node-exporter-tls\") pod \"node-exporter-rwqpn\" (UID: \"2c6458fc-1987-48f7-abc9-289365a79370\") " pod="openshift-monitoring/node-exporter-rwqpn" May 28 16:09:51.464846 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.464744 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c6458fc-1987-48f7-abc9-289365a79370-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rwqpn\" (UID: \"2c6458fc-1987-48f7-abc9-289365a79370\") " pod="openshift-monitoring/node-exporter-rwqpn" May 28 16:09:51.464846 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.464760 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dbcn\" (UniqueName: \"kubernetes.io/projected/2c6458fc-1987-48f7-abc9-289365a79370-kube-api-access-9dbcn\") pod \"node-exporter-rwqpn\" (UID: \"2c6458fc-1987-48f7-abc9-289365a79370\") " pod="openshift-monitoring/node-exporter-rwqpn" May 28 16:09:51.464846 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.464829 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2c6458fc-1987-48f7-abc9-289365a79370-node-exporter-textfile\") pod \"node-exporter-rwqpn\" (UID: \"2c6458fc-1987-48f7-abc9-289365a79370\") " pod="openshift-monitoring/node-exporter-rwqpn" May 28 16:09:51.465023 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.464858 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c6458fc-1987-48f7-abc9-289365a79370-sys\") pod \"node-exporter-rwqpn\" (UID: \"2c6458fc-1987-48f7-abc9-289365a79370\") " pod="openshift-monitoring/node-exporter-rwqpn" May 28 16:09:51.465023 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.464879 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2c6458fc-1987-48f7-abc9-289365a79370-node-exporter-wtmp\") pod \"node-exporter-rwqpn\" (UID: \"2c6458fc-1987-48f7-abc9-289365a79370\") " pod="openshift-monitoring/node-exporter-rwqpn" May 28 16:09:51.565590 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.565492 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2c6458fc-1987-48f7-abc9-289365a79370-node-exporter-textfile\") pod \"node-exporter-rwqpn\" (UID: \"2c6458fc-1987-48f7-abc9-289365a79370\") " pod="openshift-monitoring/node-exporter-rwqpn" May 28 16:09:51.565590 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.565556 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c6458fc-1987-48f7-abc9-289365a79370-sys\") pod \"node-exporter-rwqpn\" (UID: \"2c6458fc-1987-48f7-abc9-289365a79370\") " pod="openshift-monitoring/node-exporter-rwqpn" May 28 16:09:51.565590 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.565590 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2c6458fc-1987-48f7-abc9-289365a79370-node-exporter-wtmp\") pod \"node-exporter-rwqpn\" (UID: \"2c6458fc-1987-48f7-abc9-289365a79370\") " pod="openshift-monitoring/node-exporter-rwqpn" May 28 16:09:51.565837 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.565627 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2c6458fc-1987-48f7-abc9-289365a79370-node-exporter-accelerators-collector-config\") pod \"node-exporter-rwqpn\" (UID: \"2c6458fc-1987-48f7-abc9-289365a79370\") " pod="openshift-monitoring/node-exporter-rwqpn" May 28 16:09:51.565837 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.565653 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2c6458fc-1987-48f7-abc9-289365a79370-root\") pod \"node-exporter-rwqpn\" (UID: \"2c6458fc-1987-48f7-abc9-289365a79370\") " pod="openshift-monitoring/node-exporter-rwqpn" May 28 16:09:51.565837 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.565673 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c6458fc-1987-48f7-abc9-289365a79370-metrics-client-ca\") pod \"node-exporter-rwqpn\" (UID: \"2c6458fc-1987-48f7-abc9-289365a79370\") " pod="openshift-monitoring/node-exporter-rwqpn" May 28 16:09:51.565837 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.565676 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c6458fc-1987-48f7-abc9-289365a79370-sys\") pod \"node-exporter-rwqpn\" (UID: \"2c6458fc-1987-48f7-abc9-289365a79370\") " pod="openshift-monitoring/node-exporter-rwqpn" May 28 16:09:51.565837 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.565727 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2c6458fc-1987-48f7-abc9-289365a79370-root\") pod \"node-exporter-rwqpn\" (UID: \"2c6458fc-1987-48f7-abc9-289365a79370\") " pod="openshift-monitoring/node-exporter-rwqpn" May 28 16:09:51.565837 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.565765 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2c6458fc-1987-48f7-abc9-289365a79370-node-exporter-tls\") pod \"node-exporter-rwqpn\" (UID: \"2c6458fc-1987-48f7-abc9-289365a79370\") " pod="openshift-monitoring/node-exporter-rwqpn" May 28 16:09:51.565837 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.565775 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2c6458fc-1987-48f7-abc9-289365a79370-node-exporter-wtmp\") pod \"node-exporter-rwqpn\" (UID: \"2c6458fc-1987-48f7-abc9-289365a79370\") " pod="openshift-monitoring/node-exporter-rwqpn" May 28 16:09:51.565837 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.565829 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c6458fc-1987-48f7-abc9-289365a79370-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rwqpn\" (UID: \"2c6458fc-1987-48f7-abc9-289365a79370\") " pod="openshift-monitoring/node-exporter-rwqpn" May 28 16:09:51.566220 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.565852 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9dbcn\" (UniqueName: \"kubernetes.io/projected/2c6458fc-1987-48f7-abc9-289365a79370-kube-api-access-9dbcn\") pod \"node-exporter-rwqpn\" (UID: \"2c6458fc-1987-48f7-abc9-289365a79370\") " pod="openshift-monitoring/node-exporter-rwqpn" May 28 16:09:51.566220 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.565908 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2c6458fc-1987-48f7-abc9-289365a79370-node-exporter-textfile\") pod \"node-exporter-rwqpn\" (UID: \"2c6458fc-1987-48f7-abc9-289365a79370\") " pod="openshift-monitoring/node-exporter-rwqpn" May 28 16:09:51.566220 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.566130 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2c6458fc-1987-48f7-abc9-289365a79370-node-exporter-accelerators-collector-config\") pod \"node-exporter-rwqpn\" (UID: \"2c6458fc-1987-48f7-abc9-289365a79370\") " pod="openshift-monitoring/node-exporter-rwqpn" May 28 16:09:51.566346 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.566226 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c6458fc-1987-48f7-abc9-289365a79370-metrics-client-ca\") pod \"node-exporter-rwqpn\" (UID: \"2c6458fc-1987-48f7-abc9-289365a79370\") " pod="openshift-monitoring/node-exporter-rwqpn" May 28 16:09:51.568212 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.568189 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c6458fc-1987-48f7-abc9-289365a79370-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rwqpn\" (UID: \"2c6458fc-1987-48f7-abc9-289365a79370\") " pod="openshift-monitoring/node-exporter-rwqpn" May 28 16:09:51.568365 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.568349 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2c6458fc-1987-48f7-abc9-289365a79370-node-exporter-tls\") pod \"node-exporter-rwqpn\" (UID: \"2c6458fc-1987-48f7-abc9-289365a79370\") " pod="openshift-monitoring/node-exporter-rwqpn" May 28 16:09:51.573143 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.573114 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dbcn\" (UniqueName: \"kubernetes.io/projected/2c6458fc-1987-48f7-abc9-289365a79370-kube-api-access-9dbcn\") pod \"node-exporter-rwqpn\" (UID: \"2c6458fc-1987-48f7-abc9-289365a79370\") " pod="openshift-monitoring/node-exporter-rwqpn" May 28 16:09:51.624356 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.624323 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rwqpn" May 28 16:09:51.632510 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:09:51.632487 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c6458fc_1987_48f7_abc9_289365a79370.slice/crio-093e312e4087cabf851cafcea0233f6c9eb82c30f6dc6d9901963e745184fdb6 WatchSource:0}: Error finding container 093e312e4087cabf851cafcea0233f6c9eb82c30f6dc6d9901963e745184fdb6: Status 404 returned error can't find the container with id 093e312e4087cabf851cafcea0233f6c9eb82c30f6dc6d9901963e745184fdb6 May 28 16:09:51.770738 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.770691 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rwqpn" event={"ID":"2c6458fc-1987-48f7-abc9-289365a79370","Type":"ContainerStarted","Data":"093e312e4087cabf851cafcea0233f6c9eb82c30f6dc6d9901963e745184fdb6"} May 28 16:09:51.771820 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:51.771787 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6dt52" event={"ID":"3b54fd06-9913-49b9-ac98-d2916e5f0276","Type":"ContainerStarted","Data":"1dadeb6f3061be0ef185ddc403e045210578b957603bf625913a059ab6a3c4c2"} May 28 16:09:52.776076 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:52.776039 2566 generic.go:358] "Generic (PLEG): container finished" podID="2c6458fc-1987-48f7-abc9-289365a79370" containerID="0c68fc7e19997959bc04835702197f76cfd2708a7c8603e481a0c5867619890e" exitCode=0 May 28 16:09:52.776517 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:52.776127 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rwqpn" event={"ID":"2c6458fc-1987-48f7-abc9-289365a79370","Type":"ContainerDied","Data":"0c68fc7e19997959bc04835702197f76cfd2708a7c8603e481a0c5867619890e"} May 28 16:09:52.777837 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:52.777815 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6dt52" event={"ID":"3b54fd06-9913-49b9-ac98-d2916e5f0276","Type":"ContainerStarted","Data":"d0617b228d186f0a25d7cc508c998c5f27294c40b6daa9f18b0dfc7708faa73b"} May 28 16:09:52.777923 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:52.777844 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6dt52" event={"ID":"3b54fd06-9913-49b9-ac98-d2916e5f0276","Type":"ContainerStarted","Data":"bfe6b96cf32585f7ad2f300fbbf179bbcb4301332cfd6ccca7c41b3c68338061"} May 28 16:09:52.807455 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:52.807404 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-6dt52" podStartSLOduration=130.821193476 podStartE2EDuration="2m11.807387185s" podCreationTimestamp="2026-05-28 16:07:41 +0000 UTC" firstStartedPulling="2026-05-28 16:09:51.302247052 +0000 UTC m=+130.529277586" lastFinishedPulling="2026-05-28 16:09:52.288440748 +0000 UTC m=+131.515471295" observedRunningTime="2026-05-28 16:09:52.806158768 +0000 UTC m=+132.033189325" watchObservedRunningTime="2026-05-28 16:09:52.807387185 +0000 UTC m=+132.034417741" May 28 16:09:53.782460 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:53.782412 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rwqpn" event={"ID":"2c6458fc-1987-48f7-abc9-289365a79370","Type":"ContainerStarted","Data":"2545fd53bf0abb65a33e858dd455d0727b990358b8997618734915105e345b46"} May 28 16:09:53.782460 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:53.782463 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rwqpn" event={"ID":"2c6458fc-1987-48f7-abc9-289365a79370","Type":"ContainerStarted","Data":"fddd715b520c15bce0d42f069cce52e6a6eff8927be7ac77ad8a7e07f5b73f43"} May 28 16:09:53.803240 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:53.803182 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rwqpn" podStartSLOduration=1.855712394 podStartE2EDuration="2.803167739s" podCreationTimestamp="2026-05-28 16:09:51 +0000 UTC" firstStartedPulling="2026-05-28 16:09:51.634155869 +0000 UTC m=+130.861186406" lastFinishedPulling="2026-05-28 16:09:52.581611199 +0000 UTC m=+131.808641751" observedRunningTime="2026-05-28 16:09:53.801688939 +0000 UTC m=+133.028719495" watchObservedRunningTime="2026-05-28 16:09:53.803167739 +0000 UTC m=+133.030198295" May 28 16:09:59.983835 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:09:59.983799 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" May 28 16:10:00.336254 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:00.336179 2566 patch_prober.go:28] interesting pod/image-registry-658d7dd74-s7f52 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} May 28 16:10:00.336254 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:00.336237 2566 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-658d7dd74-s7f52" podUID="277f3545-b472-4ea1-be1a-42144d81f7ed" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" May 28 16:10:01.746768 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:01.746740 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-658d7dd74-s7f52" May 28 16:10:04.998760 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:04.998716 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" podUID="4c87c5ee-321e-4fa5-a36a-550f442768f0" containerName="registry" containerID="cri-o://e2c0a2db067a812cb077ec217ea4c5e383e9e9aa7b30f833f5a72fee2a257f68" gracePeriod=30 May 28 16:10:05.234529 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:05.234504 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" May 28 16:10:05.288886 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:05.288786 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k5tg5\" (UniqueName: \"kubernetes.io/projected/4c87c5ee-321e-4fa5-a36a-550f442768f0-kube-api-access-k5tg5\") pod \"4c87c5ee-321e-4fa5-a36a-550f442768f0\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " May 28 16:10:05.288886 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:05.288836 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4c87c5ee-321e-4fa5-a36a-550f442768f0-registry-tls\") pod \"4c87c5ee-321e-4fa5-a36a-550f442768f0\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " May 28 16:10:05.288886 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:05.288864 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4c87c5ee-321e-4fa5-a36a-550f442768f0-trusted-ca\") pod \"4c87c5ee-321e-4fa5-a36a-550f442768f0\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " May 28 16:10:05.289173 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:05.288894 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4c87c5ee-321e-4fa5-a36a-550f442768f0-installation-pull-secrets\") pod \"4c87c5ee-321e-4fa5-a36a-550f442768f0\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " May 28 16:10:05.289173 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:05.288914 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4c87c5ee-321e-4fa5-a36a-550f442768f0-image-registry-private-configuration\") pod \"4c87c5ee-321e-4fa5-a36a-550f442768f0\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " May 28 16:10:05.289173 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:05.288939 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4c87c5ee-321e-4fa5-a36a-550f442768f0-registry-certificates\") pod \"4c87c5ee-321e-4fa5-a36a-550f442768f0\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " May 28 16:10:05.289173 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:05.288960 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4c87c5ee-321e-4fa5-a36a-550f442768f0-ca-trust-extracted\") pod \"4c87c5ee-321e-4fa5-a36a-550f442768f0\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " May 28 16:10:05.289173 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:05.288998 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4c87c5ee-321e-4fa5-a36a-550f442768f0-bound-sa-token\") pod \"4c87c5ee-321e-4fa5-a36a-550f442768f0\" (UID: \"4c87c5ee-321e-4fa5-a36a-550f442768f0\") " May 28 16:10:05.289603 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:05.289571 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c87c5ee-321e-4fa5-a36a-550f442768f0-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4c87c5ee-321e-4fa5-a36a-550f442768f0" (UID: "4c87c5ee-321e-4fa5-a36a-550f442768f0"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 16:10:05.289857 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:05.289827 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c87c5ee-321e-4fa5-a36a-550f442768f0-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4c87c5ee-321e-4fa5-a36a-550f442768f0" (UID: "4c87c5ee-321e-4fa5-a36a-550f442768f0"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 16:10:05.291476 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:05.291429 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c87c5ee-321e-4fa5-a36a-550f442768f0-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4c87c5ee-321e-4fa5-a36a-550f442768f0" (UID: "4c87c5ee-321e-4fa5-a36a-550f442768f0"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 16:10:05.291825 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:05.291796 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c87c5ee-321e-4fa5-a36a-550f442768f0-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4c87c5ee-321e-4fa5-a36a-550f442768f0" (UID: "4c87c5ee-321e-4fa5-a36a-550f442768f0"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 16:10:05.291825 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:05.291803 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c87c5ee-321e-4fa5-a36a-550f442768f0-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4c87c5ee-321e-4fa5-a36a-550f442768f0" (UID: "4c87c5ee-321e-4fa5-a36a-550f442768f0"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 16:10:05.291953 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:05.291901 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c87c5ee-321e-4fa5-a36a-550f442768f0-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4c87c5ee-321e-4fa5-a36a-550f442768f0" (UID: "4c87c5ee-321e-4fa5-a36a-550f442768f0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 16:10:05.291953 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:05.291927 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c87c5ee-321e-4fa5-a36a-550f442768f0-kube-api-access-k5tg5" (OuterVolumeSpecName: "kube-api-access-k5tg5") pod "4c87c5ee-321e-4fa5-a36a-550f442768f0" (UID: "4c87c5ee-321e-4fa5-a36a-550f442768f0"). InnerVolumeSpecName "kube-api-access-k5tg5". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 16:10:05.298006 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:05.297978 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c87c5ee-321e-4fa5-a36a-550f442768f0-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4c87c5ee-321e-4fa5-a36a-550f442768f0" (UID: "4c87c5ee-321e-4fa5-a36a-550f442768f0"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 28 16:10:05.389901 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:05.389857 2566 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4c87c5ee-321e-4fa5-a36a-550f442768f0-registry-certificates\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" May 28 16:10:05.389901 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:05.389894 2566 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4c87c5ee-321e-4fa5-a36a-550f442768f0-ca-trust-extracted\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" May 28 16:10:05.389901 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:05.389909 2566 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4c87c5ee-321e-4fa5-a36a-550f442768f0-bound-sa-token\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" May 28 16:10:05.390145 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:05.389922 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k5tg5\" (UniqueName: \"kubernetes.io/projected/4c87c5ee-321e-4fa5-a36a-550f442768f0-kube-api-access-k5tg5\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" May 28 16:10:05.390145 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:05.389935 2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4c87c5ee-321e-4fa5-a36a-550f442768f0-registry-tls\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" May 28 16:10:05.390145 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:05.389947 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4c87c5ee-321e-4fa5-a36a-550f442768f0-trusted-ca\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" May 28 16:10:05.390145 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:05.389957 2566 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4c87c5ee-321e-4fa5-a36a-550f442768f0-installation-pull-secrets\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" May 28 16:10:05.390145 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:05.389971 2566 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4c87c5ee-321e-4fa5-a36a-550f442768f0-image-registry-private-configuration\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" May 28 16:10:05.815262 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:05.815224 2566 generic.go:358] "Generic (PLEG): container finished" podID="4c87c5ee-321e-4fa5-a36a-550f442768f0" containerID="e2c0a2db067a812cb077ec217ea4c5e383e9e9aa7b30f833f5a72fee2a257f68" exitCode=0 May 28 16:10:05.815452 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:05.815308 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" May 28 16:10:05.815452 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:05.815308 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" event={"ID":"4c87c5ee-321e-4fa5-a36a-550f442768f0","Type":"ContainerDied","Data":"e2c0a2db067a812cb077ec217ea4c5e383e9e9aa7b30f833f5a72fee2a257f68"} May 28 16:10:05.815452 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:05.815417 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cd68cf8cd-h6sdf" event={"ID":"4c87c5ee-321e-4fa5-a36a-550f442768f0","Type":"ContainerDied","Data":"1706b0267d2d220933333ba3d92ce634299786b8eec4761d0aa3cc0c75f2f9c7"} May 28 16:10:05.815452 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:05.815432 2566 scope.go:117] "RemoveContainer" containerID="e2c0a2db067a812cb077ec217ea4c5e383e9e9aa7b30f833f5a72fee2a257f68" May 28 16:10:05.823061 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:05.823043 2566 scope.go:117] "RemoveContainer" containerID="e2c0a2db067a812cb077ec217ea4c5e383e9e9aa7b30f833f5a72fee2a257f68" May 28 16:10:05.823472 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:10:05.823445 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2c0a2db067a812cb077ec217ea4c5e383e9e9aa7b30f833f5a72fee2a257f68\": container with ID starting with e2c0a2db067a812cb077ec217ea4c5e383e9e9aa7b30f833f5a72fee2a257f68 not found: ID does not exist" containerID="e2c0a2db067a812cb077ec217ea4c5e383e9e9aa7b30f833f5a72fee2a257f68" May 28 16:10:05.823569 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:05.823477 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2c0a2db067a812cb077ec217ea4c5e383e9e9aa7b30f833f5a72fee2a257f68"} err="failed to get container status \"e2c0a2db067a812cb077ec217ea4c5e383e9e9aa7b30f833f5a72fee2a257f68\": rpc error: code = NotFound desc = could not find container \"e2c0a2db067a812cb077ec217ea4c5e383e9e9aa7b30f833f5a72fee2a257f68\": container with ID starting with e2c0a2db067a812cb077ec217ea4c5e383e9e9aa7b30f833f5a72fee2a257f68 not found: ID does not exist" May 28 16:10:05.838196 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:05.838165 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7cd68cf8cd-h6sdf"] May 28 16:10:05.844528 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:05.844498 2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7cd68cf8cd-h6sdf"] May 28 16:10:07.336118 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:07.336076 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c87c5ee-321e-4fa5-a36a-550f442768f0" path="/var/lib/kubelet/pods/4c87c5ee-321e-4fa5-a36a-550f442768f0/volumes" May 28 16:10:12.077960 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:12.077922 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-58f6d868f4-nwpk6"] May 28 16:10:12.078350 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:12.078147 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c87c5ee-321e-4fa5-a36a-550f442768f0" containerName="registry" May 28 16:10:12.078350 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:12.078158 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c87c5ee-321e-4fa5-a36a-550f442768f0" containerName="registry" May 28 16:10:12.078350 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:12.078208 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="4c87c5ee-321e-4fa5-a36a-550f442768f0" containerName="registry" May 28 16:10:12.082795 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:12.082772 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58f6d868f4-nwpk6" May 28 16:10:12.085516 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:12.085491 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" May 28 16:10:12.085641 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:12.085543 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-7r9nd\"" May 28 16:10:12.085641 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:12.085491 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" May 28 16:10:12.085641 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:12.085591 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" May 28 16:10:12.086636 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:12.086622 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" May 28 16:10:12.086755 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:12.086689 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" May 28 16:10:12.086840 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:12.086753 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" May 28 16:10:12.086840 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:12.086689 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" May 28 16:10:12.091290 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:12.091251 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58f6d868f4-nwpk6"] May 28 16:10:12.145580 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:12.145547 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df3fa509-4d2c-411d-805f-e11e37048426-oauth-serving-cert\") pod \"console-58f6d868f4-nwpk6\" (UID: \"df3fa509-4d2c-411d-805f-e11e37048426\") " pod="openshift-console/console-58f6d868f4-nwpk6" May 28 16:10:12.145783 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:12.145604 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df3fa509-4d2c-411d-805f-e11e37048426-service-ca\") pod \"console-58f6d868f4-nwpk6\" (UID: \"df3fa509-4d2c-411d-805f-e11e37048426\") " pod="openshift-console/console-58f6d868f4-nwpk6" May 28 16:10:12.145783 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:12.145658 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kx9gm\" (UniqueName: \"kubernetes.io/projected/df3fa509-4d2c-411d-805f-e11e37048426-kube-api-access-kx9gm\") pod \"console-58f6d868f4-nwpk6\" (UID: \"df3fa509-4d2c-411d-805f-e11e37048426\") " pod="openshift-console/console-58f6d868f4-nwpk6" May 28 16:10:12.145783 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:12.145683 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df3fa509-4d2c-411d-805f-e11e37048426-console-serving-cert\") pod \"console-58f6d868f4-nwpk6\" (UID: \"df3fa509-4d2c-411d-805f-e11e37048426\") " pod="openshift-console/console-58f6d868f4-nwpk6" May 28 16:10:12.145783 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:12.145722 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df3fa509-4d2c-411d-805f-e11e37048426-console-oauth-config\") pod \"console-58f6d868f4-nwpk6\" (UID: \"df3fa509-4d2c-411d-805f-e11e37048426\") " pod="openshift-console/console-58f6d868f4-nwpk6" May 28 16:10:12.145783 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:12.145757 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df3fa509-4d2c-411d-805f-e11e37048426-console-config\") pod \"console-58f6d868f4-nwpk6\" (UID: \"df3fa509-4d2c-411d-805f-e11e37048426\") " pod="openshift-console/console-58f6d868f4-nwpk6" May 28 16:10:12.246311 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:12.246240 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kx9gm\" (UniqueName: \"kubernetes.io/projected/df3fa509-4d2c-411d-805f-e11e37048426-kube-api-access-kx9gm\") pod \"console-58f6d868f4-nwpk6\" (UID: \"df3fa509-4d2c-411d-805f-e11e37048426\") " pod="openshift-console/console-58f6d868f4-nwpk6" May 28 16:10:12.246501 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:12.246319 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df3fa509-4d2c-411d-805f-e11e37048426-console-serving-cert\") pod \"console-58f6d868f4-nwpk6\" (UID: \"df3fa509-4d2c-411d-805f-e11e37048426\") " pod="openshift-console/console-58f6d868f4-nwpk6" May 28 16:10:12.246501 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:12.246344 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df3fa509-4d2c-411d-805f-e11e37048426-console-oauth-config\") pod \"console-58f6d868f4-nwpk6\" (UID: \"df3fa509-4d2c-411d-805f-e11e37048426\") " pod="openshift-console/console-58f6d868f4-nwpk6" May 28 16:10:12.246501 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:12.246368 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df3fa509-4d2c-411d-805f-e11e37048426-console-config\") pod \"console-58f6d868f4-nwpk6\" (UID: \"df3fa509-4d2c-411d-805f-e11e37048426\") " pod="openshift-console/console-58f6d868f4-nwpk6" May 28 16:10:12.246501 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:12.246439 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df3fa509-4d2c-411d-805f-e11e37048426-oauth-serving-cert\") pod \"console-58f6d868f4-nwpk6\" (UID: \"df3fa509-4d2c-411d-805f-e11e37048426\") " pod="openshift-console/console-58f6d868f4-nwpk6" May 28 16:10:12.246501 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:12.246493 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df3fa509-4d2c-411d-805f-e11e37048426-service-ca\") pod \"console-58f6d868f4-nwpk6\" (UID: \"df3fa509-4d2c-411d-805f-e11e37048426\") " pod="openshift-console/console-58f6d868f4-nwpk6" May 28 16:10:12.247135 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:12.247111 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df3fa509-4d2c-411d-805f-e11e37048426-service-ca\") pod \"console-58f6d868f4-nwpk6\" (UID: \"df3fa509-4d2c-411d-805f-e11e37048426\") " pod="openshift-console/console-58f6d868f4-nwpk6" May 28 16:10:12.247266 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:12.247243 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df3fa509-4d2c-411d-805f-e11e37048426-oauth-serving-cert\") pod \"console-58f6d868f4-nwpk6\" (UID: \"df3fa509-4d2c-411d-805f-e11e37048426\") " pod="openshift-console/console-58f6d868f4-nwpk6" May 28 16:10:12.247336 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:12.247244 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df3fa509-4d2c-411d-805f-e11e37048426-console-config\") pod \"console-58f6d868f4-nwpk6\" (UID: \"df3fa509-4d2c-411d-805f-e11e37048426\") " pod="openshift-console/console-58f6d868f4-nwpk6" May 28 16:10:12.248946 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:12.248915 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df3fa509-4d2c-411d-805f-e11e37048426-console-oauth-config\") pod \"console-58f6d868f4-nwpk6\" (UID: \"df3fa509-4d2c-411d-805f-e11e37048426\") " pod="openshift-console/console-58f6d868f4-nwpk6" May 28 16:10:12.249048 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:12.248990 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df3fa509-4d2c-411d-805f-e11e37048426-console-serving-cert\") pod \"console-58f6d868f4-nwpk6\" (UID: \"df3fa509-4d2c-411d-805f-e11e37048426\") " pod="openshift-console/console-58f6d868f4-nwpk6" May 28 16:10:12.254724 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:12.254701 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kx9gm\" (UniqueName: \"kubernetes.io/projected/df3fa509-4d2c-411d-805f-e11e37048426-kube-api-access-kx9gm\") pod \"console-58f6d868f4-nwpk6\" (UID: \"df3fa509-4d2c-411d-805f-e11e37048426\") " pod="openshift-console/console-58f6d868f4-nwpk6" May 28 16:10:12.392855 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:12.392753 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58f6d868f4-nwpk6" May 28 16:10:12.522400 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:12.522371 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58f6d868f4-nwpk6"] May 28 16:10:12.528323 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:10:12.528262 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf3fa509_4d2c_411d_805f_e11e37048426.slice/crio-14be1204eb186aa6565a2667cb4f0167570872585ed6cf1c593cae86bc6ef9b5 WatchSource:0}: Error finding container 14be1204eb186aa6565a2667cb4f0167570872585ed6cf1c593cae86bc6ef9b5: Status 404 returned error can't find the container with id 14be1204eb186aa6565a2667cb4f0167570872585ed6cf1c593cae86bc6ef9b5 May 28 16:10:12.840357 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:12.840317 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58f6d868f4-nwpk6" event={"ID":"df3fa509-4d2c-411d-805f-e11e37048426","Type":"ContainerStarted","Data":"14be1204eb186aa6565a2667cb4f0167570872585ed6cf1c593cae86bc6ef9b5"} May 28 16:10:17.854035 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:17.853999 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58f6d868f4-nwpk6" event={"ID":"df3fa509-4d2c-411d-805f-e11e37048426","Type":"ContainerStarted","Data":"36596d8f3403b101b87345529315cbd81518221426422ab75a5e531fbcf00ad4"} May 28 16:10:17.873015 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:17.872923 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-58f6d868f4-nwpk6" podStartSLOduration=1.415215823 podStartE2EDuration="5.872908046s" podCreationTimestamp="2026-05-28 16:10:12 +0000 UTC" firstStartedPulling="2026-05-28 16:10:12.52969721 +0000 UTC m=+151.756727758" lastFinishedPulling="2026-05-28 16:10:16.987389432 +0000 UTC m=+156.214419981" observedRunningTime="2026-05-28 16:10:17.871993671 +0000 UTC m=+157.099024228" watchObservedRunningTime="2026-05-28 16:10:17.872908046 +0000 UTC m=+157.099938602" May 28 16:10:18.193450 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:10:18.193339 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-cwtzt" podUID="8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5" May 28 16:10:18.198524 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:10:18.198491 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-zg8g7" podUID="b104af3f-b526-4a24-956d-4d85f22a39c1" May 28 16:10:18.723573 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:18.723527 2566 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj" podUID="9f194b55-2d06-4c01-9d01-65d0587862be" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" May 28 16:10:18.856530 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:18.856490 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zg8g7" May 28 16:10:18.856530 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:18.856505 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cwtzt" May 28 16:10:22.393198 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:22.393162 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-58f6d868f4-nwpk6" May 28 16:10:22.393198 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:22.393206 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-58f6d868f4-nwpk6" May 28 16:10:22.398132 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:22.398115 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-58f6d868f4-nwpk6" May 28 16:10:22.870269 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:22.870234 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-58f6d868f4-nwpk6" May 28 16:10:23.042724 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:23.042690 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b104af3f-b526-4a24-956d-4d85f22a39c1-cert\") pod \"ingress-canary-zg8g7\" (UID: \"b104af3f-b526-4a24-956d-4d85f22a39c1\") " pod="openshift-ingress-canary/ingress-canary-zg8g7" May 28 16:10:23.042904 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:23.042735 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-metrics-tls\") pod \"dns-default-cwtzt\" (UID: \"8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5\") " pod="openshift-dns/dns-default-cwtzt" May 28 16:10:23.045164 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:23.045138 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5-metrics-tls\") pod \"dns-default-cwtzt\" (UID: \"8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5\") " pod="openshift-dns/dns-default-cwtzt" May 28 16:10:23.045318 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:23.045248 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b104af3f-b526-4a24-956d-4d85f22a39c1-cert\") pod \"ingress-canary-zg8g7\" (UID: \"b104af3f-b526-4a24-956d-4d85f22a39c1\") " pod="openshift-ingress-canary/ingress-canary-zg8g7" May 28 16:10:23.059693 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:23.059670 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ckq8r\"" May 28 16:10:23.059816 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:23.059670 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rfj6w\"" May 28 16:10:23.067905 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:23.067887 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zg8g7" May 28 16:10:23.068019 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:23.067918 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cwtzt" May 28 16:10:23.203378 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:23.203348 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zg8g7"] May 28 16:10:23.206396 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:10:23.206370 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb104af3f_b526_4a24_956d_4d85f22a39c1.slice/crio-05493947e68bf34a803456c8cd0596467365633055c3fd4f22cb7ee3bd73cb15 WatchSource:0}: Error finding container 05493947e68bf34a803456c8cd0596467365633055c3fd4f22cb7ee3bd73cb15: Status 404 returned error can't find the container with id 05493947e68bf34a803456c8cd0596467365633055c3fd4f22cb7ee3bd73cb15 May 28 16:10:23.221017 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:23.220986 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cwtzt"] May 28 16:10:23.224070 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:10:23.224035 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d9c00b9_39ab_4064_9cbc_7bc8ce8cfde5.slice/crio-2576ef604fa4085d2c37e5218de205ce3da41b8241dd960830e2a3593a275d3e WatchSource:0}: Error finding container 2576ef604fa4085d2c37e5218de205ce3da41b8241dd960830e2a3593a275d3e: Status 404 returned error can't find the container with id 2576ef604fa4085d2c37e5218de205ce3da41b8241dd960830e2a3593a275d3e May 28 16:10:23.871982 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:23.871935 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zg8g7" event={"ID":"b104af3f-b526-4a24-956d-4d85f22a39c1","Type":"ContainerStarted","Data":"05493947e68bf34a803456c8cd0596467365633055c3fd4f22cb7ee3bd73cb15"} May 28 16:10:23.873220 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:23.873190 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cwtzt" event={"ID":"8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5","Type":"ContainerStarted","Data":"2576ef604fa4085d2c37e5218de205ce3da41b8241dd960830e2a3593a275d3e"} May 28 16:10:25.880655 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:25.880566 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zg8g7" event={"ID":"b104af3f-b526-4a24-956d-4d85f22a39c1","Type":"ContainerStarted","Data":"c634bf73919c6b0e1636750f543e25bd24d0712d5b0fbf54f84e950300ba5475"} May 28 16:10:25.882195 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:25.882175 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cwtzt" event={"ID":"8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5","Type":"ContainerStarted","Data":"10eb3365a3520959d4101f807efc345ee3e3dd9d1dc909dc2dac87db00903505"} May 28 16:10:25.882255 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:25.882201 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cwtzt" event={"ID":"8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5","Type":"ContainerStarted","Data":"94d881ccc88b3efbe0e936fa92225155a5e47e3ec31e3dd2ebb9651fc54381fb"} May 28 16:10:25.882348 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:25.882329 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-cwtzt" May 28 16:10:25.896627 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:25.896580 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-zg8g7" podStartSLOduration=128.961003435 podStartE2EDuration="2m10.896563688s" podCreationTimestamp="2026-05-28 16:08:15 +0000 UTC" firstStartedPulling="2026-05-28 16:10:23.208136421 +0000 UTC m=+162.435166954" lastFinishedPulling="2026-05-28 16:10:25.14369667 +0000 UTC m=+164.370727207" observedRunningTime="2026-05-28 16:10:25.896374675 +0000 UTC m=+165.123405232" watchObservedRunningTime="2026-05-28 16:10:25.896563688 +0000 UTC m=+165.123594244" May 28 16:10:25.912805 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:25.912744 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-cwtzt" podStartSLOduration=128.998247448 podStartE2EDuration="2m10.912725646s" podCreationTimestamp="2026-05-28 16:08:15 +0000 UTC" firstStartedPulling="2026-05-28 16:10:23.225839262 +0000 UTC m=+162.452869796" lastFinishedPulling="2026-05-28 16:10:25.140317442 +0000 UTC m=+164.367347994" observedRunningTime="2026-05-28 16:10:25.911901041 +0000 UTC m=+165.138931614" watchObservedRunningTime="2026-05-28 16:10:25.912725646 +0000 UTC m=+165.139756203" May 28 16:10:26.886590 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:26.886505 2566 generic.go:358] "Generic (PLEG): container finished" podID="3e7f8fbd-4455-4478-8f61-8ded41ddd032" containerID="d69603f8367c0871108a1f991f86cd27c631bf4eda0803d9596bd8d999fd5e33" exitCode=0 May 28 16:10:26.886590 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:26.886578 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-qc7zn" event={"ID":"3e7f8fbd-4455-4478-8f61-8ded41ddd032","Type":"ContainerDied","Data":"d69603f8367c0871108a1f991f86cd27c631bf4eda0803d9596bd8d999fd5e33"} May 28 16:10:26.887053 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:26.887035 2566 scope.go:117] "RemoveContainer" containerID="d69603f8367c0871108a1f991f86cd27c631bf4eda0803d9596bd8d999fd5e33" May 28 16:10:27.891132 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:27.891097 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-qc7zn" event={"ID":"3e7f8fbd-4455-4478-8f61-8ded41ddd032","Type":"ContainerStarted","Data":"277d29f829bd379a9201567982186af0bb54779f954a63f843e28294d40990fa"} May 28 16:10:28.723523 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:28.723481 2566 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj" podUID="9f194b55-2d06-4c01-9d01-65d0587862be" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" May 28 16:10:32.178887 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:32.178851 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58f6d868f4-nwpk6"] May 28 16:10:35.889418 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:35.889390 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-cwtzt" May 28 16:10:38.723274 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:38.723231 2566 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj" podUID="9f194b55-2d06-4c01-9d01-65d0587862be" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" May 28 16:10:38.723651 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:38.723346 2566 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj" May 28 16:10:38.723923 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:38.723904 2566 kuberuntime_manager.go:1107] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"8fa59934f8ae209a0a182a5d4c676269edbcff9c1e04f71be051efdcc663e78b"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj" containerMessage="Container service-proxy failed liveness probe, will be restarted" May 28 16:10:38.723969 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:38.723948 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj" podUID="9f194b55-2d06-4c01-9d01-65d0587862be" containerName="service-proxy" containerID="cri-o://8fa59934f8ae209a0a182a5d4c676269edbcff9c1e04f71be051efdcc663e78b" gracePeriod=30 May 28 16:10:38.920860 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:38.920825 2566 generic.go:358] "Generic (PLEG): container finished" podID="9f194b55-2d06-4c01-9d01-65d0587862be" containerID="8fa59934f8ae209a0a182a5d4c676269edbcff9c1e04f71be051efdcc663e78b" exitCode=2 May 28 16:10:38.921002 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:38.920899 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj" event={"ID":"9f194b55-2d06-4c01-9d01-65d0587862be","Type":"ContainerDied","Data":"8fa59934f8ae209a0a182a5d4c676269edbcff9c1e04f71be051efdcc663e78b"} May 28 16:10:38.921002 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:38.920942 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558455b789-7s7hj" event={"ID":"9f194b55-2d06-4c01-9d01-65d0587862be","Type":"ContainerStarted","Data":"2e8ba0cf0c1d97815f9308ded727837c9f7d45bf170c1a887d5d879e7fab924b"} May 28 16:10:57.198754 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:57.198695 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-58f6d868f4-nwpk6" podUID="df3fa509-4d2c-411d-805f-e11e37048426" containerName="console" containerID="cri-o://36596d8f3403b101b87345529315cbd81518221426422ab75a5e531fbcf00ad4" gracePeriod=15 May 28 16:10:57.448030 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:57.448006 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-58f6d868f4-nwpk6_df3fa509-4d2c-411d-805f-e11e37048426/console/0.log" May 28 16:10:57.448156 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:57.448081 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58f6d868f4-nwpk6" May 28 16:10:57.500748 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:57.500706 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df3fa509-4d2c-411d-805f-e11e37048426-console-serving-cert\") pod \"df3fa509-4d2c-411d-805f-e11e37048426\" (UID: \"df3fa509-4d2c-411d-805f-e11e37048426\") " May 28 16:10:57.500949 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:57.500764 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df3fa509-4d2c-411d-805f-e11e37048426-console-oauth-config\") pod \"df3fa509-4d2c-411d-805f-e11e37048426\" (UID: \"df3fa509-4d2c-411d-805f-e11e37048426\") " May 28 16:10:57.500949 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:57.500798 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df3fa509-4d2c-411d-805f-e11e37048426-oauth-serving-cert\") pod \"df3fa509-4d2c-411d-805f-e11e37048426\" (UID: \"df3fa509-4d2c-411d-805f-e11e37048426\") " May 28 16:10:57.500949 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:57.500814 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kx9gm\" (UniqueName: \"kubernetes.io/projected/df3fa509-4d2c-411d-805f-e11e37048426-kube-api-access-kx9gm\") pod \"df3fa509-4d2c-411d-805f-e11e37048426\" (UID: \"df3fa509-4d2c-411d-805f-e11e37048426\") " May 28 16:10:57.500949 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:57.500836 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df3fa509-4d2c-411d-805f-e11e37048426-service-ca\") pod \"df3fa509-4d2c-411d-805f-e11e37048426\" (UID: \"df3fa509-4d2c-411d-805f-e11e37048426\") " May 28 16:10:57.501154 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:57.501034 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df3fa509-4d2c-411d-805f-e11e37048426-console-config\") pod \"df3fa509-4d2c-411d-805f-e11e37048426\" (UID: \"df3fa509-4d2c-411d-805f-e11e37048426\") " May 28 16:10:57.501217 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:57.501181 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df3fa509-4d2c-411d-805f-e11e37048426-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "df3fa509-4d2c-411d-805f-e11e37048426" (UID: "df3fa509-4d2c-411d-805f-e11e37048426"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 16:10:57.501273 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:57.501224 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df3fa509-4d2c-411d-805f-e11e37048426-service-ca" (OuterVolumeSpecName: "service-ca") pod "df3fa509-4d2c-411d-805f-e11e37048426" (UID: "df3fa509-4d2c-411d-805f-e11e37048426"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 16:10:57.501348 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:57.501307 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df3fa509-4d2c-411d-805f-e11e37048426-oauth-serving-cert\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" May 28 16:10:57.501348 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:57.501329 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df3fa509-4d2c-411d-805f-e11e37048426-service-ca\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" May 28 16:10:57.501472 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:57.501448 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df3fa509-4d2c-411d-805f-e11e37048426-console-config" (OuterVolumeSpecName: "console-config") pod "df3fa509-4d2c-411d-805f-e11e37048426" (UID: "df3fa509-4d2c-411d-805f-e11e37048426"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 16:10:57.503461 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:57.503435 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df3fa509-4d2c-411d-805f-e11e37048426-kube-api-access-kx9gm" (OuterVolumeSpecName: "kube-api-access-kx9gm") pod "df3fa509-4d2c-411d-805f-e11e37048426" (UID: "df3fa509-4d2c-411d-805f-e11e37048426"). InnerVolumeSpecName "kube-api-access-kx9gm". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 16:10:57.503461 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:57.503454 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df3fa509-4d2c-411d-805f-e11e37048426-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "df3fa509-4d2c-411d-805f-e11e37048426" (UID: "df3fa509-4d2c-411d-805f-e11e37048426"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 16:10:57.503590 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:57.503571 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df3fa509-4d2c-411d-805f-e11e37048426-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "df3fa509-4d2c-411d-805f-e11e37048426" (UID: "df3fa509-4d2c-411d-805f-e11e37048426"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 16:10:57.602033 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:57.601982 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df3fa509-4d2c-411d-805f-e11e37048426-console-config\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" May 28 16:10:57.602033 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:57.602028 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df3fa509-4d2c-411d-805f-e11e37048426-console-serving-cert\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" May 28 16:10:57.602033 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:57.602038 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df3fa509-4d2c-411d-805f-e11e37048426-console-oauth-config\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" May 28 16:10:57.602033 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:57.602047 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kx9gm\" (UniqueName: \"kubernetes.io/projected/df3fa509-4d2c-411d-805f-e11e37048426-kube-api-access-kx9gm\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" May 28 16:10:57.970672 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:57.970599 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-58f6d868f4-nwpk6_df3fa509-4d2c-411d-805f-e11e37048426/console/0.log" May 28 16:10:57.970672 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:57.970636 2566 generic.go:358] "Generic (PLEG): container finished" podID="df3fa509-4d2c-411d-805f-e11e37048426" containerID="36596d8f3403b101b87345529315cbd81518221426422ab75a5e531fbcf00ad4" exitCode=2 May 28 16:10:57.970849 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:57.970706 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58f6d868f4-nwpk6" May 28 16:10:57.970849 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:57.970705 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58f6d868f4-nwpk6" event={"ID":"df3fa509-4d2c-411d-805f-e11e37048426","Type":"ContainerDied","Data":"36596d8f3403b101b87345529315cbd81518221426422ab75a5e531fbcf00ad4"} May 28 16:10:57.970849 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:57.970808 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58f6d868f4-nwpk6" event={"ID":"df3fa509-4d2c-411d-805f-e11e37048426","Type":"ContainerDied","Data":"14be1204eb186aa6565a2667cb4f0167570872585ed6cf1c593cae86bc6ef9b5"} May 28 16:10:57.970849 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:57.970824 2566 scope.go:117] "RemoveContainer" containerID="36596d8f3403b101b87345529315cbd81518221426422ab75a5e531fbcf00ad4" May 28 16:10:57.978799 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:57.978774 2566 scope.go:117] "RemoveContainer" containerID="36596d8f3403b101b87345529315cbd81518221426422ab75a5e531fbcf00ad4" May 28 16:10:57.979103 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:10:57.979085 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36596d8f3403b101b87345529315cbd81518221426422ab75a5e531fbcf00ad4\": container with ID starting with 36596d8f3403b101b87345529315cbd81518221426422ab75a5e531fbcf00ad4 not found: ID does not exist" containerID="36596d8f3403b101b87345529315cbd81518221426422ab75a5e531fbcf00ad4" May 28 16:10:57.979186 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:57.979111 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36596d8f3403b101b87345529315cbd81518221426422ab75a5e531fbcf00ad4"} err="failed to get container status \"36596d8f3403b101b87345529315cbd81518221426422ab75a5e531fbcf00ad4\": rpc error: code = NotFound desc = could not find container \"36596d8f3403b101b87345529315cbd81518221426422ab75a5e531fbcf00ad4\": container with ID starting with 36596d8f3403b101b87345529315cbd81518221426422ab75a5e531fbcf00ad4 not found: ID does not exist" May 28 16:10:57.990143 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:57.990121 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58f6d868f4-nwpk6"] May 28 16:10:57.995196 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:57.995169 2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-58f6d868f4-nwpk6"] May 28 16:10:59.336588 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:10:59.336556 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df3fa509-4d2c-411d-805f-e11e37048426" path="/var/lib/kubelet/pods/df3fa509-4d2c-411d-805f-e11e37048426/volumes" May 28 16:12:41.193598 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:41.193565 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-db77q_d07065a1-27ff-4f50-9b8e-88d35f69bc89/ovn-acl-logging/0.log" May 28 16:12:41.194083 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:41.193809 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-db77q_d07065a1-27ff-4f50-9b8e-88d35f69bc89/ovn-acl-logging/0.log" May 28 16:12:41.204974 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:41.204949 2566 kubelet.go:1628] "Image garbage collection succeeded" May 28 16:12:49.202597 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:49.202564 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-wjgg7"] May 28 16:12:49.204834 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:49.202790 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df3fa509-4d2c-411d-805f-e11e37048426" containerName="console" May 28 16:12:49.204834 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:49.202800 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="df3fa509-4d2c-411d-805f-e11e37048426" containerName="console" May 28 16:12:49.204834 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:49.202845 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="df3fa509-4d2c-411d-805f-e11e37048426" containerName="console" May 28 16:12:49.205654 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:49.205637 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-wjgg7" May 28 16:12:49.208579 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:49.208554 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" May 28 16:12:49.208682 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:49.208588 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" May 28 16:12:49.208887 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:49.208872 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-66dvs\"" May 28 16:12:49.217952 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:49.217927 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-wjgg7"] May 28 16:12:49.370205 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:49.370164 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r796t\" (UniqueName: \"kubernetes.io/projected/90d3115b-329f-430b-8a8e-9ce609687270-kube-api-access-r796t\") pod \"cert-manager-operator-controller-manager-54b9655956-wjgg7\" (UID: \"90d3115b-329f-430b-8a8e-9ce609687270\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-wjgg7" May 28 16:12:49.370411 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:49.370219 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90d3115b-329f-430b-8a8e-9ce609687270-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-wjgg7\" (UID: \"90d3115b-329f-430b-8a8e-9ce609687270\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-wjgg7" May 28 16:12:49.471311 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:49.471208 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r796t\" (UniqueName: \"kubernetes.io/projected/90d3115b-329f-430b-8a8e-9ce609687270-kube-api-access-r796t\") pod \"cert-manager-operator-controller-manager-54b9655956-wjgg7\" (UID: \"90d3115b-329f-430b-8a8e-9ce609687270\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-wjgg7" May 28 16:12:49.471311 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:49.471268 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90d3115b-329f-430b-8a8e-9ce609687270-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-wjgg7\" (UID: \"90d3115b-329f-430b-8a8e-9ce609687270\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-wjgg7" May 28 16:12:49.471704 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:49.471685 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90d3115b-329f-430b-8a8e-9ce609687270-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-wjgg7\" (UID: \"90d3115b-329f-430b-8a8e-9ce609687270\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-wjgg7" May 28 16:12:49.480501 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:49.480476 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r796t\" (UniqueName: \"kubernetes.io/projected/90d3115b-329f-430b-8a8e-9ce609687270-kube-api-access-r796t\") pod \"cert-manager-operator-controller-manager-54b9655956-wjgg7\" (UID: \"90d3115b-329f-430b-8a8e-9ce609687270\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-wjgg7" May 28 16:12:49.514785 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:49.514755 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-wjgg7" May 28 16:12:49.645968 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:49.645925 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-wjgg7"] May 28 16:12:49.649683 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:12:49.649648 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90d3115b_329f_430b_8a8e_9ce609687270.slice/crio-c2526959205e473a0767611331cf959845c324cc85b5c0966819a02309ac4e24 WatchSource:0}: Error finding container c2526959205e473a0767611331cf959845c324cc85b5c0966819a02309ac4e24: Status 404 returned error can't find the container with id c2526959205e473a0767611331cf959845c324cc85b5c0966819a02309ac4e24 May 28 16:12:49.652071 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:49.652055 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 28 16:12:50.259616 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:50.259578 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-wjgg7" event={"ID":"90d3115b-329f-430b-8a8e-9ce609687270","Type":"ContainerStarted","Data":"c2526959205e473a0767611331cf959845c324cc85b5c0966819a02309ac4e24"} May 28 16:12:53.272583 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:53.272546 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-wjgg7" event={"ID":"90d3115b-329f-430b-8a8e-9ce609687270","Type":"ContainerStarted","Data":"ed4bca1380b99f4e8fc62ef268ea6f935cf3a3be6d1b68f3ac7bf2bf867e554b"} May 28 16:12:53.294482 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:53.294427 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-wjgg7" podStartSLOduration=1.6573789589999999 podStartE2EDuration="4.294411501s" podCreationTimestamp="2026-05-28 16:12:49 +0000 UTC" firstStartedPulling="2026-05-28 16:12:49.652204956 +0000 UTC m=+308.879235490" lastFinishedPulling="2026-05-28 16:12:52.289237484 +0000 UTC m=+311.516268032" observedRunningTime="2026-05-28 16:12:53.292531482 +0000 UTC m=+312.519562039" watchObservedRunningTime="2026-05-28 16:12:53.294411501 +0000 UTC m=+312.521442057" May 28 16:12:58.203353 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:58.203318 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-66lp2"] May 28 16:12:58.206068 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:58.206051 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-68b757865b-66lp2" May 28 16:12:58.208477 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:58.208455 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" May 28 16:12:58.208669 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:58.208647 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-z7w56\"" May 28 16:12:58.208737 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:58.208652 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" May 28 16:12:58.215029 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:58.214994 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-66lp2"] May 28 16:12:58.337065 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:58.337025 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glqft\" (UniqueName: \"kubernetes.io/projected/6ef285d8-093d-46d6-9ce2-4461eb70948b-kube-api-access-glqft\") pod \"cert-manager-cainjector-68b757865b-66lp2\" (UID: \"6ef285d8-093d-46d6-9ce2-4461eb70948b\") " pod="cert-manager/cert-manager-cainjector-68b757865b-66lp2" May 28 16:12:58.337247 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:58.337099 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6ef285d8-093d-46d6-9ce2-4461eb70948b-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-66lp2\" (UID: \"6ef285d8-093d-46d6-9ce2-4461eb70948b\") " pod="cert-manager/cert-manager-cainjector-68b757865b-66lp2" May 28 16:12:58.437512 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:58.437471 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-glqft\" (UniqueName: \"kubernetes.io/projected/6ef285d8-093d-46d6-9ce2-4461eb70948b-kube-api-access-glqft\") pod \"cert-manager-cainjector-68b757865b-66lp2\" (UID: \"6ef285d8-093d-46d6-9ce2-4461eb70948b\") " pod="cert-manager/cert-manager-cainjector-68b757865b-66lp2" May 28 16:12:58.437684 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:58.437526 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6ef285d8-093d-46d6-9ce2-4461eb70948b-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-66lp2\" (UID: \"6ef285d8-093d-46d6-9ce2-4461eb70948b\") " pod="cert-manager/cert-manager-cainjector-68b757865b-66lp2" May 28 16:12:58.445333 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:58.445266 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6ef285d8-093d-46d6-9ce2-4461eb70948b-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-66lp2\" (UID: \"6ef285d8-093d-46d6-9ce2-4461eb70948b\") " pod="cert-manager/cert-manager-cainjector-68b757865b-66lp2" May 28 16:12:58.445573 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:58.445553 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-glqft\" (UniqueName: \"kubernetes.io/projected/6ef285d8-093d-46d6-9ce2-4461eb70948b-kube-api-access-glqft\") pod \"cert-manager-cainjector-68b757865b-66lp2\" (UID: \"6ef285d8-093d-46d6-9ce2-4461eb70948b\") " pod="cert-manager/cert-manager-cainjector-68b757865b-66lp2" May 28 16:12:58.515430 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:58.515392 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-68b757865b-66lp2" May 28 16:12:58.662415 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:58.662376 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-66lp2"] May 28 16:12:58.665347 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:12:58.665318 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6ef285d8_093d_46d6_9ce2_4461eb70948b.slice/crio-03f61fc74931dfd382e6eecf85b84779a5c4a396f6da390ad2e32670218f1068 WatchSource:0}: Error finding container 03f61fc74931dfd382e6eecf85b84779a5c4a396f6da390ad2e32670218f1068: Status 404 returned error can't find the container with id 03f61fc74931dfd382e6eecf85b84779a5c4a396f6da390ad2e32670218f1068 May 28 16:12:59.287594 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:12:59.287554 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-68b757865b-66lp2" event={"ID":"6ef285d8-093d-46d6-9ce2-4461eb70948b","Type":"ContainerStarted","Data":"03f61fc74931dfd382e6eecf85b84779a5c4a396f6da390ad2e32670218f1068"} May 28 16:13:02.296927 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:02.296888 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-68b757865b-66lp2" event={"ID":"6ef285d8-093d-46d6-9ce2-4461eb70948b","Type":"ContainerStarted","Data":"a6e5322409a6122334c2a6e85664076ee7365cb449a87d737a27387ecf1527ce"} May 28 16:13:02.313851 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:02.313796 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-68b757865b-66lp2" podStartSLOduration=1.434878305 podStartE2EDuration="4.313772966s" podCreationTimestamp="2026-05-28 16:12:58 +0000 UTC" firstStartedPulling="2026-05-28 16:12:58.667158495 +0000 UTC m=+317.894189030" lastFinishedPulling="2026-05-28 16:13:01.546053149 +0000 UTC m=+320.773083691" observedRunningTime="2026-05-28 16:13:02.312588505 +0000 UTC m=+321.539619077" watchObservedRunningTime="2026-05-28 16:13:02.313772966 +0000 UTC m=+321.540803522" May 28 16:13:28.577824 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:28.577722 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-95cfdd646-mpl9q"] May 28 16:13:28.581703 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:28.581680 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-95cfdd646-mpl9q" May 28 16:13:28.584296 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:28.584253 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" May 28 16:13:28.584296 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:28.584269 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" May 28 16:13:28.584513 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:28.584348 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" May 28 16:13:28.584513 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:28.584421 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" May 28 16:13:28.584513 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:28.584430 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-pb8rq\"" May 28 16:13:28.590598 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:28.590576 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-95cfdd646-mpl9q"] May 28 16:13:28.657180 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:28.657140 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/73ef29d0-0d5c-4419-8b95-b5fd0b0ec7aa-webhook-cert\") pod \"opendatahub-operator-controller-manager-95cfdd646-mpl9q\" (UID: \"73ef29d0-0d5c-4419-8b95-b5fd0b0ec7aa\") " pod="opendatahub/opendatahub-operator-controller-manager-95cfdd646-mpl9q" May 28 16:13:28.657180 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:28.657183 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/73ef29d0-0d5c-4419-8b95-b5fd0b0ec7aa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-95cfdd646-mpl9q\" (UID: \"73ef29d0-0d5c-4419-8b95-b5fd0b0ec7aa\") " pod="opendatahub/opendatahub-operator-controller-manager-95cfdd646-mpl9q" May 28 16:13:28.657416 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:28.657232 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rmvg\" (UniqueName: \"kubernetes.io/projected/73ef29d0-0d5c-4419-8b95-b5fd0b0ec7aa-kube-api-access-9rmvg\") pod \"opendatahub-operator-controller-manager-95cfdd646-mpl9q\" (UID: \"73ef29d0-0d5c-4419-8b95-b5fd0b0ec7aa\") " pod="opendatahub/opendatahub-operator-controller-manager-95cfdd646-mpl9q" May 28 16:13:28.757825 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:28.757778 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/73ef29d0-0d5c-4419-8b95-b5fd0b0ec7aa-webhook-cert\") pod \"opendatahub-operator-controller-manager-95cfdd646-mpl9q\" (UID: \"73ef29d0-0d5c-4419-8b95-b5fd0b0ec7aa\") " pod="opendatahub/opendatahub-operator-controller-manager-95cfdd646-mpl9q" May 28 16:13:28.758006 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:28.757836 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/73ef29d0-0d5c-4419-8b95-b5fd0b0ec7aa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-95cfdd646-mpl9q\" (UID: \"73ef29d0-0d5c-4419-8b95-b5fd0b0ec7aa\") " pod="opendatahub/opendatahub-operator-controller-manager-95cfdd646-mpl9q" May 28 16:13:28.758006 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:28.757879 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9rmvg\" (UniqueName: \"kubernetes.io/projected/73ef29d0-0d5c-4419-8b95-b5fd0b0ec7aa-kube-api-access-9rmvg\") pod \"opendatahub-operator-controller-manager-95cfdd646-mpl9q\" (UID: \"73ef29d0-0d5c-4419-8b95-b5fd0b0ec7aa\") " pod="opendatahub/opendatahub-operator-controller-manager-95cfdd646-mpl9q" May 28 16:13:28.760655 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:28.760630 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/73ef29d0-0d5c-4419-8b95-b5fd0b0ec7aa-webhook-cert\") pod \"opendatahub-operator-controller-manager-95cfdd646-mpl9q\" (UID: \"73ef29d0-0d5c-4419-8b95-b5fd0b0ec7aa\") " pod="opendatahub/opendatahub-operator-controller-manager-95cfdd646-mpl9q" May 28 16:13:28.760711 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:28.760630 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/73ef29d0-0d5c-4419-8b95-b5fd0b0ec7aa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-95cfdd646-mpl9q\" (UID: \"73ef29d0-0d5c-4419-8b95-b5fd0b0ec7aa\") " pod="opendatahub/opendatahub-operator-controller-manager-95cfdd646-mpl9q" May 28 16:13:28.766334 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:28.766312 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rmvg\" (UniqueName: \"kubernetes.io/projected/73ef29d0-0d5c-4419-8b95-b5fd0b0ec7aa-kube-api-access-9rmvg\") pod \"opendatahub-operator-controller-manager-95cfdd646-mpl9q\" (UID: \"73ef29d0-0d5c-4419-8b95-b5fd0b0ec7aa\") " pod="opendatahub/opendatahub-operator-controller-manager-95cfdd646-mpl9q" May 28 16:13:28.893240 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:28.893146 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-95cfdd646-mpl9q" May 28 16:13:29.068491 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:29.068453 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-95cfdd646-mpl9q"] May 28 16:13:29.072813 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:13:29.072785 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73ef29d0_0d5c_4419_8b95_b5fd0b0ec7aa.slice/crio-b6376091824bd94c4554c8388a337a540279ef7e4845a111a5964829f9f19d7f WatchSource:0}: Error finding container b6376091824bd94c4554c8388a337a540279ef7e4845a111a5964829f9f19d7f: Status 404 returned error can't find the container with id b6376091824bd94c4554c8388a337a540279ef7e4845a111a5964829f9f19d7f May 28 16:13:29.368260 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:29.368222 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-95cfdd646-mpl9q" event={"ID":"73ef29d0-0d5c-4419-8b95-b5fd0b0ec7aa","Type":"ContainerStarted","Data":"b6376091824bd94c4554c8388a337a540279ef7e4845a111a5964829f9f19d7f"} May 28 16:13:32.379587 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:32.379541 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-95cfdd646-mpl9q" event={"ID":"73ef29d0-0d5c-4419-8b95-b5fd0b0ec7aa","Type":"ContainerStarted","Data":"d2c697a6df4126185b9e5a279a3c9d83efa0b89208b870841df0203803e8de61"} May 28 16:13:32.379983 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:32.379673 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-95cfdd646-mpl9q" May 28 16:13:32.397593 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:32.397537 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-95cfdd646-mpl9q" podStartSLOduration=1.942454884 podStartE2EDuration="4.397523107s" podCreationTimestamp="2026-05-28 16:13:28 +0000 UTC" firstStartedPulling="2026-05-28 16:13:29.074753505 +0000 UTC m=+348.301784038" lastFinishedPulling="2026-05-28 16:13:31.529821726 +0000 UTC m=+350.756852261" observedRunningTime="2026-05-28 16:13:32.39656331 +0000 UTC m=+351.623593866" watchObservedRunningTime="2026-05-28 16:13:32.397523107 +0000 UTC m=+351.624553644" May 28 16:13:34.541575 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:34.541538 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-677666f8d9-2rlmg"] May 28 16:13:34.545016 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:34.544987 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-677666f8d9-2rlmg" May 28 16:13:34.547813 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:34.547795 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" May 28 16:13:34.548949 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:34.548922 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" May 28 16:13:34.548949 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:34.548937 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" May 28 16:13:34.549143 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:34.548961 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" May 28 16:13:34.549143 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:34.548990 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" May 28 16:13:34.549143 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:34.549061 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-5v92j\"" May 28 16:13:34.557386 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:34.557358 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-677666f8d9-2rlmg"] May 28 16:13:34.601857 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:34.601826 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/03e20f78-fb09-4536-95cd-3eddaaf2615f-manager-config\") pod \"lws-controller-manager-677666f8d9-2rlmg\" (UID: \"03e20f78-fb09-4536-95cd-3eddaaf2615f\") " pod="openshift-lws-operator/lws-controller-manager-677666f8d9-2rlmg" May 28 16:13:34.601857 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:34.601866 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/03e20f78-fb09-4536-95cd-3eddaaf2615f-metrics-cert\") pod \"lws-controller-manager-677666f8d9-2rlmg\" (UID: \"03e20f78-fb09-4536-95cd-3eddaaf2615f\") " pod="openshift-lws-operator/lws-controller-manager-677666f8d9-2rlmg" May 28 16:13:34.602076 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:34.601885 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jnwf\" (UniqueName: \"kubernetes.io/projected/03e20f78-fb09-4536-95cd-3eddaaf2615f-kube-api-access-9jnwf\") pod \"lws-controller-manager-677666f8d9-2rlmg\" (UID: \"03e20f78-fb09-4536-95cd-3eddaaf2615f\") " pod="openshift-lws-operator/lws-controller-manager-677666f8d9-2rlmg" May 28 16:13:34.602076 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:34.601957 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/03e20f78-fb09-4536-95cd-3eddaaf2615f-cert\") pod \"lws-controller-manager-677666f8d9-2rlmg\" (UID: \"03e20f78-fb09-4536-95cd-3eddaaf2615f\") " pod="openshift-lws-operator/lws-controller-manager-677666f8d9-2rlmg" May 28 16:13:34.703100 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:34.703063 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/03e20f78-fb09-4536-95cd-3eddaaf2615f-manager-config\") pod \"lws-controller-manager-677666f8d9-2rlmg\" (UID: \"03e20f78-fb09-4536-95cd-3eddaaf2615f\") " pod="openshift-lws-operator/lws-controller-manager-677666f8d9-2rlmg" May 28 16:13:34.703100 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:34.703101 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/03e20f78-fb09-4536-95cd-3eddaaf2615f-metrics-cert\") pod \"lws-controller-manager-677666f8d9-2rlmg\" (UID: \"03e20f78-fb09-4536-95cd-3eddaaf2615f\") " pod="openshift-lws-operator/lws-controller-manager-677666f8d9-2rlmg" May 28 16:13:34.703349 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:34.703120 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9jnwf\" (UniqueName: \"kubernetes.io/projected/03e20f78-fb09-4536-95cd-3eddaaf2615f-kube-api-access-9jnwf\") pod \"lws-controller-manager-677666f8d9-2rlmg\" (UID: \"03e20f78-fb09-4536-95cd-3eddaaf2615f\") " pod="openshift-lws-operator/lws-controller-manager-677666f8d9-2rlmg" May 28 16:13:34.703349 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:34.703168 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/03e20f78-fb09-4536-95cd-3eddaaf2615f-cert\") pod \"lws-controller-manager-677666f8d9-2rlmg\" (UID: \"03e20f78-fb09-4536-95cd-3eddaaf2615f\") " pod="openshift-lws-operator/lws-controller-manager-677666f8d9-2rlmg" May 28 16:13:34.703836 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:34.703813 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/03e20f78-fb09-4536-95cd-3eddaaf2615f-manager-config\") pod \"lws-controller-manager-677666f8d9-2rlmg\" (UID: \"03e20f78-fb09-4536-95cd-3eddaaf2615f\") " pod="openshift-lws-operator/lws-controller-manager-677666f8d9-2rlmg" May 28 16:13:34.705728 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:34.705706 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/03e20f78-fb09-4536-95cd-3eddaaf2615f-cert\") pod \"lws-controller-manager-677666f8d9-2rlmg\" (UID: \"03e20f78-fb09-4536-95cd-3eddaaf2615f\") " pod="openshift-lws-operator/lws-controller-manager-677666f8d9-2rlmg" May 28 16:13:34.705825 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:34.705761 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/03e20f78-fb09-4536-95cd-3eddaaf2615f-metrics-cert\") pod \"lws-controller-manager-677666f8d9-2rlmg\" (UID: \"03e20f78-fb09-4536-95cd-3eddaaf2615f\") " pod="openshift-lws-operator/lws-controller-manager-677666f8d9-2rlmg" May 28 16:13:34.712790 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:34.712768 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jnwf\" (UniqueName: \"kubernetes.io/projected/03e20f78-fb09-4536-95cd-3eddaaf2615f-kube-api-access-9jnwf\") pod \"lws-controller-manager-677666f8d9-2rlmg\" (UID: \"03e20f78-fb09-4536-95cd-3eddaaf2615f\") " pod="openshift-lws-operator/lws-controller-manager-677666f8d9-2rlmg" May 28 16:13:34.854608 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:34.854516 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-677666f8d9-2rlmg" May 28 16:13:34.990020 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:34.989992 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-677666f8d9-2rlmg"] May 28 16:13:34.992505 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:13:34.992479 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03e20f78_fb09_4536_95cd_3eddaaf2615f.slice/crio-cf699a486477f9aeb785efa9aaab80fa1a68d300c135837319a44bba740de288 WatchSource:0}: Error finding container cf699a486477f9aeb785efa9aaab80fa1a68d300c135837319a44bba740de288: Status 404 returned error can't find the container with id cf699a486477f9aeb785efa9aaab80fa1a68d300c135837319a44bba740de288 May 28 16:13:35.389062 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:35.389015 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-677666f8d9-2rlmg" event={"ID":"03e20f78-fb09-4536-95cd-3eddaaf2615f","Type":"ContainerStarted","Data":"cf699a486477f9aeb785efa9aaab80fa1a68d300c135837319a44bba740de288"} May 28 16:13:38.400012 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:38.399974 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-677666f8d9-2rlmg" event={"ID":"03e20f78-fb09-4536-95cd-3eddaaf2615f","Type":"ContainerStarted","Data":"42fd6de6d2934d352d2551278406361bbb4a18145ec1a77fedde57b061037633"} May 28 16:13:38.400467 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:38.400210 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-677666f8d9-2rlmg" May 28 16:13:38.416349 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:38.416275 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-677666f8d9-2rlmg" podStartSLOduration=1.902998669 podStartE2EDuration="4.416260949s" podCreationTimestamp="2026-05-28 16:13:34 +0000 UTC" firstStartedPulling="2026-05-28 16:13:34.994275765 +0000 UTC m=+354.221306303" lastFinishedPulling="2026-05-28 16:13:37.507538042 +0000 UTC m=+356.734568583" observedRunningTime="2026-05-28 16:13:38.415504805 +0000 UTC m=+357.642535361" watchObservedRunningTime="2026-05-28 16:13:38.416260949 +0000 UTC m=+357.643291564" May 28 16:13:43.384438 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:43.384407 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-95cfdd646-mpl9q" May 28 16:13:49.404807 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:13:49.404772 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-677666f8d9-2rlmg" May 28 16:14:11.633567 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:11.633530 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj"] May 28 16:14:11.649505 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:11.641406 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" May 28 16:14:11.649505 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:11.646115 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-kq4pd\"" May 28 16:14:11.649505 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:11.646868 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" May 28 16:14:11.649505 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:11.647130 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" May 28 16:14:11.649505 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:11.647482 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" May 28 16:14:11.649505 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:11.649395 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj"] May 28 16:14:11.792649 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:11.792602 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f370873-429d-4cfd-9a83-be20b295ffa8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bztfxj\" (UID: \"6f370873-429d-4cfd-9a83-be20b295ffa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" May 28 16:14:11.792649 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:11.792649 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f370873-429d-4cfd-9a83-be20b295ffa8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bztfxj\" (UID: \"6f370873-429d-4cfd-9a83-be20b295ffa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" May 28 16:14:11.792873 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:11.792682 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f370873-429d-4cfd-9a83-be20b295ffa8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bztfxj\" (UID: \"6f370873-429d-4cfd-9a83-be20b295ffa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" May 28 16:14:11.792873 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:11.792710 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f370873-429d-4cfd-9a83-be20b295ffa8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bztfxj\" (UID: \"6f370873-429d-4cfd-9a83-be20b295ffa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" May 28 16:14:11.792873 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:11.792745 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t94fb\" (UniqueName: \"kubernetes.io/projected/6f370873-429d-4cfd-9a83-be20b295ffa8-kube-api-access-t94fb\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bztfxj\" (UID: \"6f370873-429d-4cfd-9a83-be20b295ffa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" May 28 16:14:11.792873 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:11.792769 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f370873-429d-4cfd-9a83-be20b295ffa8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bztfxj\" (UID: \"6f370873-429d-4cfd-9a83-be20b295ffa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" May 28 16:14:11.792873 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:11.792830 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f370873-429d-4cfd-9a83-be20b295ffa8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bztfxj\" (UID: \"6f370873-429d-4cfd-9a83-be20b295ffa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" May 28 16:14:11.792873 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:11.792871 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f370873-429d-4cfd-9a83-be20b295ffa8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bztfxj\" (UID: \"6f370873-429d-4cfd-9a83-be20b295ffa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" May 28 16:14:11.793073 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:11.792889 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f370873-429d-4cfd-9a83-be20b295ffa8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bztfxj\" (UID: \"6f370873-429d-4cfd-9a83-be20b295ffa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" May 28 16:14:11.893899 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:11.893821 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f370873-429d-4cfd-9a83-be20b295ffa8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bztfxj\" (UID: \"6f370873-429d-4cfd-9a83-be20b295ffa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" May 28 16:14:11.893899 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:11.893859 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f370873-429d-4cfd-9a83-be20b295ffa8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bztfxj\" (UID: \"6f370873-429d-4cfd-9a83-be20b295ffa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" May 28 16:14:11.893899 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:11.893887 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f370873-429d-4cfd-9a83-be20b295ffa8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bztfxj\" (UID: \"6f370873-429d-4cfd-9a83-be20b295ffa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" May 28 16:14:11.894133 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:11.893920 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f370873-429d-4cfd-9a83-be20b295ffa8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bztfxj\" (UID: \"6f370873-429d-4cfd-9a83-be20b295ffa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" May 28 16:14:11.894133 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:11.893959 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f370873-429d-4cfd-9a83-be20b295ffa8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bztfxj\" (UID: \"6f370873-429d-4cfd-9a83-be20b295ffa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" May 28 16:14:11.894133 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:11.893986 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f370873-429d-4cfd-9a83-be20b295ffa8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bztfxj\" (UID: \"6f370873-429d-4cfd-9a83-be20b295ffa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" May 28 16:14:11.894133 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:11.894031 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t94fb\" (UniqueName: \"kubernetes.io/projected/6f370873-429d-4cfd-9a83-be20b295ffa8-kube-api-access-t94fb\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bztfxj\" (UID: \"6f370873-429d-4cfd-9a83-be20b295ffa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" May 28 16:14:11.894133 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:11.894062 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f370873-429d-4cfd-9a83-be20b295ffa8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bztfxj\" (UID: \"6f370873-429d-4cfd-9a83-be20b295ffa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" May 28 16:14:11.894133 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:11.894105 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f370873-429d-4cfd-9a83-be20b295ffa8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bztfxj\" (UID: \"6f370873-429d-4cfd-9a83-be20b295ffa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" May 28 16:14:11.894446 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:11.894274 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f370873-429d-4cfd-9a83-be20b295ffa8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bztfxj\" (UID: \"6f370873-429d-4cfd-9a83-be20b295ffa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" May 28 16:14:11.894446 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:11.894334 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f370873-429d-4cfd-9a83-be20b295ffa8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bztfxj\" (UID: \"6f370873-429d-4cfd-9a83-be20b295ffa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" May 28 16:14:11.894542 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:11.894472 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f370873-429d-4cfd-9a83-be20b295ffa8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bztfxj\" (UID: \"6f370873-429d-4cfd-9a83-be20b295ffa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" May 28 16:14:11.894689 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:11.894662 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f370873-429d-4cfd-9a83-be20b295ffa8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bztfxj\" (UID: \"6f370873-429d-4cfd-9a83-be20b295ffa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" May 28 16:14:11.894752 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:11.894683 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f370873-429d-4cfd-9a83-be20b295ffa8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bztfxj\" (UID: \"6f370873-429d-4cfd-9a83-be20b295ffa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" May 28 16:14:11.896441 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:11.896412 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f370873-429d-4cfd-9a83-be20b295ffa8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bztfxj\" (UID: \"6f370873-429d-4cfd-9a83-be20b295ffa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" May 28 16:14:11.896552 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:11.896518 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f370873-429d-4cfd-9a83-be20b295ffa8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bztfxj\" (UID: \"6f370873-429d-4cfd-9a83-be20b295ffa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" May 28 16:14:11.902740 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:11.902714 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f370873-429d-4cfd-9a83-be20b295ffa8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bztfxj\" (UID: \"6f370873-429d-4cfd-9a83-be20b295ffa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" May 28 16:14:11.902985 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:11.902948 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t94fb\" (UniqueName: \"kubernetes.io/projected/6f370873-429d-4cfd-9a83-be20b295ffa8-kube-api-access-t94fb\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bztfxj\" (UID: \"6f370873-429d-4cfd-9a83-be20b295ffa8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" May 28 16:14:11.956935 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:11.956886 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" May 28 16:14:12.091140 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:12.091103 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj"] May 28 16:14:12.094194 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:14:12.094165 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f370873_429d_4cfd_9a83_be20b295ffa8.slice/crio-95458b3ee2ec2c2eab70a2f81e6902d1de7524fd75abdbb5419ac85892c548bf WatchSource:0}: Error finding container 95458b3ee2ec2c2eab70a2f81e6902d1de7524fd75abdbb5419ac85892c548bf: Status 404 returned error can't find the container with id 95458b3ee2ec2c2eab70a2f81e6902d1de7524fd75abdbb5419ac85892c548bf May 28 16:14:12.503185 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:12.503144 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" event={"ID":"6f370873-429d-4cfd-9a83-be20b295ffa8","Type":"ContainerStarted","Data":"95458b3ee2ec2c2eab70a2f81e6902d1de7524fd75abdbb5419ac85892c548bf"} May 28 16:14:14.688135 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:14.688098 2566 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} May 28 16:14:14.688534 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:14.688192 2566 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} May 28 16:14:14.688534 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:14.688234 2566 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} May 28 16:14:15.513815 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:15.513771 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" event={"ID":"6f370873-429d-4cfd-9a83-be20b295ffa8","Type":"ContainerStarted","Data":"60e1116dba39a704276208cccabfd8a03599a8452b1b5c30a60491a296d7e505"} May 28 16:14:15.534623 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:15.534570 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" podStartSLOduration=1.942709039 podStartE2EDuration="4.534553457s" podCreationTimestamp="2026-05-28 16:14:11 +0000 UTC" firstStartedPulling="2026-05-28 16:14:12.096020166 +0000 UTC m=+391.323050701" lastFinishedPulling="2026-05-28 16:14:14.687864583 +0000 UTC m=+393.914895119" observedRunningTime="2026-05-28 16:14:15.532650665 +0000 UTC m=+394.759681221" watchObservedRunningTime="2026-05-28 16:14:15.534553457 +0000 UTC m=+394.761584013" May 28 16:14:15.957523 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:15.957428 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" May 28 16:14:15.961911 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:15.961884 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" May 28 16:14:16.517568 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:16.517532 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" May 28 16:14:16.518656 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:16.518636 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bztfxj" May 28 16:14:46.563347 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:46.563310 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dvsdw"] May 28 16:14:46.565465 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:46.565439 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dvsdw" May 28 16:14:46.568070 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:46.568045 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" May 28 16:14:46.568070 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:46.568064 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-g685f\"" May 28 16:14:46.569439 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:46.569424 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" May 28 16:14:46.574004 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:46.573984 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dvsdw"] May 28 16:14:46.663504 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:46.663464 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxnf2\" (UniqueName: \"kubernetes.io/projected/55d3d41a-282b-4e81-994f-9f459b55c613-kube-api-access-fxnf2\") pod \"kuadrant-operator-catalog-dvsdw\" (UID: \"55d3d41a-282b-4e81-994f-9f459b55c613\") " pod="kuadrant-system/kuadrant-operator-catalog-dvsdw" May 28 16:14:46.764943 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:46.764895 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fxnf2\" (UniqueName: \"kubernetes.io/projected/55d3d41a-282b-4e81-994f-9f459b55c613-kube-api-access-fxnf2\") pod \"kuadrant-operator-catalog-dvsdw\" (UID: \"55d3d41a-282b-4e81-994f-9f459b55c613\") " pod="kuadrant-system/kuadrant-operator-catalog-dvsdw" May 28 16:14:46.773104 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:46.773075 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxnf2\" (UniqueName: \"kubernetes.io/projected/55d3d41a-282b-4e81-994f-9f459b55c613-kube-api-access-fxnf2\") pod \"kuadrant-operator-catalog-dvsdw\" (UID: \"55d3d41a-282b-4e81-994f-9f459b55c613\") " pod="kuadrant-system/kuadrant-operator-catalog-dvsdw" May 28 16:14:46.875962 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:46.875870 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dvsdw" May 28 16:14:46.934515 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:46.934480 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dvsdw"] May 28 16:14:47.002979 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:47.002819 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dvsdw"] May 28 16:14:47.005720 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:14:47.005679 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55d3d41a_282b_4e81_994f_9f459b55c613.slice/crio-389cac956cc4b53e9acc5ff876829b7167ec42be2b7b787190f65ebcd19b212f WatchSource:0}: Error finding container 389cac956cc4b53e9acc5ff876829b7167ec42be2b7b787190f65ebcd19b212f: Status 404 returned error can't find the container with id 389cac956cc4b53e9acc5ff876829b7167ec42be2b7b787190f65ebcd19b212f May 28 16:14:47.142263 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:47.142183 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-528wb"] May 28 16:14:47.144934 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:47.144918 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-528wb" May 28 16:14:47.151060 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:47.151036 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-528wb"] May 28 16:14:47.267963 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:47.267933 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwr6x\" (UniqueName: \"kubernetes.io/projected/c671d728-dfff-445d-8ad0-df325c0494f7-kube-api-access-mwr6x\") pod \"kuadrant-operator-catalog-528wb\" (UID: \"c671d728-dfff-445d-8ad0-df325c0494f7\") " pod="kuadrant-system/kuadrant-operator-catalog-528wb" May 28 16:14:47.368550 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:47.368514 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mwr6x\" (UniqueName: \"kubernetes.io/projected/c671d728-dfff-445d-8ad0-df325c0494f7-kube-api-access-mwr6x\") pod \"kuadrant-operator-catalog-528wb\" (UID: \"c671d728-dfff-445d-8ad0-df325c0494f7\") " pod="kuadrant-system/kuadrant-operator-catalog-528wb" May 28 16:14:47.378761 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:47.378726 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwr6x\" (UniqueName: \"kubernetes.io/projected/c671d728-dfff-445d-8ad0-df325c0494f7-kube-api-access-mwr6x\") pod \"kuadrant-operator-catalog-528wb\" (UID: \"c671d728-dfff-445d-8ad0-df325c0494f7\") " pod="kuadrant-system/kuadrant-operator-catalog-528wb" May 28 16:14:47.455102 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:47.454995 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-528wb" May 28 16:14:47.588522 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:47.588489 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-528wb"] May 28 16:14:47.591387 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:14:47.591360 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc671d728_dfff_445d_8ad0_df325c0494f7.slice/crio-14101b3a3ba33b7749a216715327ceaf8cdaa5be3739b6178d27ced4f9cae1f9 WatchSource:0}: Error finding container 14101b3a3ba33b7749a216715327ceaf8cdaa5be3739b6178d27ced4f9cae1f9: Status 404 returned error can't find the container with id 14101b3a3ba33b7749a216715327ceaf8cdaa5be3739b6178d27ced4f9cae1f9 May 28 16:14:47.617303 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:47.617246 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-528wb" event={"ID":"c671d728-dfff-445d-8ad0-df325c0494f7","Type":"ContainerStarted","Data":"14101b3a3ba33b7749a216715327ceaf8cdaa5be3739b6178d27ced4f9cae1f9"} May 28 16:14:47.618188 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:47.618158 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dvsdw" event={"ID":"55d3d41a-282b-4e81-994f-9f459b55c613","Type":"ContainerStarted","Data":"389cac956cc4b53e9acc5ff876829b7167ec42be2b7b787190f65ebcd19b212f"} May 28 16:14:50.627503 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:50.627452 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-528wb" event={"ID":"c671d728-dfff-445d-8ad0-df325c0494f7","Type":"ContainerStarted","Data":"3bd1c4c09cfee9791f0167bfe69ee685825851e2ea947e1a1a3218209ea7612d"} May 28 16:14:50.628813 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:50.628784 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dvsdw" event={"ID":"55d3d41a-282b-4e81-994f-9f459b55c613","Type":"ContainerStarted","Data":"ed5f52e5507dc06613ed7c8f0509291ca42d0efb56e023b9b11791e6fc304962"} May 28 16:14:50.628936 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:50.628858 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-dvsdw" podUID="55d3d41a-282b-4e81-994f-9f459b55c613" containerName="registry-server" containerID="cri-o://ed5f52e5507dc06613ed7c8f0509291ca42d0efb56e023b9b11791e6fc304962" gracePeriod=2 May 28 16:14:50.645855 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:50.645807 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-528wb" podStartSLOduration=1.605528541 podStartE2EDuration="3.645793246s" podCreationTimestamp="2026-05-28 16:14:47 +0000 UTC" firstStartedPulling="2026-05-28 16:14:47.592766777 +0000 UTC m=+426.819797311" lastFinishedPulling="2026-05-28 16:14:49.633031482 +0000 UTC m=+428.860062016" observedRunningTime="2026-05-28 16:14:50.644977408 +0000 UTC m=+429.872007968" watchObservedRunningTime="2026-05-28 16:14:50.645793246 +0000 UTC m=+429.872823802" May 28 16:14:50.664016 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:50.663958 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-dvsdw" podStartSLOduration=2.040870983 podStartE2EDuration="4.663940471s" podCreationTimestamp="2026-05-28 16:14:46 +0000 UTC" firstStartedPulling="2026-05-28 16:14:47.006940601 +0000 UTC m=+426.233971134" lastFinishedPulling="2026-05-28 16:14:49.630010084 +0000 UTC m=+428.857040622" observedRunningTime="2026-05-28 16:14:50.663803504 +0000 UTC m=+429.890834062" watchObservedRunningTime="2026-05-28 16:14:50.663940471 +0000 UTC m=+429.890971030" May 28 16:14:50.867916 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:50.867892 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dvsdw" May 28 16:14:50.995299 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:50.995260 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxnf2\" (UniqueName: \"kubernetes.io/projected/55d3d41a-282b-4e81-994f-9f459b55c613-kube-api-access-fxnf2\") pod \"55d3d41a-282b-4e81-994f-9f459b55c613\" (UID: \"55d3d41a-282b-4e81-994f-9f459b55c613\") " May 28 16:14:50.997638 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:50.997612 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55d3d41a-282b-4e81-994f-9f459b55c613-kube-api-access-fxnf2" (OuterVolumeSpecName: "kube-api-access-fxnf2") pod "55d3d41a-282b-4e81-994f-9f459b55c613" (UID: "55d3d41a-282b-4e81-994f-9f459b55c613"). InnerVolumeSpecName "kube-api-access-fxnf2". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 16:14:51.096415 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:51.096365 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxnf2\" (UniqueName: \"kubernetes.io/projected/55d3d41a-282b-4e81-994f-9f459b55c613-kube-api-access-fxnf2\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" May 28 16:14:51.632598 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:51.632506 2566 generic.go:358] "Generic (PLEG): container finished" podID="55d3d41a-282b-4e81-994f-9f459b55c613" containerID="ed5f52e5507dc06613ed7c8f0509291ca42d0efb56e023b9b11791e6fc304962" exitCode=0 May 28 16:14:51.633017 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:51.632598 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dvsdw" event={"ID":"55d3d41a-282b-4e81-994f-9f459b55c613","Type":"ContainerDied","Data":"ed5f52e5507dc06613ed7c8f0509291ca42d0efb56e023b9b11791e6fc304962"} May 28 16:14:51.633017 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:51.632614 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dvsdw" May 28 16:14:51.633017 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:51.632637 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dvsdw" event={"ID":"55d3d41a-282b-4e81-994f-9f459b55c613","Type":"ContainerDied","Data":"389cac956cc4b53e9acc5ff876829b7167ec42be2b7b787190f65ebcd19b212f"} May 28 16:14:51.633017 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:51.632653 2566 scope.go:117] "RemoveContainer" containerID="ed5f52e5507dc06613ed7c8f0509291ca42d0efb56e023b9b11791e6fc304962" May 28 16:14:51.640559 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:51.640542 2566 scope.go:117] "RemoveContainer" containerID="ed5f52e5507dc06613ed7c8f0509291ca42d0efb56e023b9b11791e6fc304962" May 28 16:14:51.640833 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:14:51.640815 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed5f52e5507dc06613ed7c8f0509291ca42d0efb56e023b9b11791e6fc304962\": container with ID starting with ed5f52e5507dc06613ed7c8f0509291ca42d0efb56e023b9b11791e6fc304962 not found: ID does not exist" containerID="ed5f52e5507dc06613ed7c8f0509291ca42d0efb56e023b9b11791e6fc304962" May 28 16:14:51.640893 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:51.640847 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed5f52e5507dc06613ed7c8f0509291ca42d0efb56e023b9b11791e6fc304962"} err="failed to get container status \"ed5f52e5507dc06613ed7c8f0509291ca42d0efb56e023b9b11791e6fc304962\": rpc error: code = NotFound desc = could not find container \"ed5f52e5507dc06613ed7c8f0509291ca42d0efb56e023b9b11791e6fc304962\": container with ID starting with ed5f52e5507dc06613ed7c8f0509291ca42d0efb56e023b9b11791e6fc304962 not found: ID does not exist" May 28 16:14:51.648081 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:51.648057 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dvsdw"] May 28 16:14:51.652054 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:51.652031 2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dvsdw"] May 28 16:14:53.336603 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:53.336573 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55d3d41a-282b-4e81-994f-9f459b55c613" path="/var/lib/kubelet/pods/55d3d41a-282b-4e81-994f-9f459b55c613/volumes" May 28 16:14:57.456183 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:57.456086 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-528wb" May 28 16:14:57.456183 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:57.456146 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-528wb" May 28 16:14:57.477918 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:57.477888 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-528wb" May 28 16:14:57.676827 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:14:57.676801 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-528wb" May 28 16:15:12.259069 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:12.259032 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-ln8zj"] May 28 16:15:12.259500 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:12.259325 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55d3d41a-282b-4e81-994f-9f459b55c613" containerName="registry-server" May 28 16:15:12.259500 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:12.259338 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="55d3d41a-282b-4e81-994f-9f459b55c613" containerName="registry-server" May 28 16:15:12.259500 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:12.259394 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="55d3d41a-282b-4e81-994f-9f459b55c613" containerName="registry-server" May 28 16:15:12.261194 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:12.261178 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-ln8zj" May 28 16:15:12.264136 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:12.264113 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-zmsv4\"" May 28 16:15:12.274749 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:12.274723 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-ln8zj"] May 28 16:15:12.377479 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:12.377434 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89sz2\" (UniqueName: \"kubernetes.io/projected/e639f807-870e-470b-97aa-b12fb168dc9d-kube-api-access-89sz2\") pod \"limitador-operator-controller-manager-85c4996f8c-ln8zj\" (UID: \"e639f807-870e-470b-97aa-b12fb168dc9d\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-ln8zj" May 28 16:15:12.478309 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:12.478246 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-89sz2\" (UniqueName: \"kubernetes.io/projected/e639f807-870e-470b-97aa-b12fb168dc9d-kube-api-access-89sz2\") pod \"limitador-operator-controller-manager-85c4996f8c-ln8zj\" (UID: \"e639f807-870e-470b-97aa-b12fb168dc9d\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-ln8zj" May 28 16:15:12.488719 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:12.488694 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-89sz2\" (UniqueName: \"kubernetes.io/projected/e639f807-870e-470b-97aa-b12fb168dc9d-kube-api-access-89sz2\") pod \"limitador-operator-controller-manager-85c4996f8c-ln8zj\" (UID: \"e639f807-870e-470b-97aa-b12fb168dc9d\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-ln8zj" May 28 16:15:12.571067 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:12.570959 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-ln8zj" May 28 16:15:12.696004 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:12.695969 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-ln8zj"] May 28 16:15:12.698953 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:15:12.698930 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode639f807_870e_470b_97aa_b12fb168dc9d.slice/crio-c31deceea7c9e880da66aa6246eb4a0e3714f9ce2b3e012bff77ec4f320f1435 WatchSource:0}: Error finding container c31deceea7c9e880da66aa6246eb4a0e3714f9ce2b3e012bff77ec4f320f1435: Status 404 returned error can't find the container with id c31deceea7c9e880da66aa6246eb4a0e3714f9ce2b3e012bff77ec4f320f1435 May 28 16:15:12.703817 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:12.703785 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-ln8zj" event={"ID":"e639f807-870e-470b-97aa-b12fb168dc9d","Type":"ContainerStarted","Data":"c31deceea7c9e880da66aa6246eb4a0e3714f9ce2b3e012bff77ec4f320f1435"} May 28 16:15:15.716299 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:15.716246 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-ln8zj" event={"ID":"e639f807-870e-470b-97aa-b12fb168dc9d","Type":"ContainerStarted","Data":"b476fdf379dae643805a6ff7310dfb4fe2ede433d5d3746c28ec122e0ab25e41"} May 28 16:15:15.716669 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:15.716351 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-ln8zj" May 28 16:15:15.732825 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:15.732775 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-ln8zj" podStartSLOduration=1.649186148 podStartE2EDuration="3.732758784s" podCreationTimestamp="2026-05-28 16:15:12 +0000 UTC" firstStartedPulling="2026-05-28 16:15:12.700934765 +0000 UTC m=+451.927965305" lastFinishedPulling="2026-05-28 16:15:14.784507403 +0000 UTC m=+454.011537941" observedRunningTime="2026-05-28 16:15:15.73176899 +0000 UTC m=+454.958799561" watchObservedRunningTime="2026-05-28 16:15:15.732758784 +0000 UTC m=+454.959789342" May 28 16:15:19.703406 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:19.703368 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-pldc9"] May 28 16:15:19.711023 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:19.710996 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-pldc9" May 28 16:15:19.713822 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:19.713802 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" May 28 16:15:19.713964 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:19.713949 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-5bzg5\"" May 28 16:15:19.726793 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:19.726763 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-pldc9"] May 28 16:15:19.839908 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:19.839867 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29vf8\" (UniqueName: \"kubernetes.io/projected/c5233b77-fbae-4067-8517-b7351a3dbb42-kube-api-access-29vf8\") pod \"dns-operator-controller-manager-648d5c98bc-pldc9\" (UID: \"c5233b77-fbae-4067-8517-b7351a3dbb42\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-pldc9" May 28 16:15:19.940738 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:19.940697 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-29vf8\" (UniqueName: \"kubernetes.io/projected/c5233b77-fbae-4067-8517-b7351a3dbb42-kube-api-access-29vf8\") pod \"dns-operator-controller-manager-648d5c98bc-pldc9\" (UID: \"c5233b77-fbae-4067-8517-b7351a3dbb42\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-pldc9" May 28 16:15:19.954628 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:19.954556 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29vf8\" (UniqueName: \"kubernetes.io/projected/c5233b77-fbae-4067-8517-b7351a3dbb42-kube-api-access-29vf8\") pod \"dns-operator-controller-manager-648d5c98bc-pldc9\" (UID: \"c5233b77-fbae-4067-8517-b7351a3dbb42\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-pldc9" May 28 16:15:20.021425 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.021371 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-pldc9" May 28 16:15:20.158063 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.158024 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-pldc9"] May 28 16:15:20.161875 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.161845 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-5cccdf6b74-q7q8r"] May 28 16:15:20.162542 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:15:20.162516 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5233b77_fbae_4067_8517_b7351a3dbb42.slice/crio-d97044811d327900385f2f7036ac164ffa413e7d6a9dbf24eb903c88129cfe93 WatchSource:0}: Error finding container d97044811d327900385f2f7036ac164ffa413e7d6a9dbf24eb903c88129cfe93: Status 404 returned error can't find the container with id d97044811d327900385f2f7036ac164ffa413e7d6a9dbf24eb903c88129cfe93 May 28 16:15:20.167667 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.167645 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cccdf6b74-q7q8r" May 28 16:15:20.170481 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.170456 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" May 28 16:15:20.170580 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.170511 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" May 28 16:15:20.170961 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.170781 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" May 28 16:15:20.170961 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.170800 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" May 28 16:15:20.170961 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.170831 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" May 28 16:15:20.170961 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.170878 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" May 28 16:15:20.170961 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.170910 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" May 28 16:15:20.171258 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.171128 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-7r9nd\"" May 28 16:15:20.177565 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.177540 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cccdf6b74-q7q8r"] May 28 16:15:20.181708 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.179988 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" May 28 16:15:20.345371 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.345336 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksgcq\" (UniqueName: \"kubernetes.io/projected/19e724a8-a757-4dbb-b236-7cdb069ee6e8-kube-api-access-ksgcq\") pod \"console-5cccdf6b74-q7q8r\" (UID: \"19e724a8-a757-4dbb-b236-7cdb069ee6e8\") " pod="openshift-console/console-5cccdf6b74-q7q8r" May 28 16:15:20.345371 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.345372 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/19e724a8-a757-4dbb-b236-7cdb069ee6e8-service-ca\") pod \"console-5cccdf6b74-q7q8r\" (UID: \"19e724a8-a757-4dbb-b236-7cdb069ee6e8\") " pod="openshift-console/console-5cccdf6b74-q7q8r" May 28 16:15:20.345557 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.345410 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/19e724a8-a757-4dbb-b236-7cdb069ee6e8-console-config\") pod \"console-5cccdf6b74-q7q8r\" (UID: \"19e724a8-a757-4dbb-b236-7cdb069ee6e8\") " pod="openshift-console/console-5cccdf6b74-q7q8r" May 28 16:15:20.345557 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.345478 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19e724a8-a757-4dbb-b236-7cdb069ee6e8-trusted-ca-bundle\") pod \"console-5cccdf6b74-q7q8r\" (UID: \"19e724a8-a757-4dbb-b236-7cdb069ee6e8\") " pod="openshift-console/console-5cccdf6b74-q7q8r" May 28 16:15:20.345557 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.345495 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/19e724a8-a757-4dbb-b236-7cdb069ee6e8-oauth-serving-cert\") pod \"console-5cccdf6b74-q7q8r\" (UID: \"19e724a8-a757-4dbb-b236-7cdb069ee6e8\") " pod="openshift-console/console-5cccdf6b74-q7q8r" May 28 16:15:20.345557 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.345532 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/19e724a8-a757-4dbb-b236-7cdb069ee6e8-console-serving-cert\") pod \"console-5cccdf6b74-q7q8r\" (UID: \"19e724a8-a757-4dbb-b236-7cdb069ee6e8\") " pod="openshift-console/console-5cccdf6b74-q7q8r" May 28 16:15:20.345557 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.345554 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/19e724a8-a757-4dbb-b236-7cdb069ee6e8-console-oauth-config\") pod \"console-5cccdf6b74-q7q8r\" (UID: \"19e724a8-a757-4dbb-b236-7cdb069ee6e8\") " pod="openshift-console/console-5cccdf6b74-q7q8r" May 28 16:15:20.446061 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.446018 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/19e724a8-a757-4dbb-b236-7cdb069ee6e8-console-serving-cert\") pod \"console-5cccdf6b74-q7q8r\" (UID: \"19e724a8-a757-4dbb-b236-7cdb069ee6e8\") " pod="openshift-console/console-5cccdf6b74-q7q8r" May 28 16:15:20.446061 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.446058 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/19e724a8-a757-4dbb-b236-7cdb069ee6e8-console-oauth-config\") pod \"console-5cccdf6b74-q7q8r\" (UID: \"19e724a8-a757-4dbb-b236-7cdb069ee6e8\") " pod="openshift-console/console-5cccdf6b74-q7q8r" May 28 16:15:20.446322 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.446251 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ksgcq\" (UniqueName: \"kubernetes.io/projected/19e724a8-a757-4dbb-b236-7cdb069ee6e8-kube-api-access-ksgcq\") pod \"console-5cccdf6b74-q7q8r\" (UID: \"19e724a8-a757-4dbb-b236-7cdb069ee6e8\") " pod="openshift-console/console-5cccdf6b74-q7q8r" May 28 16:15:20.446382 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.446325 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/19e724a8-a757-4dbb-b236-7cdb069ee6e8-service-ca\") pod \"console-5cccdf6b74-q7q8r\" (UID: \"19e724a8-a757-4dbb-b236-7cdb069ee6e8\") " pod="openshift-console/console-5cccdf6b74-q7q8r" May 28 16:15:20.446382 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.446362 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/19e724a8-a757-4dbb-b236-7cdb069ee6e8-console-config\") pod \"console-5cccdf6b74-q7q8r\" (UID: \"19e724a8-a757-4dbb-b236-7cdb069ee6e8\") " pod="openshift-console/console-5cccdf6b74-q7q8r" May 28 16:15:20.446474 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.446405 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19e724a8-a757-4dbb-b236-7cdb069ee6e8-trusted-ca-bundle\") pod \"console-5cccdf6b74-q7q8r\" (UID: \"19e724a8-a757-4dbb-b236-7cdb069ee6e8\") " pod="openshift-console/console-5cccdf6b74-q7q8r" May 28 16:15:20.446474 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.446430 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/19e724a8-a757-4dbb-b236-7cdb069ee6e8-oauth-serving-cert\") pod \"console-5cccdf6b74-q7q8r\" (UID: \"19e724a8-a757-4dbb-b236-7cdb069ee6e8\") " pod="openshift-console/console-5cccdf6b74-q7q8r" May 28 16:15:20.447137 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.447112 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/19e724a8-a757-4dbb-b236-7cdb069ee6e8-console-config\") pod \"console-5cccdf6b74-q7q8r\" (UID: \"19e724a8-a757-4dbb-b236-7cdb069ee6e8\") " pod="openshift-console/console-5cccdf6b74-q7q8r" May 28 16:15:20.447263 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.447164 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/19e724a8-a757-4dbb-b236-7cdb069ee6e8-service-ca\") pod \"console-5cccdf6b74-q7q8r\" (UID: \"19e724a8-a757-4dbb-b236-7cdb069ee6e8\") " pod="openshift-console/console-5cccdf6b74-q7q8r" May 28 16:15:20.447480 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.447457 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/19e724a8-a757-4dbb-b236-7cdb069ee6e8-oauth-serving-cert\") pod \"console-5cccdf6b74-q7q8r\" (UID: \"19e724a8-a757-4dbb-b236-7cdb069ee6e8\") " pod="openshift-console/console-5cccdf6b74-q7q8r" May 28 16:15:20.447646 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.447625 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19e724a8-a757-4dbb-b236-7cdb069ee6e8-trusted-ca-bundle\") pod \"console-5cccdf6b74-q7q8r\" (UID: \"19e724a8-a757-4dbb-b236-7cdb069ee6e8\") " pod="openshift-console/console-5cccdf6b74-q7q8r" May 28 16:15:20.448882 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.448865 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/19e724a8-a757-4dbb-b236-7cdb069ee6e8-console-oauth-config\") pod \"console-5cccdf6b74-q7q8r\" (UID: \"19e724a8-a757-4dbb-b236-7cdb069ee6e8\") " pod="openshift-console/console-5cccdf6b74-q7q8r" May 28 16:15:20.448998 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.448982 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/19e724a8-a757-4dbb-b236-7cdb069ee6e8-console-serving-cert\") pod \"console-5cccdf6b74-q7q8r\" (UID: \"19e724a8-a757-4dbb-b236-7cdb069ee6e8\") " pod="openshift-console/console-5cccdf6b74-q7q8r" May 28 16:15:20.455322 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.455304 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksgcq\" (UniqueName: \"kubernetes.io/projected/19e724a8-a757-4dbb-b236-7cdb069ee6e8-kube-api-access-ksgcq\") pod \"console-5cccdf6b74-q7q8r\" (UID: \"19e724a8-a757-4dbb-b236-7cdb069ee6e8\") " pod="openshift-console/console-5cccdf6b74-q7q8r" May 28 16:15:20.478349 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.478319 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cccdf6b74-q7q8r" May 28 16:15:20.660014 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.659990 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cccdf6b74-q7q8r"] May 28 16:15:20.661606 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:15:20.661578 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19e724a8_a757_4dbb_b236_7cdb069ee6e8.slice/crio-dd684f42b54e073081720017f4ed178d8caea8fe8bffbf524d6eeb0def5acac8 WatchSource:0}: Error finding container dd684f42b54e073081720017f4ed178d8caea8fe8bffbf524d6eeb0def5acac8: Status 404 returned error can't find the container with id dd684f42b54e073081720017f4ed178d8caea8fe8bffbf524d6eeb0def5acac8 May 28 16:15:20.733474 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.733433 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-pldc9" event={"ID":"c5233b77-fbae-4067-8517-b7351a3dbb42","Type":"ContainerStarted","Data":"d97044811d327900385f2f7036ac164ffa413e7d6a9dbf24eb903c88129cfe93"} May 28 16:15:20.734968 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.734938 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cccdf6b74-q7q8r" event={"ID":"19e724a8-a757-4dbb-b236-7cdb069ee6e8","Type":"ContainerStarted","Data":"0f59c8dc4cdac1151611622b5756821cf9a067f8a745f20b8161e092ecd380d7"} May 28 16:15:20.735093 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.734976 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cccdf6b74-q7q8r" event={"ID":"19e724a8-a757-4dbb-b236-7cdb069ee6e8","Type":"ContainerStarted","Data":"dd684f42b54e073081720017f4ed178d8caea8fe8bffbf524d6eeb0def5acac8"} May 28 16:15:20.755557 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:20.755509 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-5cccdf6b74-q7q8r" podStartSLOduration=0.755490089 podStartE2EDuration="755.490089ms" podCreationTimestamp="2026-05-28 16:15:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 16:15:20.754195289 +0000 UTC m=+459.981225872" watchObservedRunningTime="2026-05-28 16:15:20.755490089 +0000 UTC m=+459.982520650" May 28 16:15:22.751414 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:22.751379 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-pldc9" event={"ID":"c5233b77-fbae-4067-8517-b7351a3dbb42","Type":"ContainerStarted","Data":"5053c8874a3e62c0a4eeba22f04dde5468b84fc121e29ccb3bfe7a578459393d"} May 28 16:15:22.751775 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:22.751545 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-pldc9" May 28 16:15:22.772980 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:22.772925 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-pldc9" podStartSLOduration=1.26009179 podStartE2EDuration="3.772911033s" podCreationTimestamp="2026-05-28 16:15:19 +0000 UTC" firstStartedPulling="2026-05-28 16:15:20.164663856 +0000 UTC m=+459.391694390" lastFinishedPulling="2026-05-28 16:15:22.677483095 +0000 UTC m=+461.904513633" observedRunningTime="2026-05-28 16:15:22.77192635 +0000 UTC m=+461.998956909" watchObservedRunningTime="2026-05-28 16:15:22.772911033 +0000 UTC m=+461.999941589" May 28 16:15:24.343060 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:24.343016 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-287l4"] May 28 16:15:24.346373 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:24.346351 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-287l4" May 28 16:15:24.349035 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:24.349014 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-7gld9\"" May 28 16:15:24.360530 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:24.360505 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-287l4"] May 28 16:15:24.481576 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:24.481543 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b3094868-455d-4773-a993-31f71d276816-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-287l4\" (UID: \"b3094868-455d-4773-a993-31f71d276816\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-287l4" May 28 16:15:24.481741 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:24.481585 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fccqq\" (UniqueName: \"kubernetes.io/projected/b3094868-455d-4773-a993-31f71d276816-kube-api-access-fccqq\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-287l4\" (UID: \"b3094868-455d-4773-a993-31f71d276816\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-287l4" May 28 16:15:24.582064 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:24.582012 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b3094868-455d-4773-a993-31f71d276816-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-287l4\" (UID: \"b3094868-455d-4773-a993-31f71d276816\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-287l4" May 28 16:15:24.582064 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:24.582075 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fccqq\" (UniqueName: \"kubernetes.io/projected/b3094868-455d-4773-a993-31f71d276816-kube-api-access-fccqq\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-287l4\" (UID: \"b3094868-455d-4773-a993-31f71d276816\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-287l4" May 28 16:15:24.582452 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:24.582434 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b3094868-455d-4773-a993-31f71d276816-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-287l4\" (UID: \"b3094868-455d-4773-a993-31f71d276816\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-287l4" May 28 16:15:24.591690 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:24.591662 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fccqq\" (UniqueName: \"kubernetes.io/projected/b3094868-455d-4773-a993-31f71d276816-kube-api-access-fccqq\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-287l4\" (UID: \"b3094868-455d-4773-a993-31f71d276816\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-287l4" May 28 16:15:24.655857 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:24.655768 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-287l4" May 28 16:15:24.792320 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:24.792271 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-287l4"] May 28 16:15:24.797061 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:15:24.797031 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3094868_455d_4773_a993_31f71d276816.slice/crio-dd5b873b9a9e57a6d1e9f78463772177df1d6d33426b45c1534b5e55128d5c9c WatchSource:0}: Error finding container dd5b873b9a9e57a6d1e9f78463772177df1d6d33426b45c1534b5e55128d5c9c: Status 404 returned error can't find the container with id dd5b873b9a9e57a6d1e9f78463772177df1d6d33426b45c1534b5e55128d5c9c May 28 16:15:25.762725 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:25.762690 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-287l4" event={"ID":"b3094868-455d-4773-a993-31f71d276816","Type":"ContainerStarted","Data":"dd5b873b9a9e57a6d1e9f78463772177df1d6d33426b45c1534b5e55128d5c9c"} May 28 16:15:26.722134 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:26.722100 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-ln8zj" May 28 16:15:30.478860 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:30.478810 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5cccdf6b74-q7q8r" May 28 16:15:30.478860 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:30.478859 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5cccdf6b74-q7q8r" May 28 16:15:30.484169 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:30.484144 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5cccdf6b74-q7q8r" May 28 16:15:30.784529 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:30.784438 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-287l4" event={"ID":"b3094868-455d-4773-a993-31f71d276816","Type":"ContainerStarted","Data":"4571ad9989069833185d1bf0789619afd535d0e102fbc37fbfc472427e34a1d8"} May 28 16:15:30.788355 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:30.788330 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5cccdf6b74-q7q8r" May 28 16:15:30.803780 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:30.803717 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-287l4" podStartSLOduration=1.7621497929999999 podStartE2EDuration="6.803695231s" podCreationTimestamp="2026-05-28 16:15:24 +0000 UTC" firstStartedPulling="2026-05-28 16:15:24.799608463 +0000 UTC m=+464.026639002" lastFinishedPulling="2026-05-28 16:15:29.8411539 +0000 UTC m=+469.068184440" observedRunningTime="2026-05-28 16:15:30.802588324 +0000 UTC m=+470.029618908" watchObservedRunningTime="2026-05-28 16:15:30.803695231 +0000 UTC m=+470.030725788" May 28 16:15:31.788068 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:31.788039 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-287l4" May 28 16:15:33.757660 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:33.757629 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-pldc9" May 28 16:15:42.793602 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:42.793560 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-287l4" May 28 16:15:43.745340 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:43.745307 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-7w55z"] May 28 16:15:43.749834 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:43.749816 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-7w55z" May 28 16:15:43.759035 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:43.758991 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-7w55z"] May 28 16:15:43.832507 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:43.832475 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tcfq\" (UniqueName: \"kubernetes.io/projected/b6dc0dd5-3f7f-4018-9c95-755dedc8652c-kube-api-access-2tcfq\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-7w55z\" (UID: \"b6dc0dd5-3f7f-4018-9c95-755dedc8652c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-7w55z" May 28 16:15:43.832904 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:43.832518 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b6dc0dd5-3f7f-4018-9c95-755dedc8652c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-7w55z\" (UID: \"b6dc0dd5-3f7f-4018-9c95-755dedc8652c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-7w55z" May 28 16:15:43.933848 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:43.933808 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2tcfq\" (UniqueName: \"kubernetes.io/projected/b6dc0dd5-3f7f-4018-9c95-755dedc8652c-kube-api-access-2tcfq\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-7w55z\" (UID: \"b6dc0dd5-3f7f-4018-9c95-755dedc8652c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-7w55z" May 28 16:15:43.934018 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:43.933879 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b6dc0dd5-3f7f-4018-9c95-755dedc8652c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-7w55z\" (UID: \"b6dc0dd5-3f7f-4018-9c95-755dedc8652c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-7w55z" May 28 16:15:43.934326 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:43.934274 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b6dc0dd5-3f7f-4018-9c95-755dedc8652c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-7w55z\" (UID: \"b6dc0dd5-3f7f-4018-9c95-755dedc8652c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-7w55z" May 28 16:15:43.945668 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:43.945634 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tcfq\" (UniqueName: \"kubernetes.io/projected/b6dc0dd5-3f7f-4018-9c95-755dedc8652c-kube-api-access-2tcfq\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-7w55z\" (UID: \"b6dc0dd5-3f7f-4018-9c95-755dedc8652c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-7w55z" May 28 16:15:44.060316 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.060212 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-7w55z" May 28 16:15:44.191890 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.191856 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-7w55z"] May 28 16:15:44.195203 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:15:44.195170 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6dc0dd5_3f7f_4018_9c95_755dedc8652c.slice/crio-461dde3619df58e21a3b5b7424372a72e90717edafc20f922af73412a6dd82f7 WatchSource:0}: Error finding container 461dde3619df58e21a3b5b7424372a72e90717edafc20f922af73412a6dd82f7: Status 404 returned error can't find the container with id 461dde3619df58e21a3b5b7424372a72e90717edafc20f922af73412a6dd82f7 May 28 16:15:44.582845 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.582811 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-287l4"] May 28 16:15:44.583129 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.583081 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-287l4" podUID="b3094868-455d-4773-a993-31f71d276816" containerName="manager" containerID="cri-o://4571ad9989069833185d1bf0789619afd535d0e102fbc37fbfc472427e34a1d8" gracePeriod=2 May 28 16:15:44.588050 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.588022 2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-287l4"] May 28 16:15:44.601000 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.600366 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-7w55z"] May 28 16:15:44.608446 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.608413 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2nxr"] May 28 16:15:44.608899 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.608874 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3094868-455d-4773-a993-31f71d276816" containerName="manager" May 28 16:15:44.609048 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.609036 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3094868-455d-4773-a993-31f71d276816" containerName="manager" May 28 16:15:44.609249 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.609238 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="b3094868-455d-4773-a993-31f71d276816" containerName="manager" May 28 16:15:44.612380 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.612357 2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-7w55z"] May 28 16:15:44.612604 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.612581 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2nxr" May 28 16:15:44.623853 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.623821 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-ln8zj"] May 28 16:15:44.624148 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.624121 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-ln8zj" podUID="e639f807-870e-470b-97aa-b12fb168dc9d" containerName="manager" containerID="cri-o://b476fdf379dae643805a6ff7310dfb4fe2ede433d5d3746c28ec122e0ab25e41" gracePeriod=2 May 28 16:15:44.633110 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.632766 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-24p6g"] May 28 16:15:44.633270 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.633217 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6dc0dd5-3f7f-4018-9c95-755dedc8652c" containerName="manager" May 28 16:15:44.633270 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.633237 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6dc0dd5-3f7f-4018-9c95-755dedc8652c" containerName="manager" May 28 16:15:44.633410 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.633361 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="b6dc0dd5-3f7f-4018-9c95-755dedc8652c" containerName="manager" May 28 16:15:44.636242 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.636217 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2nxr"] May 28 16:15:44.636385 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.636370 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-24p6g" May 28 16:15:44.638197 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.638134 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d47bl\" (UniqueName: \"kubernetes.io/projected/af626d62-0b3c-4859-88f6-06e59921a4f4-kube-api-access-d47bl\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-m2nxr\" (UID: \"af626d62-0b3c-4859-88f6-06e59921a4f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2nxr" May 28 16:15:44.638197 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.638188 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af626d62-0b3c-4859-88f6-06e59921a4f4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-m2nxr\" (UID: \"af626d62-0b3c-4859-88f6-06e59921a4f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2nxr" May 28 16:15:44.640235 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.640212 2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-ln8zj"] May 28 16:15:44.675040 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.675000 2566 status_manager.go:895] "Failed to get status for pod" podUID="b3094868-455d-4773-a993-31f71d276816" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-287l4" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-287l4\" is forbidden: User \"system:node:ip-10-0-143-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-102.ec2.internal' and this object" May 28 16:15:44.677828 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.677799 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-24p6g"] May 28 16:15:44.702005 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.701972 2566 status_manager.go:895] "Failed to get status for pod" podUID="b3094868-455d-4773-a993-31f71d276816" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-287l4" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-287l4\" is forbidden: User \"system:node:ip-10-0-143-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-102.ec2.internal' and this object" May 28 16:15:44.704007 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.703972 2566 status_manager.go:895] "Failed to get status for pod" podUID="e639f807-870e-470b-97aa-b12fb168dc9d" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-ln8zj" err="pods \"limitador-operator-controller-manager-85c4996f8c-ln8zj\" is forbidden: User \"system:node:ip-10-0-143-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-102.ec2.internal' and this object" May 28 16:15:44.738787 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.738761 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d47bl\" (UniqueName: \"kubernetes.io/projected/af626d62-0b3c-4859-88f6-06e59921a4f4-kube-api-access-d47bl\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-m2nxr\" (UID: \"af626d62-0b3c-4859-88f6-06e59921a4f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2nxr" May 28 16:15:44.738864 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.738808 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm6vx\" (UniqueName: \"kubernetes.io/projected/35c45c40-4aca-407d-ac1c-0d7e65f5cb25-kube-api-access-sm6vx\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-24p6g\" (UID: \"35c45c40-4aca-407d-ac1c-0d7e65f5cb25\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-24p6g" May 28 16:15:44.738864 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.738840 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af626d62-0b3c-4859-88f6-06e59921a4f4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-m2nxr\" (UID: \"af626d62-0b3c-4859-88f6-06e59921a4f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2nxr" May 28 16:15:44.738969 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.738882 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35c45c40-4aca-407d-ac1c-0d7e65f5cb25-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-24p6g\" (UID: \"35c45c40-4aca-407d-ac1c-0d7e65f5cb25\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-24p6g" May 28 16:15:44.739237 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.739211 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af626d62-0b3c-4859-88f6-06e59921a4f4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-m2nxr\" (UID: \"af626d62-0b3c-4859-88f6-06e59921a4f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2nxr" May 28 16:15:44.747884 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.747828 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d47bl\" (UniqueName: \"kubernetes.io/projected/af626d62-0b3c-4859-88f6-06e59921a4f4-kube-api-access-d47bl\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-m2nxr\" (UID: \"af626d62-0b3c-4859-88f6-06e59921a4f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2nxr" May 28 16:15:44.833987 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:15:44.833913 2566 kuberuntime_manager.go:623] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-7w55z" container="manager" May 28 16:15:44.835553 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.835523 2566 generic.go:358] "Generic (PLEG): container finished" podID="b3094868-455d-4773-a993-31f71d276816" containerID="4571ad9989069833185d1bf0789619afd535d0e102fbc37fbfc472427e34a1d8" exitCode=0 May 28 16:15:44.836272 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.836247 2566 status_manager.go:895] "Failed to get status for pod" podUID="e639f807-870e-470b-97aa-b12fb168dc9d" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-ln8zj" err="pods \"limitador-operator-controller-manager-85c4996f8c-ln8zj\" is forbidden: User \"system:node:ip-10-0-143-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-102.ec2.internal' and this object" May 28 16:15:44.837094 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.837075 2566 generic.go:358] "Generic (PLEG): container finished" podID="e639f807-870e-470b-97aa-b12fb168dc9d" containerID="b476fdf379dae643805a6ff7310dfb4fe2ede433d5d3746c28ec122e0ab25e41" exitCode=0 May 28 16:15:44.838307 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.838257 2566 status_manager.go:895] "Failed to get status for pod" podUID="b6dc0dd5-3f7f-4018-9c95-755dedc8652c" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-7w55z" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-7w55z\" is forbidden: User \"system:node:ip-10-0-143-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-102.ec2.internal' and this object" May 28 16:15:44.839574 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.839554 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sm6vx\" (UniqueName: \"kubernetes.io/projected/35c45c40-4aca-407d-ac1c-0d7e65f5cb25-kube-api-access-sm6vx\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-24p6g\" (UID: \"35c45c40-4aca-407d-ac1c-0d7e65f5cb25\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-24p6g" May 28 16:15:44.839662 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.839591 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35c45c40-4aca-407d-ac1c-0d7e65f5cb25-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-24p6g\" (UID: \"35c45c40-4aca-407d-ac1c-0d7e65f5cb25\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-24p6g" May 28 16:15:44.839960 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.839944 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35c45c40-4aca-407d-ac1c-0d7e65f5cb25-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-24p6g\" (UID: \"35c45c40-4aca-407d-ac1c-0d7e65f5cb25\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-24p6g" May 28 16:15:44.840483 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.840461 2566 status_manager.go:895] "Failed to get status for pod" podUID="b3094868-455d-4773-a993-31f71d276816" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-287l4" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-287l4\" is forbidden: User \"system:node:ip-10-0-143-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-102.ec2.internal' and this object" May 28 16:15:44.851211 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.851188 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm6vx\" (UniqueName: \"kubernetes.io/projected/35c45c40-4aca-407d-ac1c-0d7e65f5cb25-kube-api-access-sm6vx\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-24p6g\" (UID: \"35c45c40-4aca-407d-ac1c-0d7e65f5cb25\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-24p6g" May 28 16:15:44.853978 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.853956 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-287l4" May 28 16:15:44.856530 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.856505 2566 status_manager.go:895] "Failed to get status for pod" podUID="e639f807-870e-470b-97aa-b12fb168dc9d" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-ln8zj" err="pods \"limitador-operator-controller-manager-85c4996f8c-ln8zj\" is forbidden: User \"system:node:ip-10-0-143-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-102.ec2.internal' and this object" May 28 16:15:44.858595 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.858574 2566 status_manager.go:895] "Failed to get status for pod" podUID="b6dc0dd5-3f7f-4018-9c95-755dedc8652c" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-7w55z" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-7w55z\" is forbidden: User \"system:node:ip-10-0-143-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-102.ec2.internal' and this object" May 28 16:15:44.860462 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.860437 2566 status_manager.go:895] "Failed to get status for pod" podUID="b3094868-455d-4773-a993-31f71d276816" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-287l4" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-287l4\" is forbidden: User \"system:node:ip-10-0-143-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-102.ec2.internal' and this object" May 28 16:15:44.861756 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.861733 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-ln8zj" May 28 16:15:44.863683 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.863661 2566 status_manager.go:895] "Failed to get status for pod" podUID="b3094868-455d-4773-a993-31f71d276816" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-287l4" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-287l4\" is forbidden: User \"system:node:ip-10-0-143-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-102.ec2.internal' and this object" May 28 16:15:44.865575 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.865556 2566 status_manager.go:895] "Failed to get status for pod" podUID="e639f807-870e-470b-97aa-b12fb168dc9d" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-ln8zj" err="pods \"limitador-operator-controller-manager-85c4996f8c-ln8zj\" is forbidden: User \"system:node:ip-10-0-143-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-102.ec2.internal' and this object" May 28 16:15:44.867267 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.867247 2566 status_manager.go:895] "Failed to get status for pod" podUID="b6dc0dd5-3f7f-4018-9c95-755dedc8652c" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-7w55z" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-7w55z\" is forbidden: User \"system:node:ip-10-0-143-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-102.ec2.internal' and this object" May 28 16:15:44.940345 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.940307 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fccqq\" (UniqueName: \"kubernetes.io/projected/b3094868-455d-4773-a993-31f71d276816-kube-api-access-fccqq\") pod \"b3094868-455d-4773-a993-31f71d276816\" (UID: \"b3094868-455d-4773-a993-31f71d276816\") " May 28 16:15:44.940540 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.940373 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b3094868-455d-4773-a993-31f71d276816-extensions-socket-volume\") pod \"b3094868-455d-4773-a993-31f71d276816\" (UID: \"b3094868-455d-4773-a993-31f71d276816\") " May 28 16:15:44.940540 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.940394 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-89sz2\" (UniqueName: \"kubernetes.io/projected/e639f807-870e-470b-97aa-b12fb168dc9d-kube-api-access-89sz2\") pod \"e639f807-870e-470b-97aa-b12fb168dc9d\" (UID: \"e639f807-870e-470b-97aa-b12fb168dc9d\") " May 28 16:15:44.940803 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.940777 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3094868-455d-4773-a993-31f71d276816-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b3094868-455d-4773-a993-31f71d276816" (UID: "b3094868-455d-4773-a993-31f71d276816"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 28 16:15:44.942568 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.942545 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e639f807-870e-470b-97aa-b12fb168dc9d-kube-api-access-89sz2" (OuterVolumeSpecName: "kube-api-access-89sz2") pod "e639f807-870e-470b-97aa-b12fb168dc9d" (UID: "e639f807-870e-470b-97aa-b12fb168dc9d"). InnerVolumeSpecName "kube-api-access-89sz2". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 16:15:44.942568 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:44.942556 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3094868-455d-4773-a993-31f71d276816-kube-api-access-fccqq" (OuterVolumeSpecName: "kube-api-access-fccqq") pod "b3094868-455d-4773-a993-31f71d276816" (UID: "b3094868-455d-4773-a993-31f71d276816"). InnerVolumeSpecName "kube-api-access-fccqq". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 16:15:45.017466 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:45.017424 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2nxr" May 28 16:15:45.023288 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:45.023259 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-24p6g" May 28 16:15:45.041548 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:45.041515 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b3094868-455d-4773-a993-31f71d276816-extensions-socket-volume\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" May 28 16:15:45.041548 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:45.041551 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-89sz2\" (UniqueName: \"kubernetes.io/projected/e639f807-870e-470b-97aa-b12fb168dc9d-kube-api-access-89sz2\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" May 28 16:15:45.041716 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:45.041569 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fccqq\" (UniqueName: \"kubernetes.io/projected/b3094868-455d-4773-a993-31f71d276816-kube-api-access-fccqq\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" May 28 16:15:45.168209 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:45.168177 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2nxr"] May 28 16:15:45.172187 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:15:45.172157 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf626d62_0b3c_4859_88f6_06e59921a4f4.slice/crio-ef6b38cb5a9bc7dcf16a86b51b5a5fb0db16115d706ecf663844d62f4eee21c5 WatchSource:0}: Error finding container ef6b38cb5a9bc7dcf16a86b51b5a5fb0db16115d706ecf663844d62f4eee21c5: Status 404 returned error can't find the container with id ef6b38cb5a9bc7dcf16a86b51b5a5fb0db16115d706ecf663844d62f4eee21c5 May 28 16:15:45.184759 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:45.184688 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-24p6g"] May 28 16:15:45.187903 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:15:45.187879 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35c45c40_4aca_407d_ac1c_0d7e65f5cb25.slice/crio-de84f8f90870cb3519bf5fac842d4ca136e94b0f9514ce9a26089662543cff8f WatchSource:0}: Error finding container de84f8f90870cb3519bf5fac842d4ca136e94b0f9514ce9a26089662543cff8f: Status 404 returned error can't find the container with id de84f8f90870cb3519bf5fac842d4ca136e94b0f9514ce9a26089662543cff8f May 28 16:15:45.337182 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:45.337145 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3094868-455d-4773-a993-31f71d276816" path="/var/lib/kubelet/pods/b3094868-455d-4773-a993-31f71d276816/volumes" May 28 16:15:45.337529 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:45.337516 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e639f807-870e-470b-97aa-b12fb168dc9d" path="/var/lib/kubelet/pods/e639f807-870e-470b-97aa-b12fb168dc9d/volumes" May 28 16:15:45.842522 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:45.842480 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-7w55z" podUID="b6dc0dd5-3f7f-4018-9c95-755dedc8652c" containerName="manager" containerID="cri-o://7dd4b62935d70350c288cb6fb8b533b55246ad16422eac67abfdc6a48a99e6ee" gracePeriod=2 May 28 16:15:45.843682 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:45.843664 2566 scope.go:117] "RemoveContainer" containerID="4571ad9989069833185d1bf0789619afd535d0e102fbc37fbfc472427e34a1d8" May 28 16:15:45.843798 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:45.843666 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-287l4" May 28 16:15:45.845382 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:45.845345 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-24p6g" event={"ID":"35c45c40-4aca-407d-ac1c-0d7e65f5cb25","Type":"ContainerStarted","Data":"0d597cfd5869dd5391385002847eac192cc2a9674ff766ffdb2cc89aa2bfedf5"} May 28 16:15:45.845487 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:45.845395 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-24p6g" event={"ID":"35c45c40-4aca-407d-ac1c-0d7e65f5cb25","Type":"ContainerStarted","Data":"de84f8f90870cb3519bf5fac842d4ca136e94b0f9514ce9a26089662543cff8f"} May 28 16:15:45.845542 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:45.845532 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-24p6g" May 28 16:15:45.847187 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:45.847168 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-ln8zj" May 28 16:15:45.848832 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:45.848803 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2nxr" event={"ID":"af626d62-0b3c-4859-88f6-06e59921a4f4","Type":"ContainerStarted","Data":"81afcf0a68aa74bc17ff65705d8fcad1a6b68e089008c369a9a5b6f5b37fef3e"} May 28 16:15:45.848932 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:45.848838 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2nxr" event={"ID":"af626d62-0b3c-4859-88f6-06e59921a4f4","Type":"ContainerStarted","Data":"ef6b38cb5a9bc7dcf16a86b51b5a5fb0db16115d706ecf663844d62f4eee21c5"} May 28 16:15:45.848983 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:45.848949 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2nxr" May 28 16:15:45.854211 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:45.854195 2566 scope.go:117] "RemoveContainer" containerID="b476fdf379dae643805a6ff7310dfb4fe2ede433d5d3746c28ec122e0ab25e41" May 28 16:15:45.868941 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:45.868897 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-24p6g" podStartSLOduration=1.868879017 podStartE2EDuration="1.868879017s" podCreationTimestamp="2026-05-28 16:15:44 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 16:15:45.867199025 +0000 UTC m=+485.094229580" watchObservedRunningTime="2026-05-28 16:15:45.868879017 +0000 UTC m=+485.095909575" May 28 16:15:45.888770 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:45.888720 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2nxr" podStartSLOduration=1.888706108 podStartE2EDuration="1.888706108s" podCreationTimestamp="2026-05-28 16:15:44 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 16:15:45.887260638 +0000 UTC m=+485.114291207" watchObservedRunningTime="2026-05-28 16:15:45.888706108 +0000 UTC m=+485.115736663" May 28 16:15:46.078333 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:46.078310 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-7w55z" May 28 16:15:46.080563 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:46.080536 2566 status_manager.go:895] "Failed to get status for pod" podUID="b6dc0dd5-3f7f-4018-9c95-755dedc8652c" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-7w55z" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-7w55z\" is forbidden: User \"system:node:ip-10-0-143-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-102.ec2.internal' and this object" May 28 16:15:46.149399 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:46.149306 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2tcfq\" (UniqueName: \"kubernetes.io/projected/b6dc0dd5-3f7f-4018-9c95-755dedc8652c-kube-api-access-2tcfq\") pod \"b6dc0dd5-3f7f-4018-9c95-755dedc8652c\" (UID: \"b6dc0dd5-3f7f-4018-9c95-755dedc8652c\") " May 28 16:15:46.149399 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:46.149354 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b6dc0dd5-3f7f-4018-9c95-755dedc8652c-extensions-socket-volume\") pod \"b6dc0dd5-3f7f-4018-9c95-755dedc8652c\" (UID: \"b6dc0dd5-3f7f-4018-9c95-755dedc8652c\") " May 28 16:15:46.149756 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:46.149730 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6dc0dd5-3f7f-4018-9c95-755dedc8652c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b6dc0dd5-3f7f-4018-9c95-755dedc8652c" (UID: "b6dc0dd5-3f7f-4018-9c95-755dedc8652c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 28 16:15:46.151550 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:46.151523 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6dc0dd5-3f7f-4018-9c95-755dedc8652c-kube-api-access-2tcfq" (OuterVolumeSpecName: "kube-api-access-2tcfq") pod "b6dc0dd5-3f7f-4018-9c95-755dedc8652c" (UID: "b6dc0dd5-3f7f-4018-9c95-755dedc8652c"). InnerVolumeSpecName "kube-api-access-2tcfq". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 16:15:46.250019 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:46.249984 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2tcfq\" (UniqueName: \"kubernetes.io/projected/b6dc0dd5-3f7f-4018-9c95-755dedc8652c-kube-api-access-2tcfq\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" May 28 16:15:46.250019 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:46.250016 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b6dc0dd5-3f7f-4018-9c95-755dedc8652c-extensions-socket-volume\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" May 28 16:15:46.854527 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:46.854493 2566 generic.go:358] "Generic (PLEG): container finished" podID="b6dc0dd5-3f7f-4018-9c95-755dedc8652c" containerID="7dd4b62935d70350c288cb6fb8b533b55246ad16422eac67abfdc6a48a99e6ee" exitCode=0 May 28 16:15:46.854980 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:46.854546 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-7w55z" May 28 16:15:46.854980 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:46.854588 2566 scope.go:117] "RemoveContainer" containerID="7dd4b62935d70350c288cb6fb8b533b55246ad16422eac67abfdc6a48a99e6ee" May 28 16:15:46.857398 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:46.857371 2566 status_manager.go:895] "Failed to get status for pod" podUID="b6dc0dd5-3f7f-4018-9c95-755dedc8652c" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-7w55z" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-7w55z\" is forbidden: User \"system:node:ip-10-0-143-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-102.ec2.internal' and this object" May 28 16:15:46.862692 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:46.862669 2566 scope.go:117] "RemoveContainer" containerID="7dd4b62935d70350c288cb6fb8b533b55246ad16422eac67abfdc6a48a99e6ee" May 28 16:15:46.862976 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:15:46.862958 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7dd4b62935d70350c288cb6fb8b533b55246ad16422eac67abfdc6a48a99e6ee\": container with ID starting with 7dd4b62935d70350c288cb6fb8b533b55246ad16422eac67abfdc6a48a99e6ee not found: ID does not exist" containerID="7dd4b62935d70350c288cb6fb8b533b55246ad16422eac67abfdc6a48a99e6ee" May 28 16:15:46.863046 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:46.862985 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dd4b62935d70350c288cb6fb8b533b55246ad16422eac67abfdc6a48a99e6ee"} err="failed to get container status \"7dd4b62935d70350c288cb6fb8b533b55246ad16422eac67abfdc6a48a99e6ee\": rpc error: code = NotFound desc = could not find container \"7dd4b62935d70350c288cb6fb8b533b55246ad16422eac67abfdc6a48a99e6ee\": container with ID starting with 7dd4b62935d70350c288cb6fb8b533b55246ad16422eac67abfdc6a48a99e6ee not found: ID does not exist" May 28 16:15:46.865183 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:46.865163 2566 status_manager.go:895] "Failed to get status for pod" podUID="b6dc0dd5-3f7f-4018-9c95-755dedc8652c" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-7w55z" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-7w55z\" is forbidden: User \"system:node:ip-10-0-143-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-102.ec2.internal' and this object" May 28 16:15:47.337538 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:47.337510 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6dc0dd5-3f7f-4018-9c95-755dedc8652c" path="/var/lib/kubelet/pods/b6dc0dd5-3f7f-4018-9c95-755dedc8652c/volumes" May 28 16:15:56.857305 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:56.857255 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-24p6g" May 28 16:15:56.857701 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:56.857473 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2nxr" May 28 16:15:56.932938 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:56.932904 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2nxr"] May 28 16:15:56.933158 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:56.933133 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2nxr" podUID="af626d62-0b3c-4859-88f6-06e59921a4f4" containerName="manager" containerID="cri-o://81afcf0a68aa74bc17ff65705d8fcad1a6b68e089008c369a9a5b6f5b37fef3e" gracePeriod=10 May 28 16:15:57.170163 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.170141 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2nxr" May 28 16:15:57.238569 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.238534 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d47bl\" (UniqueName: \"kubernetes.io/projected/af626d62-0b3c-4859-88f6-06e59921a4f4-kube-api-access-d47bl\") pod \"af626d62-0b3c-4859-88f6-06e59921a4f4\" (UID: \"af626d62-0b3c-4859-88f6-06e59921a4f4\") " May 28 16:15:57.238708 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.238635 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af626d62-0b3c-4859-88f6-06e59921a4f4-extensions-socket-volume\") pod \"af626d62-0b3c-4859-88f6-06e59921a4f4\" (UID: \"af626d62-0b3c-4859-88f6-06e59921a4f4\") " May 28 16:15:57.239027 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.238997 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af626d62-0b3c-4859-88f6-06e59921a4f4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "af626d62-0b3c-4859-88f6-06e59921a4f4" (UID: "af626d62-0b3c-4859-88f6-06e59921a4f4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 28 16:15:57.240777 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.240750 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af626d62-0b3c-4859-88f6-06e59921a4f4-kube-api-access-d47bl" (OuterVolumeSpecName: "kube-api-access-d47bl") pod "af626d62-0b3c-4859-88f6-06e59921a4f4" (UID: "af626d62-0b3c-4859-88f6-06e59921a4f4"). InnerVolumeSpecName "kube-api-access-d47bl". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 16:15:57.289104 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.289071 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-ldjsl"] May 28 16:15:57.289378 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.289364 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af626d62-0b3c-4859-88f6-06e59921a4f4" containerName="manager" May 28 16:15:57.289378 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.289378 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="af626d62-0b3c-4859-88f6-06e59921a4f4" containerName="manager" May 28 16:15:57.289467 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.289395 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e639f807-870e-470b-97aa-b12fb168dc9d" containerName="manager" May 28 16:15:57.289467 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.289400 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e639f807-870e-470b-97aa-b12fb168dc9d" containerName="manager" May 28 16:15:57.289467 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.289455 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="e639f807-870e-470b-97aa-b12fb168dc9d" containerName="manager" May 28 16:15:57.289467 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.289464 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="af626d62-0b3c-4859-88f6-06e59921a4f4" containerName="manager" May 28 16:15:57.291569 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.291552 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-ldjsl" May 28 16:15:57.302847 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.302823 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-ldjsl"] May 28 16:15:57.339055 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.339027 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/758df53e-4133-4eaa-bdbf-bcb7f469f44a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-ldjsl\" (UID: \"758df53e-4133-4eaa-bdbf-bcb7f469f44a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-ldjsl" May 28 16:15:57.339215 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.339073 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46f2w\" (UniqueName: \"kubernetes.io/projected/758df53e-4133-4eaa-bdbf-bcb7f469f44a-kube-api-access-46f2w\") pod \"kuadrant-operator-controller-manager-55c7f4c975-ldjsl\" (UID: \"758df53e-4133-4eaa-bdbf-bcb7f469f44a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-ldjsl" May 28 16:15:57.339215 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.339115 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d47bl\" (UniqueName: \"kubernetes.io/projected/af626d62-0b3c-4859-88f6-06e59921a4f4-kube-api-access-d47bl\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" May 28 16:15:57.339215 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.339126 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af626d62-0b3c-4859-88f6-06e59921a4f4-extensions-socket-volume\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" May 28 16:15:57.440315 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.440192 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/758df53e-4133-4eaa-bdbf-bcb7f469f44a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-ldjsl\" (UID: \"758df53e-4133-4eaa-bdbf-bcb7f469f44a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-ldjsl" May 28 16:15:57.440315 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.440250 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-46f2w\" (UniqueName: \"kubernetes.io/projected/758df53e-4133-4eaa-bdbf-bcb7f469f44a-kube-api-access-46f2w\") pod \"kuadrant-operator-controller-manager-55c7f4c975-ldjsl\" (UID: \"758df53e-4133-4eaa-bdbf-bcb7f469f44a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-ldjsl" May 28 16:15:57.440684 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.440662 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/758df53e-4133-4eaa-bdbf-bcb7f469f44a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-ldjsl\" (UID: \"758df53e-4133-4eaa-bdbf-bcb7f469f44a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-ldjsl" May 28 16:15:57.449465 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.449442 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-46f2w\" (UniqueName: \"kubernetes.io/projected/758df53e-4133-4eaa-bdbf-bcb7f469f44a-kube-api-access-46f2w\") pod \"kuadrant-operator-controller-manager-55c7f4c975-ldjsl\" (UID: \"758df53e-4133-4eaa-bdbf-bcb7f469f44a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-ldjsl" May 28 16:15:57.601092 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.601050 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-ldjsl" May 28 16:15:57.747431 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.747398 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-ldjsl"] May 28 16:15:57.750695 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:15:57.750668 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod758df53e_4133_4eaa_bdbf_bcb7f469f44a.slice/crio-d4666715e1f6e49b204ed3a710e8651a48d166c3a18914946732200ca6255d6d WatchSource:0}: Error finding container d4666715e1f6e49b204ed3a710e8651a48d166c3a18914946732200ca6255d6d: Status 404 returned error can't find the container with id d4666715e1f6e49b204ed3a710e8651a48d166c3a18914946732200ca6255d6d May 28 16:15:57.891627 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.891595 2566 generic.go:358] "Generic (PLEG): container finished" podID="af626d62-0b3c-4859-88f6-06e59921a4f4" containerID="81afcf0a68aa74bc17ff65705d8fcad1a6b68e089008c369a9a5b6f5b37fef3e" exitCode=0 May 28 16:15:57.892025 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.891659 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2nxr" event={"ID":"af626d62-0b3c-4859-88f6-06e59921a4f4","Type":"ContainerDied","Data":"81afcf0a68aa74bc17ff65705d8fcad1a6b68e089008c369a9a5b6f5b37fef3e"} May 28 16:15:57.892025 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.891683 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2nxr" event={"ID":"af626d62-0b3c-4859-88f6-06e59921a4f4","Type":"ContainerDied","Data":"ef6b38cb5a9bc7dcf16a86b51b5a5fb0db16115d706ecf663844d62f4eee21c5"} May 28 16:15:57.892025 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.891693 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2nxr" May 28 16:15:57.892025 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.891699 2566 scope.go:117] "RemoveContainer" containerID="81afcf0a68aa74bc17ff65705d8fcad1a6b68e089008c369a9a5b6f5b37fef3e" May 28 16:15:57.893238 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.893166 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-ldjsl" event={"ID":"758df53e-4133-4eaa-bdbf-bcb7f469f44a","Type":"ContainerStarted","Data":"776f5f58c17a11f3570c803bd5a08580303860f69326d22280f1e27b88e7e4f2"} May 28 16:15:57.893238 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.893200 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-ldjsl" event={"ID":"758df53e-4133-4eaa-bdbf-bcb7f469f44a","Type":"ContainerStarted","Data":"d4666715e1f6e49b204ed3a710e8651a48d166c3a18914946732200ca6255d6d"} May 28 16:15:57.893413 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.893271 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-ldjsl" May 28 16:15:57.900095 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.900070 2566 scope.go:117] "RemoveContainer" containerID="81afcf0a68aa74bc17ff65705d8fcad1a6b68e089008c369a9a5b6f5b37fef3e" May 28 16:15:57.900408 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:15:57.900387 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81afcf0a68aa74bc17ff65705d8fcad1a6b68e089008c369a9a5b6f5b37fef3e\": container with ID starting with 81afcf0a68aa74bc17ff65705d8fcad1a6b68e089008c369a9a5b6f5b37fef3e not found: ID does not exist" containerID="81afcf0a68aa74bc17ff65705d8fcad1a6b68e089008c369a9a5b6f5b37fef3e" May 28 16:15:57.900470 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.900417 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81afcf0a68aa74bc17ff65705d8fcad1a6b68e089008c369a9a5b6f5b37fef3e"} err="failed to get container status \"81afcf0a68aa74bc17ff65705d8fcad1a6b68e089008c369a9a5b6f5b37fef3e\": rpc error: code = NotFound desc = could not find container \"81afcf0a68aa74bc17ff65705d8fcad1a6b68e089008c369a9a5b6f5b37fef3e\": container with ID starting with 81afcf0a68aa74bc17ff65705d8fcad1a6b68e089008c369a9a5b6f5b37fef3e not found: ID does not exist" May 28 16:15:57.911575 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.911547 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2nxr"] May 28 16:15:57.914707 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.914685 2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2nxr"] May 28 16:15:57.930851 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:57.930802 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-ldjsl" podStartSLOduration=0.930789211 podStartE2EDuration="930.789211ms" podCreationTimestamp="2026-05-28 16:15:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 16:15:57.929084488 +0000 UTC m=+497.156115043" watchObservedRunningTime="2026-05-28 16:15:57.930789211 +0000 UTC m=+497.157819767" May 28 16:15:59.336657 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:15:59.336623 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af626d62-0b3c-4859-88f6-06e59921a4f4" path="/var/lib/kubelet/pods/af626d62-0b3c-4859-88f6-06e59921a4f4/volumes" May 28 16:16:08.900006 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:08.899974 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-ldjsl" May 28 16:16:08.951825 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:08.951791 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-24p6g"] May 28 16:16:08.952050 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:08.952022 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-24p6g" podUID="35c45c40-4aca-407d-ac1c-0d7e65f5cb25" containerName="manager" containerID="cri-o://0d597cfd5869dd5391385002847eac192cc2a9674ff766ffdb2cc89aa2bfedf5" gracePeriod=10 May 28 16:16:09.188026 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:09.188005 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-24p6g" May 28 16:16:09.235746 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:09.235705 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sm6vx\" (UniqueName: \"kubernetes.io/projected/35c45c40-4aca-407d-ac1c-0d7e65f5cb25-kube-api-access-sm6vx\") pod \"35c45c40-4aca-407d-ac1c-0d7e65f5cb25\" (UID: \"35c45c40-4aca-407d-ac1c-0d7e65f5cb25\") " May 28 16:16:09.235934 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:09.235757 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35c45c40-4aca-407d-ac1c-0d7e65f5cb25-extensions-socket-volume\") pod \"35c45c40-4aca-407d-ac1c-0d7e65f5cb25\" (UID: \"35c45c40-4aca-407d-ac1c-0d7e65f5cb25\") " May 28 16:16:09.236162 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:09.236141 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35c45c40-4aca-407d-ac1c-0d7e65f5cb25-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "35c45c40-4aca-407d-ac1c-0d7e65f5cb25" (UID: "35c45c40-4aca-407d-ac1c-0d7e65f5cb25"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 28 16:16:09.237981 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:09.237952 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35c45c40-4aca-407d-ac1c-0d7e65f5cb25-kube-api-access-sm6vx" (OuterVolumeSpecName: "kube-api-access-sm6vx") pod "35c45c40-4aca-407d-ac1c-0d7e65f5cb25" (UID: "35c45c40-4aca-407d-ac1c-0d7e65f5cb25"). InnerVolumeSpecName "kube-api-access-sm6vx". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 16:16:09.338891 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:09.338851 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sm6vx\" (UniqueName: \"kubernetes.io/projected/35c45c40-4aca-407d-ac1c-0d7e65f5cb25-kube-api-access-sm6vx\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" May 28 16:16:09.338891 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:09.338888 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35c45c40-4aca-407d-ac1c-0d7e65f5cb25-extensions-socket-volume\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" May 28 16:16:09.935341 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:09.935303 2566 generic.go:358] "Generic (PLEG): container finished" podID="35c45c40-4aca-407d-ac1c-0d7e65f5cb25" containerID="0d597cfd5869dd5391385002847eac192cc2a9674ff766ffdb2cc89aa2bfedf5" exitCode=0 May 28 16:16:09.935810 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:09.935368 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-24p6g" event={"ID":"35c45c40-4aca-407d-ac1c-0d7e65f5cb25","Type":"ContainerDied","Data":"0d597cfd5869dd5391385002847eac192cc2a9674ff766ffdb2cc89aa2bfedf5"} May 28 16:16:09.935810 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:09.935374 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-24p6g" May 28 16:16:09.935810 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:09.935394 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-24p6g" event={"ID":"35c45c40-4aca-407d-ac1c-0d7e65f5cb25","Type":"ContainerDied","Data":"de84f8f90870cb3519bf5fac842d4ca136e94b0f9514ce9a26089662543cff8f"} May 28 16:16:09.935810 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:09.935410 2566 scope.go:117] "RemoveContainer" containerID="0d597cfd5869dd5391385002847eac192cc2a9674ff766ffdb2cc89aa2bfedf5" May 28 16:16:09.943331 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:09.943270 2566 scope.go:117] "RemoveContainer" containerID="0d597cfd5869dd5391385002847eac192cc2a9674ff766ffdb2cc89aa2bfedf5" May 28 16:16:09.943675 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:16:09.943651 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d597cfd5869dd5391385002847eac192cc2a9674ff766ffdb2cc89aa2bfedf5\": container with ID starting with 0d597cfd5869dd5391385002847eac192cc2a9674ff766ffdb2cc89aa2bfedf5 not found: ID does not exist" containerID="0d597cfd5869dd5391385002847eac192cc2a9674ff766ffdb2cc89aa2bfedf5" May 28 16:16:09.943720 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:09.943685 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d597cfd5869dd5391385002847eac192cc2a9674ff766ffdb2cc89aa2bfedf5"} err="failed to get container status \"0d597cfd5869dd5391385002847eac192cc2a9674ff766ffdb2cc89aa2bfedf5\": rpc error: code = NotFound desc = could not find container \"0d597cfd5869dd5391385002847eac192cc2a9674ff766ffdb2cc89aa2bfedf5\": container with ID starting with 0d597cfd5869dd5391385002847eac192cc2a9674ff766ffdb2cc89aa2bfedf5 not found: ID does not exist" May 28 16:16:09.953179 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:09.953153 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-24p6g"] May 28 16:16:09.959554 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:09.959533 2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-24p6g"] May 28 16:16:11.337029 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:11.336994 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35c45c40-4aca-407d-ac1c-0d7e65f5cb25" path="/var/lib/kubelet/pods/35c45c40-4aca-407d-ac1c-0d7e65f5cb25/volumes" May 28 16:16:13.337262 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.337230 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v"] May 28 16:16:13.337625 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.337501 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35c45c40-4aca-407d-ac1c-0d7e65f5cb25" containerName="manager" May 28 16:16:13.337625 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.337512 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="35c45c40-4aca-407d-ac1c-0d7e65f5cb25" containerName="manager" May 28 16:16:13.337625 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.337582 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="35c45c40-4aca-407d-ac1c-0d7e65f5cb25" containerName="manager" May 28 16:16:13.340023 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.340006 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" May 28 16:16:13.342543 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.342523 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-kksks\"" May 28 16:16:13.351576 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.351210 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v"] May 28 16:16:13.472890 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.472845 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrjw6\" (UniqueName: \"kubernetes.io/projected/73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499-kube-api-access-qrjw6\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-xrk5v\" (UID: \"73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" May 28 16:16:13.472890 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.472888 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-xrk5v\" (UID: \"73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" May 28 16:16:13.473124 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.472908 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499-istio-token\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-xrk5v\" (UID: \"73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" May 28 16:16:13.473124 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.472933 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-xrk5v\" (UID: \"73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" May 28 16:16:13.473124 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.472985 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499-workload-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-xrk5v\" (UID: \"73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" May 28 16:16:13.473124 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.473011 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-xrk5v\" (UID: \"73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" May 28 16:16:13.473124 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.473040 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499-credential-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-xrk5v\" (UID: \"73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" May 28 16:16:13.473124 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.473107 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499-workload-certs\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-xrk5v\" (UID: \"73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" May 28 16:16:13.473367 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.473141 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499-istio-data\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-xrk5v\" (UID: \"73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" May 28 16:16:13.574007 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.573964 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499-workload-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-xrk5v\" (UID: \"73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" May 28 16:16:13.574007 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.574007 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-xrk5v\" (UID: \"73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" May 28 16:16:13.574260 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.574042 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499-credential-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-xrk5v\" (UID: \"73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" May 28 16:16:13.574260 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.574084 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499-workload-certs\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-xrk5v\" (UID: \"73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" May 28 16:16:13.574260 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.574114 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499-istio-data\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-xrk5v\" (UID: \"73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" May 28 16:16:13.574260 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.574146 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qrjw6\" (UniqueName: \"kubernetes.io/projected/73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499-kube-api-access-qrjw6\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-xrk5v\" (UID: \"73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" May 28 16:16:13.574260 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.574174 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-xrk5v\" (UID: \"73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" May 28 16:16:13.574260 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.574203 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499-istio-token\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-xrk5v\" (UID: \"73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" May 28 16:16:13.574260 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.574228 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-xrk5v\" (UID: \"73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" May 28 16:16:13.574702 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.574675 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499-istio-data\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-xrk5v\" (UID: \"73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" May 28 16:16:13.574755 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.574705 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499-workload-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-xrk5v\" (UID: \"73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" May 28 16:16:13.574755 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.574739 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499-credential-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-xrk5v\" (UID: \"73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" May 28 16:16:13.574855 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.574815 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499-workload-certs\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-xrk5v\" (UID: \"73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" May 28 16:16:13.575116 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.575092 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-xrk5v\" (UID: \"73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" May 28 16:16:13.576774 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.576753 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-xrk5v\" (UID: \"73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" May 28 16:16:13.576870 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.576794 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-xrk5v\" (UID: \"73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" May 28 16:16:13.582147 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.582121 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499-istio-token\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-xrk5v\" (UID: \"73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" May 28 16:16:13.582232 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.582170 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrjw6\" (UniqueName: \"kubernetes.io/projected/73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499-kube-api-access-qrjw6\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-xrk5v\" (UID: \"73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" May 28 16:16:13.652581 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.652485 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" May 28 16:16:13.783886 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.783850 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v"] May 28 16:16:13.787340 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:16:13.787314 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73f49f56_c9e6_4ce3_9ce3_cbd1b0f07499.slice/crio-30d14c7671b76ea7936f2c173556301366ffda1d471156b4303965d56d2bd3ae WatchSource:0}: Error finding container 30d14c7671b76ea7936f2c173556301366ffda1d471156b4303965d56d2bd3ae: Status 404 returned error can't find the container with id 30d14c7671b76ea7936f2c173556301366ffda1d471156b4303965d56d2bd3ae May 28 16:16:13.789498 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.789457 2566 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} May 28 16:16:13.789592 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.789520 2566 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} May 28 16:16:13.789592 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.789549 2566 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} May 28 16:16:13.956215 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.956114 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" event={"ID":"73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499","Type":"ContainerStarted","Data":"17f8d8d04db51e962dafb7c032911ce2435ef2725ea029a1e4a3417d2cec8fa1"} May 28 16:16:13.956215 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.956156 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" event={"ID":"73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499","Type":"ContainerStarted","Data":"30d14c7671b76ea7936f2c173556301366ffda1d471156b4303965d56d2bd3ae"} May 28 16:16:13.975705 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:13.975643 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" podStartSLOduration=0.97562873 podStartE2EDuration="975.62873ms" podCreationTimestamp="2026-05-28 16:16:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 16:16:13.973332396 +0000 UTC m=+513.200362953" watchObservedRunningTime="2026-05-28 16:16:13.97562873 +0000 UTC m=+513.202659286" May 28 16:16:14.653153 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:14.653110 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" May 28 16:16:14.658404 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:14.658377 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" May 28 16:16:14.959657 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:14.959577 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" May 28 16:16:14.960508 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:14.960488 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-xrk5v" May 28 16:16:24.835139 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:24.835101 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-8zgr5"] May 28 16:16:24.837430 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:24.837413 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-8zgr5" May 28 16:16:24.840000 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:24.839978 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-2lxkv\"" May 28 16:16:24.840104 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:24.840051 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" May 28 16:16:24.845968 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:24.845943 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-8zgr5"] May 28 16:16:24.930813 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:24.930780 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-8zgr5"] May 28 16:16:24.975709 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:24.975667 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9f11a414-e4b4-4c77-8fd1-1d4b915de778-config-file\") pod \"limitador-limitador-7d549b5b-8zgr5\" (UID: \"9f11a414-e4b4-4c77-8fd1-1d4b915de778\") " pod="kuadrant-system/limitador-limitador-7d549b5b-8zgr5" May 28 16:16:24.975868 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:24.975802 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbd67\" (UniqueName: \"kubernetes.io/projected/9f11a414-e4b4-4c77-8fd1-1d4b915de778-kube-api-access-cbd67\") pod \"limitador-limitador-7d549b5b-8zgr5\" (UID: \"9f11a414-e4b4-4c77-8fd1-1d4b915de778\") " pod="kuadrant-system/limitador-limitador-7d549b5b-8zgr5" May 28 16:16:25.076562 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:25.076524 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9f11a414-e4b4-4c77-8fd1-1d4b915de778-config-file\") pod \"limitador-limitador-7d549b5b-8zgr5\" (UID: \"9f11a414-e4b4-4c77-8fd1-1d4b915de778\") " pod="kuadrant-system/limitador-limitador-7d549b5b-8zgr5" May 28 16:16:25.076738 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:25.076602 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cbd67\" (UniqueName: \"kubernetes.io/projected/9f11a414-e4b4-4c77-8fd1-1d4b915de778-kube-api-access-cbd67\") pod \"limitador-limitador-7d549b5b-8zgr5\" (UID: \"9f11a414-e4b4-4c77-8fd1-1d4b915de778\") " pod="kuadrant-system/limitador-limitador-7d549b5b-8zgr5" May 28 16:16:25.077182 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:25.077163 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9f11a414-e4b4-4c77-8fd1-1d4b915de778-config-file\") pod \"limitador-limitador-7d549b5b-8zgr5\" (UID: \"9f11a414-e4b4-4c77-8fd1-1d4b915de778\") " pod="kuadrant-system/limitador-limitador-7d549b5b-8zgr5" May 28 16:16:25.084767 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:25.084743 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbd67\" (UniqueName: \"kubernetes.io/projected/9f11a414-e4b4-4c77-8fd1-1d4b915de778-kube-api-access-cbd67\") pod \"limitador-limitador-7d549b5b-8zgr5\" (UID: \"9f11a414-e4b4-4c77-8fd1-1d4b915de778\") " pod="kuadrant-system/limitador-limitador-7d549b5b-8zgr5" May 28 16:16:25.148982 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:25.148894 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-8zgr5" May 28 16:16:25.284069 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:25.284044 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-8zgr5"] May 28 16:16:25.286999 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:16:25.286959 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f11a414_e4b4_4c77_8fd1_1d4b915de778.slice/crio-f44a140df0d0644fb84f6251f4c37791bd554b08e9966fa9168534cce18982b4 WatchSource:0}: Error finding container f44a140df0d0644fb84f6251f4c37791bd554b08e9966fa9168534cce18982b4: Status 404 returned error can't find the container with id f44a140df0d0644fb84f6251f4c37791bd554b08e9966fa9168534cce18982b4 May 28 16:16:25.624933 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:25.624845 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-cfjht"] May 28 16:16:25.627947 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:25.627929 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-cfjht" May 28 16:16:25.630544 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:25.630521 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-kdsdw\"" May 28 16:16:25.632145 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:25.632113 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-cfjht"] May 28 16:16:25.782735 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:25.782704 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l6cf\" (UniqueName: \"kubernetes.io/projected/df29c2dd-78d7-4529-8b30-4385b3f9536f-kube-api-access-2l6cf\") pod \"authorino-f99f4b5cd-cfjht\" (UID: \"df29c2dd-78d7-4529-8b30-4385b3f9536f\") " pod="kuadrant-system/authorino-f99f4b5cd-cfjht" May 28 16:16:25.884111 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:25.884021 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2l6cf\" (UniqueName: \"kubernetes.io/projected/df29c2dd-78d7-4529-8b30-4385b3f9536f-kube-api-access-2l6cf\") pod \"authorino-f99f4b5cd-cfjht\" (UID: \"df29c2dd-78d7-4529-8b30-4385b3f9536f\") " pod="kuadrant-system/authorino-f99f4b5cd-cfjht" May 28 16:16:25.891368 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:25.891346 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l6cf\" (UniqueName: \"kubernetes.io/projected/df29c2dd-78d7-4529-8b30-4385b3f9536f-kube-api-access-2l6cf\") pod \"authorino-f99f4b5cd-cfjht\" (UID: \"df29c2dd-78d7-4529-8b30-4385b3f9536f\") " pod="kuadrant-system/authorino-f99f4b5cd-cfjht" May 28 16:16:25.938031 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:25.937992 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-cfjht" May 28 16:16:26.006101 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:26.005998 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-8zgr5" event={"ID":"9f11a414-e4b4-4c77-8fd1-1d4b915de778","Type":"ContainerStarted","Data":"f44a140df0d0644fb84f6251f4c37791bd554b08e9966fa9168534cce18982b4"} May 28 16:16:26.122371 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:26.122312 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-cfjht"] May 28 16:16:26.131606 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:16:26.131573 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf29c2dd_78d7_4529_8b30_4385b3f9536f.slice/crio-e4495cce9aabc468048dfb982308ce359ec3bc52631a2984ae83a75ab2379f00 WatchSource:0}: Error finding container e4495cce9aabc468048dfb982308ce359ec3bc52631a2984ae83a75ab2379f00: Status 404 returned error can't find the container with id e4495cce9aabc468048dfb982308ce359ec3bc52631a2984ae83a75ab2379f00 May 28 16:16:27.012078 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:27.012036 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-cfjht" event={"ID":"df29c2dd-78d7-4529-8b30-4385b3f9536f","Type":"ContainerStarted","Data":"e4495cce9aabc468048dfb982308ce359ec3bc52631a2984ae83a75ab2379f00"} May 28 16:16:29.545752 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:29.545715 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-cfjht"] May 28 16:16:30.025610 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:30.025564 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-8zgr5" event={"ID":"9f11a414-e4b4-4c77-8fd1-1d4b915de778","Type":"ContainerStarted","Data":"d4174b9d1ca8233a2b16507361bdf07a9f10f7dc289150db7d29b8f3d75b919c"} May 28 16:16:30.025816 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:30.025657 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7d549b5b-8zgr5" May 28 16:16:30.027361 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:30.027320 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-cfjht" event={"ID":"df29c2dd-78d7-4529-8b30-4385b3f9536f","Type":"ContainerStarted","Data":"4c02fd0e84d25d98a9c9ecc66d8446f551b5b36bda248e04b0afe1c85fa0eab1"} May 28 16:16:30.027491 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:30.027398 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-f99f4b5cd-cfjht" podUID="df29c2dd-78d7-4529-8b30-4385b3f9536f" containerName="authorino" containerID="cri-o://4c02fd0e84d25d98a9c9ecc66d8446f551b5b36bda248e04b0afe1c85fa0eab1" gracePeriod=30 May 28 16:16:30.042497 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:30.042451 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7d549b5b-8zgr5" podStartSLOduration=1.410519624 podStartE2EDuration="6.042437911s" podCreationTimestamp="2026-05-28 16:16:24 +0000 UTC" firstStartedPulling="2026-05-28 16:16:25.288981307 +0000 UTC m=+524.516011861" lastFinishedPulling="2026-05-28 16:16:29.9208996 +0000 UTC m=+529.147930148" observedRunningTime="2026-05-28 16:16:30.040019275 +0000 UTC m=+529.267049841" watchObservedRunningTime="2026-05-28 16:16:30.042437911 +0000 UTC m=+529.269468466" May 28 16:16:30.052902 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:30.052856 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-f99f4b5cd-cfjht" podStartSLOduration=1.260275223 podStartE2EDuration="5.052841638s" podCreationTimestamp="2026-05-28 16:16:25 +0000 UTC" firstStartedPulling="2026-05-28 16:16:26.134140292 +0000 UTC m=+525.361170830" lastFinishedPulling="2026-05-28 16:16:29.926706705 +0000 UTC m=+529.153737245" observedRunningTime="2026-05-28 16:16:30.052376666 +0000 UTC m=+529.279407222" watchObservedRunningTime="2026-05-28 16:16:30.052841638 +0000 UTC m=+529.279872228" May 28 16:16:30.283057 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:30.283027 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-cfjht" May 28 16:16:30.427783 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:30.427739 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2l6cf\" (UniqueName: \"kubernetes.io/projected/df29c2dd-78d7-4529-8b30-4385b3f9536f-kube-api-access-2l6cf\") pod \"df29c2dd-78d7-4529-8b30-4385b3f9536f\" (UID: \"df29c2dd-78d7-4529-8b30-4385b3f9536f\") " May 28 16:16:30.430020 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:30.429994 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df29c2dd-78d7-4529-8b30-4385b3f9536f-kube-api-access-2l6cf" (OuterVolumeSpecName: "kube-api-access-2l6cf") pod "df29c2dd-78d7-4529-8b30-4385b3f9536f" (UID: "df29c2dd-78d7-4529-8b30-4385b3f9536f"). InnerVolumeSpecName "kube-api-access-2l6cf". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 16:16:30.529492 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:30.529454 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2l6cf\" (UniqueName: \"kubernetes.io/projected/df29c2dd-78d7-4529-8b30-4385b3f9536f-kube-api-access-2l6cf\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" May 28 16:16:31.031796 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:31.031760 2566 generic.go:358] "Generic (PLEG): container finished" podID="df29c2dd-78d7-4529-8b30-4385b3f9536f" containerID="4c02fd0e84d25d98a9c9ecc66d8446f551b5b36bda248e04b0afe1c85fa0eab1" exitCode=2 May 28 16:16:31.032204 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:31.031813 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-cfjht" May 28 16:16:31.032204 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:31.031845 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-cfjht" event={"ID":"df29c2dd-78d7-4529-8b30-4385b3f9536f","Type":"ContainerDied","Data":"4c02fd0e84d25d98a9c9ecc66d8446f551b5b36bda248e04b0afe1c85fa0eab1"} May 28 16:16:31.032204 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:31.031886 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-cfjht" event={"ID":"df29c2dd-78d7-4529-8b30-4385b3f9536f","Type":"ContainerDied","Data":"e4495cce9aabc468048dfb982308ce359ec3bc52631a2984ae83a75ab2379f00"} May 28 16:16:31.032204 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:31.031906 2566 scope.go:117] "RemoveContainer" containerID="4c02fd0e84d25d98a9c9ecc66d8446f551b5b36bda248e04b0afe1c85fa0eab1" May 28 16:16:31.040973 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:31.040959 2566 scope.go:117] "RemoveContainer" containerID="4c02fd0e84d25d98a9c9ecc66d8446f551b5b36bda248e04b0afe1c85fa0eab1" May 28 16:16:31.041257 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:16:31.041238 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c02fd0e84d25d98a9c9ecc66d8446f551b5b36bda248e04b0afe1c85fa0eab1\": container with ID starting with 4c02fd0e84d25d98a9c9ecc66d8446f551b5b36bda248e04b0afe1c85fa0eab1 not found: ID does not exist" containerID="4c02fd0e84d25d98a9c9ecc66d8446f551b5b36bda248e04b0afe1c85fa0eab1" May 28 16:16:31.041394 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:31.041269 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c02fd0e84d25d98a9c9ecc66d8446f551b5b36bda248e04b0afe1c85fa0eab1"} err="failed to get container status \"4c02fd0e84d25d98a9c9ecc66d8446f551b5b36bda248e04b0afe1c85fa0eab1\": rpc error: code = NotFound desc = could not find container \"4c02fd0e84d25d98a9c9ecc66d8446f551b5b36bda248e04b0afe1c85fa0eab1\": container with ID starting with 4c02fd0e84d25d98a9c9ecc66d8446f551b5b36bda248e04b0afe1c85fa0eab1 not found: ID does not exist" May 28 16:16:31.051801 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:31.051780 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-cfjht"] May 28 16:16:31.054231 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:31.054211 2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-cfjht"] May 28 16:16:31.337301 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:31.337254 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df29c2dd-78d7-4529-8b30-4385b3f9536f" path="/var/lib/kubelet/pods/df29c2dd-78d7-4529-8b30-4385b3f9536f/volumes" May 28 16:16:40.453378 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:40.453332 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-8zgr5"] May 28 16:16:40.453965 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:40.453651 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-7d549b5b-8zgr5" podUID="9f11a414-e4b4-4c77-8fd1-1d4b915de778" containerName="limitador" containerID="cri-o://d4174b9d1ca8233a2b16507361bdf07a9f10f7dc289150db7d29b8f3d75b919c" gracePeriod=30 May 28 16:16:40.454479 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:40.454457 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7d549b5b-8zgr5" May 28 16:16:40.994205 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:40.994181 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-8zgr5" May 28 16:16:41.010607 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:41.010499 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9f11a414-e4b4-4c77-8fd1-1d4b915de778-config-file\") pod \"9f11a414-e4b4-4c77-8fd1-1d4b915de778\" (UID: \"9f11a414-e4b4-4c77-8fd1-1d4b915de778\") " May 28 16:16:41.010766 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:41.010582 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbd67\" (UniqueName: \"kubernetes.io/projected/9f11a414-e4b4-4c77-8fd1-1d4b915de778-kube-api-access-cbd67\") pod \"9f11a414-e4b4-4c77-8fd1-1d4b915de778\" (UID: \"9f11a414-e4b4-4c77-8fd1-1d4b915de778\") " May 28 16:16:41.010990 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:41.010956 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f11a414-e4b4-4c77-8fd1-1d4b915de778-config-file" (OuterVolumeSpecName: "config-file") pod "9f11a414-e4b4-4c77-8fd1-1d4b915de778" (UID: "9f11a414-e4b4-4c77-8fd1-1d4b915de778"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 16:16:41.013386 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:41.013354 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f11a414-e4b4-4c77-8fd1-1d4b915de778-kube-api-access-cbd67" (OuterVolumeSpecName: "kube-api-access-cbd67") pod "9f11a414-e4b4-4c77-8fd1-1d4b915de778" (UID: "9f11a414-e4b4-4c77-8fd1-1d4b915de778"). InnerVolumeSpecName "kube-api-access-cbd67". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 16:16:41.067469 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:41.067428 2566 generic.go:358] "Generic (PLEG): container finished" podID="9f11a414-e4b4-4c77-8fd1-1d4b915de778" containerID="d4174b9d1ca8233a2b16507361bdf07a9f10f7dc289150db7d29b8f3d75b919c" exitCode=0 May 28 16:16:41.067635 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:41.067491 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-8zgr5" May 28 16:16:41.067635 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:41.067519 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-8zgr5" event={"ID":"9f11a414-e4b4-4c77-8fd1-1d4b915de778","Type":"ContainerDied","Data":"d4174b9d1ca8233a2b16507361bdf07a9f10f7dc289150db7d29b8f3d75b919c"} May 28 16:16:41.067635 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:41.067552 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-8zgr5" event={"ID":"9f11a414-e4b4-4c77-8fd1-1d4b915de778","Type":"ContainerDied","Data":"f44a140df0d0644fb84f6251f4c37791bd554b08e9966fa9168534cce18982b4"} May 28 16:16:41.067635 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:41.067566 2566 scope.go:117] "RemoveContainer" containerID="d4174b9d1ca8233a2b16507361bdf07a9f10f7dc289150db7d29b8f3d75b919c" May 28 16:16:41.076320 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:41.076300 2566 scope.go:117] "RemoveContainer" containerID="d4174b9d1ca8233a2b16507361bdf07a9f10f7dc289150db7d29b8f3d75b919c" May 28 16:16:41.076624 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:16:41.076602 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4174b9d1ca8233a2b16507361bdf07a9f10f7dc289150db7d29b8f3d75b919c\": container with ID starting with d4174b9d1ca8233a2b16507361bdf07a9f10f7dc289150db7d29b8f3d75b919c not found: ID does not exist" containerID="d4174b9d1ca8233a2b16507361bdf07a9f10f7dc289150db7d29b8f3d75b919c" May 28 16:16:41.076671 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:41.076635 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4174b9d1ca8233a2b16507361bdf07a9f10f7dc289150db7d29b8f3d75b919c"} err="failed to get container status \"d4174b9d1ca8233a2b16507361bdf07a9f10f7dc289150db7d29b8f3d75b919c\": rpc error: code = NotFound desc = could not find container \"d4174b9d1ca8233a2b16507361bdf07a9f10f7dc289150db7d29b8f3d75b919c\": container with ID starting with d4174b9d1ca8233a2b16507361bdf07a9f10f7dc289150db7d29b8f3d75b919c not found: ID does not exist" May 28 16:16:41.088386 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:41.088347 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-8zgr5"] May 28 16:16:41.090890 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:41.090865 2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-8zgr5"] May 28 16:16:41.111555 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:41.111524 2566 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9f11a414-e4b4-4c77-8fd1-1d4b915de778-config-file\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" May 28 16:16:41.111555 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:41.111554 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbd67\" (UniqueName: \"kubernetes.io/projected/9f11a414-e4b4-4c77-8fd1-1d4b915de778-kube-api-access-cbd67\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" May 28 16:16:41.337516 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:41.337443 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f11a414-e4b4-4c77-8fd1-1d4b915de778" path="/var/lib/kubelet/pods/9f11a414-e4b4-4c77-8fd1-1d4b915de778/volumes" May 28 16:16:43.472152 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:43.472115 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-68c4fbbd6f-mnfdq"] May 28 16:16:43.472584 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:43.472433 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df29c2dd-78d7-4529-8b30-4385b3f9536f" containerName="authorino" May 28 16:16:43.472584 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:43.472445 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="df29c2dd-78d7-4529-8b30-4385b3f9536f" containerName="authorino" May 28 16:16:43.472584 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:43.472457 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f11a414-e4b4-4c77-8fd1-1d4b915de778" containerName="limitador" May 28 16:16:43.472584 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:43.472462 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f11a414-e4b4-4c77-8fd1-1d4b915de778" containerName="limitador" May 28 16:16:43.472584 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:43.472515 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="df29c2dd-78d7-4529-8b30-4385b3f9536f" containerName="authorino" May 28 16:16:43.472584 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:43.472523 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="9f11a414-e4b4-4c77-8fd1-1d4b915de778" containerName="limitador" May 28 16:16:43.475511 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:43.475494 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-mnfdq" May 28 16:16:43.477969 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:43.477947 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-q2xnt\"" May 28 16:16:43.478449 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:43.478427 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" May 28 16:16:43.484371 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:43.484349 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-mnfdq"] May 28 16:16:43.529632 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:43.529593 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/415ee9e9-d535-4622-8634-5aed28c23d70-data\") pod \"postgres-68c4fbbd6f-mnfdq\" (UID: \"415ee9e9-d535-4622-8634-5aed28c23d70\") " pod="opendatahub/postgres-68c4fbbd6f-mnfdq" May 28 16:16:43.529803 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:43.529660 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbxws\" (UniqueName: \"kubernetes.io/projected/415ee9e9-d535-4622-8634-5aed28c23d70-kube-api-access-jbxws\") pod \"postgres-68c4fbbd6f-mnfdq\" (UID: \"415ee9e9-d535-4622-8634-5aed28c23d70\") " pod="opendatahub/postgres-68c4fbbd6f-mnfdq" May 28 16:16:43.630229 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:43.630189 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/415ee9e9-d535-4622-8634-5aed28c23d70-data\") pod \"postgres-68c4fbbd6f-mnfdq\" (UID: \"415ee9e9-d535-4622-8634-5aed28c23d70\") " pod="opendatahub/postgres-68c4fbbd6f-mnfdq" May 28 16:16:43.630452 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:43.630262 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jbxws\" (UniqueName: \"kubernetes.io/projected/415ee9e9-d535-4622-8634-5aed28c23d70-kube-api-access-jbxws\") pod \"postgres-68c4fbbd6f-mnfdq\" (UID: \"415ee9e9-d535-4622-8634-5aed28c23d70\") " pod="opendatahub/postgres-68c4fbbd6f-mnfdq" May 28 16:16:43.630612 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:43.630591 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/415ee9e9-d535-4622-8634-5aed28c23d70-data\") pod \"postgres-68c4fbbd6f-mnfdq\" (UID: \"415ee9e9-d535-4622-8634-5aed28c23d70\") " pod="opendatahub/postgres-68c4fbbd6f-mnfdq" May 28 16:16:43.639131 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:43.639107 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbxws\" (UniqueName: \"kubernetes.io/projected/415ee9e9-d535-4622-8634-5aed28c23d70-kube-api-access-jbxws\") pod \"postgres-68c4fbbd6f-mnfdq\" (UID: \"415ee9e9-d535-4622-8634-5aed28c23d70\") " pod="opendatahub/postgres-68c4fbbd6f-mnfdq" May 28 16:16:43.788052 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:43.788010 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-mnfdq" May 28 16:16:43.918589 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:43.918566 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-mnfdq"] May 28 16:16:43.920991 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:16:43.920945 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod415ee9e9_d535_4622_8634_5aed28c23d70.slice/crio-494aa7a15bb75b716a56d796402a1370af3d872380735a625f671679338295e9 WatchSource:0}: Error finding container 494aa7a15bb75b716a56d796402a1370af3d872380735a625f671679338295e9: Status 404 returned error can't find the container with id 494aa7a15bb75b716a56d796402a1370af3d872380735a625f671679338295e9 May 28 16:16:44.076895 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:44.076807 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-mnfdq" event={"ID":"415ee9e9-d535-4622-8634-5aed28c23d70","Type":"ContainerStarted","Data":"494aa7a15bb75b716a56d796402a1370af3d872380735a625f671679338295e9"} May 28 16:16:49.095888 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:49.095848 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-mnfdq" event={"ID":"415ee9e9-d535-4622-8634-5aed28c23d70","Type":"ContainerStarted","Data":"053fc160b66c00bae9e2195bee8245f8e2e4a780aa121b4ff84235a672c30975"} May 28 16:16:49.096261 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:49.095950 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-68c4fbbd6f-mnfdq" May 28 16:16:49.111380 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:49.111319 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/postgres-68c4fbbd6f-mnfdq" podStartSLOduration=1.357035228 podStartE2EDuration="6.111302156s" podCreationTimestamp="2026-05-28 16:16:43 +0000 UTC" firstStartedPulling="2026-05-28 16:16:43.922546243 +0000 UTC m=+543.149576793" lastFinishedPulling="2026-05-28 16:16:48.676813185 +0000 UTC m=+547.903843721" observedRunningTime="2026-05-28 16:16:49.110403947 +0000 UTC m=+548.337434504" watchObservedRunningTime="2026-05-28 16:16:49.111302156 +0000 UTC m=+548.338332708" May 28 16:16:55.127566 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:55.127533 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-68c4fbbd6f-mnfdq" May 28 16:16:55.702138 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:55.702096 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8b475cf9f-b2nvt"] May 28 16:16:55.704884 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:55.704859 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-b2nvt" May 28 16:16:55.707220 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:55.707190 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-kdsdw\"" May 28 16:16:55.711203 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:55.711181 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-b2nvt"] May 28 16:16:55.735175 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:55.735149 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggjm8\" (UniqueName: \"kubernetes.io/projected/67aee331-bf79-4412-b1c3-e67b7171abdf-kube-api-access-ggjm8\") pod \"authorino-8b475cf9f-b2nvt\" (UID: \"67aee331-bf79-4412-b1c3-e67b7171abdf\") " pod="kuadrant-system/authorino-8b475cf9f-b2nvt" May 28 16:16:55.835741 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:55.835709 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ggjm8\" (UniqueName: \"kubernetes.io/projected/67aee331-bf79-4412-b1c3-e67b7171abdf-kube-api-access-ggjm8\") pod \"authorino-8b475cf9f-b2nvt\" (UID: \"67aee331-bf79-4412-b1c3-e67b7171abdf\") " pod="kuadrant-system/authorino-8b475cf9f-b2nvt" May 28 16:16:55.848904 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:55.848867 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggjm8\" (UniqueName: \"kubernetes.io/projected/67aee331-bf79-4412-b1c3-e67b7171abdf-kube-api-access-ggjm8\") pod \"authorino-8b475cf9f-b2nvt\" (UID: \"67aee331-bf79-4412-b1c3-e67b7171abdf\") " pod="kuadrant-system/authorino-8b475cf9f-b2nvt" May 28 16:16:55.867762 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:55.867734 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-b2nvt"] May 28 16:16:55.867967 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:55.867956 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-b2nvt" May 28 16:16:55.897224 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:55.897190 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56fdd757f5-wr2bj"] May 28 16:16:55.900101 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:55.900075 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56fdd757f5-wr2bj" May 28 16:16:55.902457 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:55.902436 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" May 28 16:16:55.936817 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:55.936783 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a6f122e8-8d41-41a9-be9d-be43a8ec506b-tls-cert\") pod \"authorino-56fdd757f5-wr2bj\" (UID: \"a6f122e8-8d41-41a9-be9d-be43a8ec506b\") " pod="kuadrant-system/authorino-56fdd757f5-wr2bj" May 28 16:16:55.936954 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:55.936826 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6sw5\" (UniqueName: \"kubernetes.io/projected/a6f122e8-8d41-41a9-be9d-be43a8ec506b-kube-api-access-l6sw5\") pod \"authorino-56fdd757f5-wr2bj\" (UID: \"a6f122e8-8d41-41a9-be9d-be43a8ec506b\") " pod="kuadrant-system/authorino-56fdd757f5-wr2bj" May 28 16:16:55.944605 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:55.944548 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56fdd757f5-wr2bj"] May 28 16:16:55.963529 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:55.963451 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56fdd757f5-wr2bj"] May 28 16:16:55.963711 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:16:55.963691 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-l6sw5 tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-56fdd757f5-wr2bj" podUID="a6f122e8-8d41-41a9-be9d-be43a8ec506b" May 28 16:16:55.989667 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:55.989636 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-85bd9b9885-kj45q"] May 28 16:16:55.993806 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:55.993532 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85bd9b9885-kj45q" May 28 16:16:56.006159 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:56.006135 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-85bd9b9885-kj45q"] May 28 16:16:56.007496 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:56.007472 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-b2nvt"] May 28 16:16:56.009206 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:16:56.009183 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod67aee331_bf79_4412_b1c3_e67b7171abdf.slice/crio-f14c2ecc025b41a740d7bac3e2d74bbf41d8290d2fd62ba314592d1233daedbd WatchSource:0}: Error finding container f14c2ecc025b41a740d7bac3e2d74bbf41d8290d2fd62ba314592d1233daedbd: Status 404 returned error can't find the container with id f14c2ecc025b41a740d7bac3e2d74bbf41d8290d2fd62ba314592d1233daedbd May 28 16:16:56.037695 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:56.037668 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f8bc\" (UniqueName: \"kubernetes.io/projected/7c2708a4-2139-4ada-845f-9cf4248a1ac8-kube-api-access-4f8bc\") pod \"authorino-85bd9b9885-kj45q\" (UID: \"7c2708a4-2139-4ada-845f-9cf4248a1ac8\") " pod="kuadrant-system/authorino-85bd9b9885-kj45q" May 28 16:16:56.037837 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:56.037735 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7c2708a4-2139-4ada-845f-9cf4248a1ac8-tls-cert\") pod \"authorino-85bd9b9885-kj45q\" (UID: \"7c2708a4-2139-4ada-845f-9cf4248a1ac8\") " pod="kuadrant-system/authorino-85bd9b9885-kj45q" May 28 16:16:56.037837 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:56.037774 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a6f122e8-8d41-41a9-be9d-be43a8ec506b-tls-cert\") pod \"authorino-56fdd757f5-wr2bj\" (UID: \"a6f122e8-8d41-41a9-be9d-be43a8ec506b\") " pod="kuadrant-system/authorino-56fdd757f5-wr2bj" May 28 16:16:56.037837 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:56.037799 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l6sw5\" (UniqueName: \"kubernetes.io/projected/a6f122e8-8d41-41a9-be9d-be43a8ec506b-kube-api-access-l6sw5\") pod \"authorino-56fdd757f5-wr2bj\" (UID: \"a6f122e8-8d41-41a9-be9d-be43a8ec506b\") " pod="kuadrant-system/authorino-56fdd757f5-wr2bj" May 28 16:16:56.040328 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:56.040307 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a6f122e8-8d41-41a9-be9d-be43a8ec506b-tls-cert\") pod \"authorino-56fdd757f5-wr2bj\" (UID: \"a6f122e8-8d41-41a9-be9d-be43a8ec506b\") " pod="kuadrant-system/authorino-56fdd757f5-wr2bj" May 28 16:16:56.045686 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:56.045663 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6sw5\" (UniqueName: \"kubernetes.io/projected/a6f122e8-8d41-41a9-be9d-be43a8ec506b-kube-api-access-l6sw5\") pod \"authorino-56fdd757f5-wr2bj\" (UID: \"a6f122e8-8d41-41a9-be9d-be43a8ec506b\") " pod="kuadrant-system/authorino-56fdd757f5-wr2bj" May 28 16:16:56.119173 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:56.119136 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-b2nvt" event={"ID":"67aee331-bf79-4412-b1c3-e67b7171abdf","Type":"ContainerStarted","Data":"f14c2ecc025b41a740d7bac3e2d74bbf41d8290d2fd62ba314592d1233daedbd"} May 28 16:16:56.119173 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:56.119158 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56fdd757f5-wr2bj" May 28 16:16:56.123749 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:56.123719 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56fdd757f5-wr2bj" May 28 16:16:56.138788 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:56.138765 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7c2708a4-2139-4ada-845f-9cf4248a1ac8-tls-cert\") pod \"authorino-85bd9b9885-kj45q\" (UID: \"7c2708a4-2139-4ada-845f-9cf4248a1ac8\") " pod="kuadrant-system/authorino-85bd9b9885-kj45q" May 28 16:16:56.139080 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:56.138836 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4f8bc\" (UniqueName: \"kubernetes.io/projected/7c2708a4-2139-4ada-845f-9cf4248a1ac8-kube-api-access-4f8bc\") pod \"authorino-85bd9b9885-kj45q\" (UID: \"7c2708a4-2139-4ada-845f-9cf4248a1ac8\") " pod="kuadrant-system/authorino-85bd9b9885-kj45q" May 28 16:16:56.141458 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:56.141439 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7c2708a4-2139-4ada-845f-9cf4248a1ac8-tls-cert\") pod \"authorino-85bd9b9885-kj45q\" (UID: \"7c2708a4-2139-4ada-845f-9cf4248a1ac8\") " pod="kuadrant-system/authorino-85bd9b9885-kj45q" May 28 16:16:56.146830 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:56.146806 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f8bc\" (UniqueName: \"kubernetes.io/projected/7c2708a4-2139-4ada-845f-9cf4248a1ac8-kube-api-access-4f8bc\") pod \"authorino-85bd9b9885-kj45q\" (UID: \"7c2708a4-2139-4ada-845f-9cf4248a1ac8\") " pod="kuadrant-system/authorino-85bd9b9885-kj45q" May 28 16:16:56.240134 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:56.240099 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a6f122e8-8d41-41a9-be9d-be43a8ec506b-tls-cert\") pod \"a6f122e8-8d41-41a9-be9d-be43a8ec506b\" (UID: \"a6f122e8-8d41-41a9-be9d-be43a8ec506b\") " May 28 16:16:56.240317 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:56.240190 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6sw5\" (UniqueName: \"kubernetes.io/projected/a6f122e8-8d41-41a9-be9d-be43a8ec506b-kube-api-access-l6sw5\") pod \"a6f122e8-8d41-41a9-be9d-be43a8ec506b\" (UID: \"a6f122e8-8d41-41a9-be9d-be43a8ec506b\") " May 28 16:16:56.242435 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:56.242404 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6f122e8-8d41-41a9-be9d-be43a8ec506b-kube-api-access-l6sw5" (OuterVolumeSpecName: "kube-api-access-l6sw5") pod "a6f122e8-8d41-41a9-be9d-be43a8ec506b" (UID: "a6f122e8-8d41-41a9-be9d-be43a8ec506b"). InnerVolumeSpecName "kube-api-access-l6sw5". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 16:16:56.242435 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:56.242423 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6f122e8-8d41-41a9-be9d-be43a8ec506b-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "a6f122e8-8d41-41a9-be9d-be43a8ec506b" (UID: "a6f122e8-8d41-41a9-be9d-be43a8ec506b"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 16:16:56.302582 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:56.302543 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85bd9b9885-kj45q" May 28 16:16:56.341810 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:56.341775 2566 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a6f122e8-8d41-41a9-be9d-be43a8ec506b-tls-cert\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" May 28 16:16:56.341810 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:56.341805 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6sw5\" (UniqueName: \"kubernetes.io/projected/a6f122e8-8d41-41a9-be9d-be43a8ec506b-kube-api-access-l6sw5\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" May 28 16:16:56.436633 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:56.436553 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-85bd9b9885-kj45q"] May 28 16:16:56.439256 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:16:56.439232 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7c2708a4_2139_4ada_845f_9cf4248a1ac8.slice/crio-54316b3bba72d74df15e002db0204d4a12220071309708e8bb0fb5ccffee3cca WatchSource:0}: Error finding container 54316b3bba72d74df15e002db0204d4a12220071309708e8bb0fb5ccffee3cca: Status 404 returned error can't find the container with id 54316b3bba72d74df15e002db0204d4a12220071309708e8bb0fb5ccffee3cca May 28 16:16:57.124397 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:57.124291 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-b2nvt" event={"ID":"67aee331-bf79-4412-b1c3-e67b7171abdf","Type":"ContainerStarted","Data":"3daf12e62381a18a1d9f5c79988509dd5e4acb0133f5eb27aa9fda1141e48b41"} May 28 16:16:57.124397 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:57.124369 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-8b475cf9f-b2nvt" podUID="67aee331-bf79-4412-b1c3-e67b7171abdf" containerName="authorino" containerID="cri-o://3daf12e62381a18a1d9f5c79988509dd5e4acb0133f5eb27aa9fda1141e48b41" gracePeriod=30 May 28 16:16:57.125826 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:57.125777 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-85bd9b9885-kj45q" event={"ID":"7c2708a4-2139-4ada-845f-9cf4248a1ac8","Type":"ContainerStarted","Data":"bb33238390d27759a58531172d583620aa99d8835fd5d288b3e0eca324436f28"} May 28 16:16:57.125826 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:57.125812 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-85bd9b9885-kj45q" event={"ID":"7c2708a4-2139-4ada-845f-9cf4248a1ac8","Type":"ContainerStarted","Data":"54316b3bba72d74df15e002db0204d4a12220071309708e8bb0fb5ccffee3cca"} May 28 16:16:57.125826 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:57.125829 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56fdd757f5-wr2bj" May 28 16:16:57.139043 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:57.139001 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-8b475cf9f-b2nvt" podStartSLOduration=1.7509123180000001 podStartE2EDuration="2.13898647s" podCreationTimestamp="2026-05-28 16:16:55 +0000 UTC" firstStartedPulling="2026-05-28 16:16:56.010527898 +0000 UTC m=+555.237558433" lastFinishedPulling="2026-05-28 16:16:56.398602037 +0000 UTC m=+555.625632585" observedRunningTime="2026-05-28 16:16:57.138311521 +0000 UTC m=+556.365342075" watchObservedRunningTime="2026-05-28 16:16:57.13898647 +0000 UTC m=+556.366017027" May 28 16:16:57.161696 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:57.161662 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56fdd757f5-wr2bj"] May 28 16:16:57.162914 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:57.162881 2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56fdd757f5-wr2bj"] May 28 16:16:57.175055 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:57.175003 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-85bd9b9885-kj45q" podStartSLOduration=1.7818571570000001 podStartE2EDuration="2.174989184s" podCreationTimestamp="2026-05-28 16:16:55 +0000 UTC" firstStartedPulling="2026-05-28 16:16:56.440456574 +0000 UTC m=+555.667487108" lastFinishedPulling="2026-05-28 16:16:56.833588598 +0000 UTC m=+556.060619135" observedRunningTime="2026-05-28 16:16:57.173479081 +0000 UTC m=+556.400509641" watchObservedRunningTime="2026-05-28 16:16:57.174989184 +0000 UTC m=+556.402019740" May 28 16:16:57.337205 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:57.337169 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6f122e8-8d41-41a9-be9d-be43a8ec506b" path="/var/lib/kubelet/pods/a6f122e8-8d41-41a9-be9d-be43a8ec506b/volumes" May 28 16:16:57.368440 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:57.368420 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-b2nvt" May 28 16:16:57.453173 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:57.453093 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ggjm8\" (UniqueName: \"kubernetes.io/projected/67aee331-bf79-4412-b1c3-e67b7171abdf-kube-api-access-ggjm8\") pod \"67aee331-bf79-4412-b1c3-e67b7171abdf\" (UID: \"67aee331-bf79-4412-b1c3-e67b7171abdf\") " May 28 16:16:57.455313 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:57.455265 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67aee331-bf79-4412-b1c3-e67b7171abdf-kube-api-access-ggjm8" (OuterVolumeSpecName: "kube-api-access-ggjm8") pod "67aee331-bf79-4412-b1c3-e67b7171abdf" (UID: "67aee331-bf79-4412-b1c3-e67b7171abdf"). InnerVolumeSpecName "kube-api-access-ggjm8". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 16:16:57.553699 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:57.553661 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ggjm8\" (UniqueName: \"kubernetes.io/projected/67aee331-bf79-4412-b1c3-e67b7171abdf-kube-api-access-ggjm8\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" May 28 16:16:58.130174 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:58.130079 2566 generic.go:358] "Generic (PLEG): container finished" podID="67aee331-bf79-4412-b1c3-e67b7171abdf" containerID="3daf12e62381a18a1d9f5c79988509dd5e4acb0133f5eb27aa9fda1141e48b41" exitCode=0 May 28 16:16:58.130174 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:58.130139 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-b2nvt" May 28 16:16:58.130174 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:58.130145 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-b2nvt" event={"ID":"67aee331-bf79-4412-b1c3-e67b7171abdf","Type":"ContainerDied","Data":"3daf12e62381a18a1d9f5c79988509dd5e4acb0133f5eb27aa9fda1141e48b41"} May 28 16:16:58.130485 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:58.130183 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-b2nvt" event={"ID":"67aee331-bf79-4412-b1c3-e67b7171abdf","Type":"ContainerDied","Data":"f14c2ecc025b41a740d7bac3e2d74bbf41d8290d2fd62ba314592d1233daedbd"} May 28 16:16:58.130485 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:58.130199 2566 scope.go:117] "RemoveContainer" containerID="3daf12e62381a18a1d9f5c79988509dd5e4acb0133f5eb27aa9fda1141e48b41" May 28 16:16:58.138471 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:58.138451 2566 scope.go:117] "RemoveContainer" containerID="3daf12e62381a18a1d9f5c79988509dd5e4acb0133f5eb27aa9fda1141e48b41" May 28 16:16:58.138727 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:16:58.138708 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3daf12e62381a18a1d9f5c79988509dd5e4acb0133f5eb27aa9fda1141e48b41\": container with ID starting with 3daf12e62381a18a1d9f5c79988509dd5e4acb0133f5eb27aa9fda1141e48b41 not found: ID does not exist" containerID="3daf12e62381a18a1d9f5c79988509dd5e4acb0133f5eb27aa9fda1141e48b41" May 28 16:16:58.138771 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:58.138738 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3daf12e62381a18a1d9f5c79988509dd5e4acb0133f5eb27aa9fda1141e48b41"} err="failed to get container status \"3daf12e62381a18a1d9f5c79988509dd5e4acb0133f5eb27aa9fda1141e48b41\": rpc error: code = NotFound desc = could not find container \"3daf12e62381a18a1d9f5c79988509dd5e4acb0133f5eb27aa9fda1141e48b41\": container with ID starting with 3daf12e62381a18a1d9f5c79988509dd5e4acb0133f5eb27aa9fda1141e48b41 not found: ID does not exist" May 28 16:16:58.150719 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:58.150697 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-b2nvt"] May 28 16:16:58.154070 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:58.154046 2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-b2nvt"] May 28 16:16:59.337004 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:16:59.336973 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67aee331-bf79-4412-b1c3-e67b7171abdf" path="/var/lib/kubelet/pods/67aee331-bf79-4412-b1c3-e67b7171abdf/volumes" May 28 16:17:41.221066 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:41.221032 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-db77q_d07065a1-27ff-4f50-9b8e-88d35f69bc89/ovn-acl-logging/0.log" May 28 16:17:41.221603 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:41.221167 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-db77q_d07065a1-27ff-4f50-9b8e-88d35f69bc89/ovn-acl-logging/0.log" May 28 16:17:46.169861 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:46.169821 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-lrp9w"] May 28 16:17:46.170243 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:46.170150 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67aee331-bf79-4412-b1c3-e67b7171abdf" containerName="authorino" May 28 16:17:46.170243 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:46.170164 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="67aee331-bf79-4412-b1c3-e67b7171abdf" containerName="authorino" May 28 16:17:46.170243 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:46.170217 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="67aee331-bf79-4412-b1c3-e67b7171abdf" containerName="authorino" May 28 16:17:46.172781 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:46.172763 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-lrp9w" May 28 16:17:46.176746 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:46.176725 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-fqmhw\"" May 28 16:17:46.176872 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:46.176746 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" May 28 16:17:46.176872 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:46.176789 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" May 28 16:17:46.176872 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:46.176791 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" May 28 16:17:46.184358 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:46.184334 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-lrp9w"] May 28 16:17:46.271320 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:46.271269 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9ba08fa-d3ea-4ef0-949c-c2f210aa3174-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-lrp9w\" (UID: \"a9ba08fa-d3ea-4ef0-949c-c2f210aa3174\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-lrp9w" May 28 16:17:46.271502 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:46.271331 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxtd6\" (UniqueName: \"kubernetes.io/projected/a9ba08fa-d3ea-4ef0-949c-c2f210aa3174-kube-api-access-pxtd6\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-lrp9w\" (UID: \"a9ba08fa-d3ea-4ef0-949c-c2f210aa3174\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-lrp9w" May 28 16:17:46.271502 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:46.271424 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9ba08fa-d3ea-4ef0-949c-c2f210aa3174-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-lrp9w\" (UID: \"a9ba08fa-d3ea-4ef0-949c-c2f210aa3174\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-lrp9w" May 28 16:17:46.271502 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:46.271460 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9ba08fa-d3ea-4ef0-949c-c2f210aa3174-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-lrp9w\" (UID: \"a9ba08fa-d3ea-4ef0-949c-c2f210aa3174\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-lrp9w" May 28 16:17:46.271502 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:46.271489 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a9ba08fa-d3ea-4ef0-949c-c2f210aa3174-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-lrp9w\" (UID: \"a9ba08fa-d3ea-4ef0-949c-c2f210aa3174\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-lrp9w" May 28 16:17:46.271657 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:46.271562 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9ba08fa-d3ea-4ef0-949c-c2f210aa3174-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-lrp9w\" (UID: \"a9ba08fa-d3ea-4ef0-949c-c2f210aa3174\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-lrp9w" May 28 16:17:46.372636 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:46.372592 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9ba08fa-d3ea-4ef0-949c-c2f210aa3174-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-lrp9w\" (UID: \"a9ba08fa-d3ea-4ef0-949c-c2f210aa3174\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-lrp9w" May 28 16:17:46.372823 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:46.372670 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pxtd6\" (UniqueName: \"kubernetes.io/projected/a9ba08fa-d3ea-4ef0-949c-c2f210aa3174-kube-api-access-pxtd6\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-lrp9w\" (UID: \"a9ba08fa-d3ea-4ef0-949c-c2f210aa3174\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-lrp9w" May 28 16:17:46.372823 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:46.372731 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9ba08fa-d3ea-4ef0-949c-c2f210aa3174-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-lrp9w\" (UID: \"a9ba08fa-d3ea-4ef0-949c-c2f210aa3174\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-lrp9w" May 28 16:17:46.372823 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:46.372755 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9ba08fa-d3ea-4ef0-949c-c2f210aa3174-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-lrp9w\" (UID: \"a9ba08fa-d3ea-4ef0-949c-c2f210aa3174\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-lrp9w" May 28 16:17:46.372823 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:46.372780 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a9ba08fa-d3ea-4ef0-949c-c2f210aa3174-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-lrp9w\" (UID: \"a9ba08fa-d3ea-4ef0-949c-c2f210aa3174\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-lrp9w" May 28 16:17:46.373042 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:46.372837 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9ba08fa-d3ea-4ef0-949c-c2f210aa3174-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-lrp9w\" (UID: \"a9ba08fa-d3ea-4ef0-949c-c2f210aa3174\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-lrp9w" May 28 16:17:46.373240 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:46.373218 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9ba08fa-d3ea-4ef0-949c-c2f210aa3174-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-lrp9w\" (UID: \"a9ba08fa-d3ea-4ef0-949c-c2f210aa3174\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-lrp9w" May 28 16:17:46.373349 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:46.373242 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9ba08fa-d3ea-4ef0-949c-c2f210aa3174-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-lrp9w\" (UID: \"a9ba08fa-d3ea-4ef0-949c-c2f210aa3174\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-lrp9w" May 28 16:17:46.373349 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:46.373334 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a9ba08fa-d3ea-4ef0-949c-c2f210aa3174-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-lrp9w\" (UID: \"a9ba08fa-d3ea-4ef0-949c-c2f210aa3174\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-lrp9w" May 28 16:17:46.375147 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:46.375121 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9ba08fa-d3ea-4ef0-949c-c2f210aa3174-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-lrp9w\" (UID: \"a9ba08fa-d3ea-4ef0-949c-c2f210aa3174\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-lrp9w" May 28 16:17:46.375419 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:46.375399 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9ba08fa-d3ea-4ef0-949c-c2f210aa3174-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-lrp9w\" (UID: \"a9ba08fa-d3ea-4ef0-949c-c2f210aa3174\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-lrp9w" May 28 16:17:46.380226 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:46.380201 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxtd6\" (UniqueName: \"kubernetes.io/projected/a9ba08fa-d3ea-4ef0-949c-c2f210aa3174-kube-api-access-pxtd6\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-lrp9w\" (UID: \"a9ba08fa-d3ea-4ef0-949c-c2f210aa3174\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-lrp9w" May 28 16:17:46.485507 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:46.485474 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-lrp9w" May 28 16:17:46.650969 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:46.650941 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-lrp9w"] May 28 16:17:46.653148 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:17:46.653123 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9ba08fa_d3ea_4ef0_949c_c2f210aa3174.slice/crio-3de2c19884c8340c637005a988b76f9b0a411c9678ec963f743beb8519ebd96d WatchSource:0}: Error finding container 3de2c19884c8340c637005a988b76f9b0a411c9678ec963f743beb8519ebd96d: Status 404 returned error can't find the container with id 3de2c19884c8340c637005a988b76f9b0a411c9678ec963f743beb8519ebd96d May 28 16:17:47.295821 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:47.295780 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-lrp9w" event={"ID":"a9ba08fa-d3ea-4ef0-949c-c2f210aa3174","Type":"ContainerStarted","Data":"3de2c19884c8340c637005a988b76f9b0a411c9678ec963f743beb8519ebd96d"} May 28 16:17:53.323456 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:53.323416 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-lrp9w" event={"ID":"a9ba08fa-d3ea-4ef0-949c-c2f210aa3174","Type":"ContainerStarted","Data":"286ce5c68c13a98629d7452dcef28217ff98a096b8fbda6fa738ed18b21cd359"} May 28 16:17:59.345790 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:59.345750 2566 generic.go:358] "Generic (PLEG): container finished" podID="a9ba08fa-d3ea-4ef0-949c-c2f210aa3174" containerID="286ce5c68c13a98629d7452dcef28217ff98a096b8fbda6fa738ed18b21cd359" exitCode=0 May 28 16:17:59.346165 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:59.345822 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-lrp9w" event={"ID":"a9ba08fa-d3ea-4ef0-949c-c2f210aa3174","Type":"ContainerDied","Data":"286ce5c68c13a98629d7452dcef28217ff98a096b8fbda6fa738ed18b21cd359"} May 28 16:17:59.346503 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:17:59.346489 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 28 16:18:01.355060 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:01.355021 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-lrp9w" event={"ID":"a9ba08fa-d3ea-4ef0-949c-c2f210aa3174","Type":"ContainerStarted","Data":"d1793fc3c9c131d0de90fa212b1c0a9651ec224e2be184e9051d88cd67641895"} May 28 16:18:01.355551 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:01.355206 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-lrp9w" May 28 16:18:01.374865 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:01.374810 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-lrp9w" podStartSLOduration=1.470049474 podStartE2EDuration="15.374795155s" podCreationTimestamp="2026-05-28 16:17:46 +0000 UTC" firstStartedPulling="2026-05-28 16:17:46.65497784 +0000 UTC m=+605.882008378" lastFinishedPulling="2026-05-28 16:18:00.559723512 +0000 UTC m=+619.786754059" observedRunningTime="2026-05-28 16:18:01.372259673 +0000 UTC m=+620.599290228" watchObservedRunningTime="2026-05-28 16:18:01.374795155 +0000 UTC m=+620.601825711" May 28 16:18:12.378214 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:12.378179 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-lrp9w" May 28 16:18:19.552339 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:19.552293 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv"] May 28 16:18:19.587366 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:19.587329 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv"] May 28 16:18:19.587520 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:19.587465 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv" May 28 16:18:19.604380 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:19.604340 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" May 28 16:18:19.682059 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:19.682012 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7gmr\" (UniqueName: \"kubernetes.io/projected/65339a31-2330-4b93-86f0-109ca6c34fd5-kube-api-access-v7gmr\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv\" (UID: \"65339a31-2330-4b93-86f0-109ca6c34fd5\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv" May 28 16:18:19.682059 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:19.682062 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65339a31-2330-4b93-86f0-109ca6c34fd5-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv\" (UID: \"65339a31-2330-4b93-86f0-109ca6c34fd5\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv" May 28 16:18:19.682268 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:19.682117 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/65339a31-2330-4b93-86f0-109ca6c34fd5-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv\" (UID: \"65339a31-2330-4b93-86f0-109ca6c34fd5\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv" May 28 16:18:19.682268 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:19.682139 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65339a31-2330-4b93-86f0-109ca6c34fd5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv\" (UID: \"65339a31-2330-4b93-86f0-109ca6c34fd5\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv" May 28 16:18:19.682268 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:19.682201 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65339a31-2330-4b93-86f0-109ca6c34fd5-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv\" (UID: \"65339a31-2330-4b93-86f0-109ca6c34fd5\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv" May 28 16:18:19.682268 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:19.682258 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65339a31-2330-4b93-86f0-109ca6c34fd5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv\" (UID: \"65339a31-2330-4b93-86f0-109ca6c34fd5\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv" May 28 16:18:19.782985 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:19.782937 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/65339a31-2330-4b93-86f0-109ca6c34fd5-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv\" (UID: \"65339a31-2330-4b93-86f0-109ca6c34fd5\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv" May 28 16:18:19.782985 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:19.782990 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65339a31-2330-4b93-86f0-109ca6c34fd5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv\" (UID: \"65339a31-2330-4b93-86f0-109ca6c34fd5\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv" May 28 16:18:19.783223 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:19.783015 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65339a31-2330-4b93-86f0-109ca6c34fd5-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv\" (UID: \"65339a31-2330-4b93-86f0-109ca6c34fd5\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv" May 28 16:18:19.783223 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:19.783053 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65339a31-2330-4b93-86f0-109ca6c34fd5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv\" (UID: \"65339a31-2330-4b93-86f0-109ca6c34fd5\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv" May 28 16:18:19.783223 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:19.783077 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v7gmr\" (UniqueName: \"kubernetes.io/projected/65339a31-2330-4b93-86f0-109ca6c34fd5-kube-api-access-v7gmr\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv\" (UID: \"65339a31-2330-4b93-86f0-109ca6c34fd5\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv" May 28 16:18:19.783223 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:19.783100 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65339a31-2330-4b93-86f0-109ca6c34fd5-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv\" (UID: \"65339a31-2330-4b93-86f0-109ca6c34fd5\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv" May 28 16:18:19.783473 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:19.783454 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/65339a31-2330-4b93-86f0-109ca6c34fd5-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv\" (UID: \"65339a31-2330-4b93-86f0-109ca6c34fd5\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv" May 28 16:18:19.783530 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:19.783468 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65339a31-2330-4b93-86f0-109ca6c34fd5-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv\" (UID: \"65339a31-2330-4b93-86f0-109ca6c34fd5\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv" May 28 16:18:19.783587 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:19.783556 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65339a31-2330-4b93-86f0-109ca6c34fd5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv\" (UID: \"65339a31-2330-4b93-86f0-109ca6c34fd5\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv" May 28 16:18:19.785500 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:19.785478 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65339a31-2330-4b93-86f0-109ca6c34fd5-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv\" (UID: \"65339a31-2330-4b93-86f0-109ca6c34fd5\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv" May 28 16:18:19.785833 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:19.785812 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65339a31-2330-4b93-86f0-109ca6c34fd5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv\" (UID: \"65339a31-2330-4b93-86f0-109ca6c34fd5\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv" May 28 16:18:19.790350 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:19.790330 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7gmr\" (UniqueName: \"kubernetes.io/projected/65339a31-2330-4b93-86f0-109ca6c34fd5-kube-api-access-v7gmr\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv\" (UID: \"65339a31-2330-4b93-86f0-109ca6c34fd5\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv" May 28 16:18:19.899007 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:19.898917 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv" May 28 16:18:20.038707 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:20.038680 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv"] May 28 16:18:20.040503 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:18:20.040468 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65339a31_2330_4b93_86f0_109ca6c34fd5.slice/crio-7fce8a4c0c8ecd7c82495daa2125df15cdeea9f9241afef4c077ea2d2514a023 WatchSource:0}: Error finding container 7fce8a4c0c8ecd7c82495daa2125df15cdeea9f9241afef4c077ea2d2514a023: Status 404 returned error can't find the container with id 7fce8a4c0c8ecd7c82495daa2125df15cdeea9f9241afef4c077ea2d2514a023 May 28 16:18:20.414428 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:20.414383 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv" event={"ID":"65339a31-2330-4b93-86f0-109ca6c34fd5","Type":"ContainerStarted","Data":"6b7d85743e81178bc5fd4c2978894c405ab3ffe4d3450ff797ea6607053b977c"} May 28 16:18:20.414428 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:20.414430 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv" event={"ID":"65339a31-2330-4b93-86f0-109ca6c34fd5","Type":"ContainerStarted","Data":"7fce8a4c0c8ecd7c82495daa2125df15cdeea9f9241afef4c077ea2d2514a023"} May 28 16:18:26.439317 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:26.439259 2566 generic.go:358] "Generic (PLEG): container finished" podID="65339a31-2330-4b93-86f0-109ca6c34fd5" containerID="6b7d85743e81178bc5fd4c2978894c405ab3ffe4d3450ff797ea6607053b977c" exitCode=0 May 28 16:18:26.439317 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:26.439318 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv" event={"ID":"65339a31-2330-4b93-86f0-109ca6c34fd5","Type":"ContainerDied","Data":"6b7d85743e81178bc5fd4c2978894c405ab3ffe4d3450ff797ea6607053b977c"} May 28 16:18:27.444658 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:27.444625 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv" event={"ID":"65339a31-2330-4b93-86f0-109ca6c34fd5","Type":"ContainerStarted","Data":"ed15df083de296737347900d80943240b136080b9cdbd40f1ec1ba2bf4a35eb2"} May 28 16:18:27.445119 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:27.444887 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv" May 28 16:18:27.467201 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:27.467132 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv" podStartSLOduration=8.130818356 podStartE2EDuration="8.467114234s" podCreationTimestamp="2026-05-28 16:18:19 +0000 UTC" firstStartedPulling="2026-05-28 16:18:26.439924079 +0000 UTC m=+645.666954613" lastFinishedPulling="2026-05-28 16:18:26.776219954 +0000 UTC m=+646.003250491" observedRunningTime="2026-05-28 16:18:27.466755513 +0000 UTC m=+646.693786071" watchObservedRunningTime="2026-05-28 16:18:27.467114234 +0000 UTC m=+646.694144792" May 28 16:18:38.462498 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:38.462461 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv" May 28 16:18:50.451547 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:50.451503 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27"] May 28 16:18:50.456666 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:50.456639 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27" May 28 16:18:50.464373 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:50.464332 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27"] May 28 16:18:50.467378 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:50.467350 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" May 28 16:18:50.562388 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:50.562346 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fd005158-0229-48c0-8ce9-b869b8add8f4-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27\" (UID: \"fd005158-0229-48c0-8ce9-b869b8add8f4\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27" May 28 16:18:50.562561 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:50.562412 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd005158-0229-48c0-8ce9-b869b8add8f4-model-cache\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27\" (UID: \"fd005158-0229-48c0-8ce9-b869b8add8f4\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27" May 28 16:18:50.562561 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:50.562436 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd005158-0229-48c0-8ce9-b869b8add8f4-tls-certs\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27\" (UID: \"fd005158-0229-48c0-8ce9-b869b8add8f4\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27" May 28 16:18:50.562561 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:50.562503 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd005158-0229-48c0-8ce9-b869b8add8f4-dshm\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27\" (UID: \"fd005158-0229-48c0-8ce9-b869b8add8f4\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27" May 28 16:18:50.562561 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:50.562552 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdkmf\" (UniqueName: \"kubernetes.io/projected/fd005158-0229-48c0-8ce9-b869b8add8f4-kube-api-access-xdkmf\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27\" (UID: \"fd005158-0229-48c0-8ce9-b869b8add8f4\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27" May 28 16:18:50.562698 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:50.562588 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd005158-0229-48c0-8ce9-b869b8add8f4-home\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27\" (UID: \"fd005158-0229-48c0-8ce9-b869b8add8f4\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27" May 28 16:18:50.663399 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:50.663364 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd005158-0229-48c0-8ce9-b869b8add8f4-tls-certs\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27\" (UID: \"fd005158-0229-48c0-8ce9-b869b8add8f4\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27" May 28 16:18:50.663613 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:50.663414 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd005158-0229-48c0-8ce9-b869b8add8f4-dshm\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27\" (UID: \"fd005158-0229-48c0-8ce9-b869b8add8f4\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27" May 28 16:18:50.663613 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:50.663441 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xdkmf\" (UniqueName: \"kubernetes.io/projected/fd005158-0229-48c0-8ce9-b869b8add8f4-kube-api-access-xdkmf\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27\" (UID: \"fd005158-0229-48c0-8ce9-b869b8add8f4\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27" May 28 16:18:50.663613 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:50.663477 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd005158-0229-48c0-8ce9-b869b8add8f4-home\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27\" (UID: \"fd005158-0229-48c0-8ce9-b869b8add8f4\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27" May 28 16:18:50.663613 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:50.663495 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fd005158-0229-48c0-8ce9-b869b8add8f4-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27\" (UID: \"fd005158-0229-48c0-8ce9-b869b8add8f4\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27" May 28 16:18:50.663613 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:50.663544 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd005158-0229-48c0-8ce9-b869b8add8f4-model-cache\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27\" (UID: \"fd005158-0229-48c0-8ce9-b869b8add8f4\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27" May 28 16:18:50.663936 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:50.663913 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd005158-0229-48c0-8ce9-b869b8add8f4-home\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27\" (UID: \"fd005158-0229-48c0-8ce9-b869b8add8f4\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27" May 28 16:18:50.663976 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:50.663936 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd005158-0229-48c0-8ce9-b869b8add8f4-model-cache\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27\" (UID: \"fd005158-0229-48c0-8ce9-b869b8add8f4\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27" May 28 16:18:50.664021 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:50.663997 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fd005158-0229-48c0-8ce9-b869b8add8f4-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27\" (UID: \"fd005158-0229-48c0-8ce9-b869b8add8f4\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27" May 28 16:18:50.665827 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:50.665804 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd005158-0229-48c0-8ce9-b869b8add8f4-dshm\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27\" (UID: \"fd005158-0229-48c0-8ce9-b869b8add8f4\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27" May 28 16:18:50.666059 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:50.666041 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd005158-0229-48c0-8ce9-b869b8add8f4-tls-certs\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27\" (UID: \"fd005158-0229-48c0-8ce9-b869b8add8f4\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27" May 28 16:18:50.672303 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:50.672263 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdkmf\" (UniqueName: \"kubernetes.io/projected/fd005158-0229-48c0-8ce9-b869b8add8f4-kube-api-access-xdkmf\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27\" (UID: \"fd005158-0229-48c0-8ce9-b869b8add8f4\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27" May 28 16:18:50.769347 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:50.769299 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27" May 28 16:18:50.905347 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:50.905325 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27"] May 28 16:18:50.909057 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:18:50.908993 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd005158_0229_48c0_8ce9_b869b8add8f4.slice/crio-ee2d9d26e355e342e01caf16794f34d39a00d8892ea346c8e8c9fbea1fc7118c WatchSource:0}: Error finding container ee2d9d26e355e342e01caf16794f34d39a00d8892ea346c8e8c9fbea1fc7118c: Status 404 returned error can't find the container with id ee2d9d26e355e342e01caf16794f34d39a00d8892ea346c8e8c9fbea1fc7118c May 28 16:18:51.528990 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:51.528946 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27" event={"ID":"fd005158-0229-48c0-8ce9-b869b8add8f4","Type":"ContainerStarted","Data":"5f81c30404684952e3d19de460313031cd31204d5d295cec158e4026982207cc"} May 28 16:18:51.528990 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:51.528988 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27" event={"ID":"fd005158-0229-48c0-8ce9-b869b8add8f4","Type":"ContainerStarted","Data":"ee2d9d26e355e342e01caf16794f34d39a00d8892ea346c8e8c9fbea1fc7118c"} May 28 16:18:56.546257 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:56.546223 2566 generic.go:358] "Generic (PLEG): container finished" podID="fd005158-0229-48c0-8ce9-b869b8add8f4" containerID="5f81c30404684952e3d19de460313031cd31204d5d295cec158e4026982207cc" exitCode=0 May 28 16:18:56.546623 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:56.546313 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27" event={"ID":"fd005158-0229-48c0-8ce9-b869b8add8f4","Type":"ContainerDied","Data":"5f81c30404684952e3d19de460313031cd31204d5d295cec158e4026982207cc"} May 28 16:18:57.551396 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:57.551358 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27" event={"ID":"fd005158-0229-48c0-8ce9-b869b8add8f4","Type":"ContainerStarted","Data":"30487e57de71f3936684faa43259dc5dc93a0b60400b925d91a6604a1bcb8f43"} May 28 16:18:57.551769 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:57.551587 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27" May 28 16:18:57.579431 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:18:57.579379 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27" podStartSLOduration=7.119633316 podStartE2EDuration="7.57936538s" podCreationTimestamp="2026-05-28 16:18:50 +0000 UTC" firstStartedPulling="2026-05-28 16:18:56.546896504 +0000 UTC m=+675.773927039" lastFinishedPulling="2026-05-28 16:18:57.006628556 +0000 UTC m=+676.233659103" observedRunningTime="2026-05-28 16:18:57.576994854 +0000 UTC m=+676.804025410" watchObservedRunningTime="2026-05-28 16:18:57.57936538 +0000 UTC m=+676.806395956" May 28 16:19:08.567179 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:19:08.567146 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27" May 28 16:19:33.340152 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:19:33.340121 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-85bd9b9885-kj45q"] May 28 16:19:33.340711 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:19:33.340330 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-85bd9b9885-kj45q" podUID="7c2708a4-2139-4ada-845f-9cf4248a1ac8" containerName="authorino" containerID="cri-o://bb33238390d27759a58531172d583620aa99d8835fd5d288b3e0eca324436f28" gracePeriod=30 May 28 16:19:33.591077 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:19:33.591016 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85bd9b9885-kj45q" May 28 16:19:33.656455 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:19:33.656402 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4f8bc\" (UniqueName: \"kubernetes.io/projected/7c2708a4-2139-4ada-845f-9cf4248a1ac8-kube-api-access-4f8bc\") pod \"7c2708a4-2139-4ada-845f-9cf4248a1ac8\" (UID: \"7c2708a4-2139-4ada-845f-9cf4248a1ac8\") " May 28 16:19:33.656661 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:19:33.656514 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7c2708a4-2139-4ada-845f-9cf4248a1ac8-tls-cert\") pod \"7c2708a4-2139-4ada-845f-9cf4248a1ac8\" (UID: \"7c2708a4-2139-4ada-845f-9cf4248a1ac8\") " May 28 16:19:33.658705 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:19:33.658670 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c2708a4-2139-4ada-845f-9cf4248a1ac8-kube-api-access-4f8bc" (OuterVolumeSpecName: "kube-api-access-4f8bc") pod "7c2708a4-2139-4ada-845f-9cf4248a1ac8" (UID: "7c2708a4-2139-4ada-845f-9cf4248a1ac8"). InnerVolumeSpecName "kube-api-access-4f8bc". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 16:19:33.666902 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:19:33.666875 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c2708a4-2139-4ada-845f-9cf4248a1ac8-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "7c2708a4-2139-4ada-845f-9cf4248a1ac8" (UID: "7c2708a4-2139-4ada-845f-9cf4248a1ac8"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 16:19:33.671902 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:19:33.671876 2566 generic.go:358] "Generic (PLEG): container finished" podID="7c2708a4-2139-4ada-845f-9cf4248a1ac8" containerID="bb33238390d27759a58531172d583620aa99d8835fd5d288b3e0eca324436f28" exitCode=0 May 28 16:19:33.672020 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:19:33.671925 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85bd9b9885-kj45q" May 28 16:19:33.672020 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:19:33.671953 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-85bd9b9885-kj45q" event={"ID":"7c2708a4-2139-4ada-845f-9cf4248a1ac8","Type":"ContainerDied","Data":"bb33238390d27759a58531172d583620aa99d8835fd5d288b3e0eca324436f28"} May 28 16:19:33.672020 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:19:33.671988 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-85bd9b9885-kj45q" event={"ID":"7c2708a4-2139-4ada-845f-9cf4248a1ac8","Type":"ContainerDied","Data":"54316b3bba72d74df15e002db0204d4a12220071309708e8bb0fb5ccffee3cca"} May 28 16:19:33.672020 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:19:33.672008 2566 scope.go:117] "RemoveContainer" containerID="bb33238390d27759a58531172d583620aa99d8835fd5d288b3e0eca324436f28" May 28 16:19:33.680686 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:19:33.680667 2566 scope.go:117] "RemoveContainer" containerID="bb33238390d27759a58531172d583620aa99d8835fd5d288b3e0eca324436f28" May 28 16:19:33.680948 ip-10-0-143-102 kubenswrapper[2566]: E0528 16:19:33.680927 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb33238390d27759a58531172d583620aa99d8835fd5d288b3e0eca324436f28\": container with ID starting with bb33238390d27759a58531172d583620aa99d8835fd5d288b3e0eca324436f28 not found: ID does not exist" containerID="bb33238390d27759a58531172d583620aa99d8835fd5d288b3e0eca324436f28" May 28 16:19:33.680995 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:19:33.680957 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb33238390d27759a58531172d583620aa99d8835fd5d288b3e0eca324436f28"} err="failed to get container status \"bb33238390d27759a58531172d583620aa99d8835fd5d288b3e0eca324436f28\": rpc error: code = NotFound desc = could not find container \"bb33238390d27759a58531172d583620aa99d8835fd5d288b3e0eca324436f28\": container with ID starting with bb33238390d27759a58531172d583620aa99d8835fd5d288b3e0eca324436f28 not found: ID does not exist" May 28 16:19:33.691524 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:19:33.691501 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-85bd9b9885-kj45q"] May 28 16:19:33.694878 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:19:33.694856 2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-85bd9b9885-kj45q"] May 28 16:19:33.757345 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:19:33.757307 2566 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7c2708a4-2139-4ada-845f-9cf4248a1ac8-tls-cert\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" May 28 16:19:33.757345 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:19:33.757343 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4f8bc\" (UniqueName: \"kubernetes.io/projected/7c2708a4-2139-4ada-845f-9cf4248a1ac8-kube-api-access-4f8bc\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" May 28 16:19:35.337497 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:19:35.337464 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c2708a4-2139-4ada-845f-9cf4248a1ac8" path="/var/lib/kubelet/pods/7c2708a4-2139-4ada-845f-9cf4248a1ac8/volumes" May 28 16:22:41.251910 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:22:41.251881 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-db77q_d07065a1-27ff-4f50-9b8e-88d35f69bc89/ovn-acl-logging/0.log" May 28 16:22:41.252685 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:22:41.252666 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-db77q_d07065a1-27ff-4f50-9b8e-88d35f69bc89/ovn-acl-logging/0.log" May 28 16:27:41.273087 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:27:41.273054 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-db77q_d07065a1-27ff-4f50-9b8e-88d35f69bc89/ovn-acl-logging/0.log" May 28 16:27:41.274270 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:27:41.274242 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-db77q_d07065a1-27ff-4f50-9b8e-88d35f69bc89/ovn-acl-logging/0.log" May 28 16:31:14.559335 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:31:14.559294 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-ldjsl"] May 28 16:31:14.559748 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:31:14.559532 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-ldjsl" podUID="758df53e-4133-4eaa-bdbf-bcb7f469f44a" containerName="manager" containerID="cri-o://776f5f58c17a11f3570c803bd5a08580303860f69326d22280f1e27b88e7e4f2" gracePeriod=10 May 28 16:31:15.057773 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:31:15.057736 2566 generic.go:358] "Generic (PLEG): container finished" podID="758df53e-4133-4eaa-bdbf-bcb7f469f44a" containerID="776f5f58c17a11f3570c803bd5a08580303860f69326d22280f1e27b88e7e4f2" exitCode=0 May 28 16:31:15.057943 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:31:15.057813 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-ldjsl" event={"ID":"758df53e-4133-4eaa-bdbf-bcb7f469f44a","Type":"ContainerDied","Data":"776f5f58c17a11f3570c803bd5a08580303860f69326d22280f1e27b88e7e4f2"} May 28 16:31:15.611587 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:31:15.611561 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-ldjsl" May 28 16:31:15.712389 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:31:15.712345 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-46f2w\" (UniqueName: \"kubernetes.io/projected/758df53e-4133-4eaa-bdbf-bcb7f469f44a-kube-api-access-46f2w\") pod \"758df53e-4133-4eaa-bdbf-bcb7f469f44a\" (UID: \"758df53e-4133-4eaa-bdbf-bcb7f469f44a\") " May 28 16:31:15.712589 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:31:15.712425 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/758df53e-4133-4eaa-bdbf-bcb7f469f44a-extensions-socket-volume\") pod \"758df53e-4133-4eaa-bdbf-bcb7f469f44a\" (UID: \"758df53e-4133-4eaa-bdbf-bcb7f469f44a\") " May 28 16:31:15.712850 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:31:15.712820 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/758df53e-4133-4eaa-bdbf-bcb7f469f44a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "758df53e-4133-4eaa-bdbf-bcb7f469f44a" (UID: "758df53e-4133-4eaa-bdbf-bcb7f469f44a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 28 16:31:15.714785 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:31:15.714767 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/758df53e-4133-4eaa-bdbf-bcb7f469f44a-kube-api-access-46f2w" (OuterVolumeSpecName: "kube-api-access-46f2w") pod "758df53e-4133-4eaa-bdbf-bcb7f469f44a" (UID: "758df53e-4133-4eaa-bdbf-bcb7f469f44a"). InnerVolumeSpecName "kube-api-access-46f2w". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 16:31:15.813557 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:31:15.813511 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/758df53e-4133-4eaa-bdbf-bcb7f469f44a-extensions-socket-volume\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" May 28 16:31:15.813557 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:31:15.813546 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46f2w\" (UniqueName: \"kubernetes.io/projected/758df53e-4133-4eaa-bdbf-bcb7f469f44a-kube-api-access-46f2w\") on node \"ip-10-0-143-102.ec2.internal\" DevicePath \"\"" May 28 16:31:16.062379 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:31:16.062346 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-ldjsl" May 28 16:31:16.062572 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:31:16.062345 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-ldjsl" event={"ID":"758df53e-4133-4eaa-bdbf-bcb7f469f44a","Type":"ContainerDied","Data":"d4666715e1f6e49b204ed3a710e8651a48d166c3a18914946732200ca6255d6d"} May 28 16:31:16.062572 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:31:16.062470 2566 scope.go:117] "RemoveContainer" containerID="776f5f58c17a11f3570c803bd5a08580303860f69326d22280f1e27b88e7e4f2" May 28 16:31:16.083784 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:31:16.083757 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-ldjsl"] May 28 16:31:16.087756 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:31:16.087733 2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-ldjsl"] May 28 16:31:17.337399 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:31:17.337363 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="758df53e-4133-4eaa-bdbf-bcb7f469f44a" path="/var/lib/kubelet/pods/758df53e-4133-4eaa-bdbf-bcb7f469f44a/volumes" May 28 16:32:20.724206 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:32:20.724163 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-99fqv"] May 28 16:32:20.724696 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:32:20.724490 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="758df53e-4133-4eaa-bdbf-bcb7f469f44a" containerName="manager" May 28 16:32:20.724696 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:32:20.724503 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="758df53e-4133-4eaa-bdbf-bcb7f469f44a" containerName="manager" May 28 16:32:20.724696 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:32:20.724529 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c2708a4-2139-4ada-845f-9cf4248a1ac8" containerName="authorino" May 28 16:32:20.724696 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:32:20.724539 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c2708a4-2139-4ada-845f-9cf4248a1ac8" containerName="authorino" May 28 16:32:20.724696 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:32:20.724599 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="758df53e-4133-4eaa-bdbf-bcb7f469f44a" containerName="manager" May 28 16:32:20.724696 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:32:20.724615 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="7c2708a4-2139-4ada-845f-9cf4248a1ac8" containerName="authorino" May 28 16:32:20.726853 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:32:20.726831 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-99fqv" May 28 16:32:20.729472 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:32:20.729450 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-7gld9\"" May 28 16:32:20.739031 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:32:20.739010 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-99fqv"] May 28 16:32:20.789844 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:32:20.789811 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvkz9\" (UniqueName: \"kubernetes.io/projected/545a04e9-3395-44c1-9791-d2a681b87bf5-kube-api-access-kvkz9\") pod \"kuadrant-operator-controller-manager-55c7f4c975-99fqv\" (UID: \"545a04e9-3395-44c1-9791-d2a681b87bf5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-99fqv" May 28 16:32:20.790013 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:32:20.789870 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/545a04e9-3395-44c1-9791-d2a681b87bf5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-99fqv\" (UID: \"545a04e9-3395-44c1-9791-d2a681b87bf5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-99fqv" May 28 16:32:20.890888 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:32:20.890851 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kvkz9\" (UniqueName: \"kubernetes.io/projected/545a04e9-3395-44c1-9791-d2a681b87bf5-kube-api-access-kvkz9\") pod \"kuadrant-operator-controller-manager-55c7f4c975-99fqv\" (UID: \"545a04e9-3395-44c1-9791-d2a681b87bf5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-99fqv" May 28 16:32:20.891060 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:32:20.890909 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/545a04e9-3395-44c1-9791-d2a681b87bf5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-99fqv\" (UID: \"545a04e9-3395-44c1-9791-d2a681b87bf5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-99fqv" May 28 16:32:20.891259 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:32:20.891242 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/545a04e9-3395-44c1-9791-d2a681b87bf5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-99fqv\" (UID: \"545a04e9-3395-44c1-9791-d2a681b87bf5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-99fqv" May 28 16:32:20.899223 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:32:20.899190 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvkz9\" (UniqueName: \"kubernetes.io/projected/545a04e9-3395-44c1-9791-d2a681b87bf5-kube-api-access-kvkz9\") pod \"kuadrant-operator-controller-manager-55c7f4c975-99fqv\" (UID: \"545a04e9-3395-44c1-9791-d2a681b87bf5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-99fqv" May 28 16:32:21.037383 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:32:21.037264 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-99fqv" May 28 16:32:21.174618 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:32:21.174581 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-99fqv"] May 28 16:32:21.177912 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:32:21.177884 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod545a04e9_3395_44c1_9791_d2a681b87bf5.slice/crio-4df3d7bb72ad1b719f1b511cb25ad21a86f3c452b1794b3b92a5daaba81934a2 WatchSource:0}: Error finding container 4df3d7bb72ad1b719f1b511cb25ad21a86f3c452b1794b3b92a5daaba81934a2: Status 404 returned error can't find the container with id 4df3d7bb72ad1b719f1b511cb25ad21a86f3c452b1794b3b92a5daaba81934a2 May 28 16:32:21.180347 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:32:21.180328 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 28 16:32:21.281507 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:32:21.281468 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-99fqv" event={"ID":"545a04e9-3395-44c1-9791-d2a681b87bf5","Type":"ContainerStarted","Data":"1e2c825182c0b8932db34def3faed8a3991acf2283b6a2f6e597e9197d98ba50"} May 28 16:32:21.281507 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:32:21.281510 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-99fqv" event={"ID":"545a04e9-3395-44c1-9791-d2a681b87bf5","Type":"ContainerStarted","Data":"4df3d7bb72ad1b719f1b511cb25ad21a86f3c452b1794b3b92a5daaba81934a2"} May 28 16:32:21.281730 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:32:21.281584 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-99fqv" May 28 16:32:21.319098 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:32:21.318972 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-99fqv" podStartSLOduration=1.318953653 podStartE2EDuration="1.318953653s" podCreationTimestamp="2026-05-28 16:32:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 16:32:21.317317113 +0000 UTC m=+1480.544347667" watchObservedRunningTime="2026-05-28 16:32:21.318953653 +0000 UTC m=+1480.545984210" May 28 16:32:32.287451 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:32:32.287414 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-99fqv" May 28 16:32:41.295029 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:32:41.294994 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-db77q_d07065a1-27ff-4f50-9b8e-88d35f69bc89/ovn-acl-logging/0.log" May 28 16:32:41.296399 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:32:41.296376 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-db77q_d07065a1-27ff-4f50-9b8e-88d35f69bc89/ovn-acl-logging/0.log" May 28 16:37:41.319433 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:37:41.319399 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-db77q_d07065a1-27ff-4f50-9b8e-88d35f69bc89/ovn-acl-logging/0.log" May 28 16:37:41.321170 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:37:41.321151 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-db77q_d07065a1-27ff-4f50-9b8e-88d35f69bc89/ovn-acl-logging/0.log" May 28 16:42:11.305827 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:11.305791 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-95cfdd646-mpl9q_73ef29d0-0d5c-4419-8b95-b5fd0b0ec7aa/manager/0.log" May 28 16:42:11.542207 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:11.542164 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-mnfdq_415ee9e9-d535-4622-8634-5aed28c23d70/postgres/0.log" May 28 16:42:13.073126 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:13.073094 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-pldc9_c5233b77-fbae-4067-8517-b7351a3dbb42/manager/0.log" May 28 16:42:13.318776 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:13.318742 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-528wb_c671d728-dfff-445d-8ad0-df325c0494f7/registry-server/0.log" May 28 16:42:13.461208 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:13.461122 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-99fqv_545a04e9-3395-44c1-9791-d2a681b87bf5/manager/0.log" May 28 16:42:14.075965 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:14.075934 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-686dffbf4bztfxj_6f370873-429d-4cfd-9a83-be20b295ffa8/istio-proxy/0.log" May 28 16:42:14.569804 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:14.569771 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d6f447cf5-xrk5v_73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499/istio-proxy/0.log" May 28 16:42:15.309747 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:15.309714 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27_fd005158-0229-48c0-8ce9-b869b8add8f4/storage-initializer/0.log" May 28 16:42:15.317112 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:15.317089 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7bb4cdb4d7-gcc27_fd005158-0229-48c0-8ce9-b869b8add8f4/main/0.log" May 28 16:42:15.435866 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:15.435836 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84db68679b-lrp9w_a9ba08fa-d3ea-4ef0-949c-c2f210aa3174/storage-initializer/0.log" May 28 16:42:15.443741 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:15.443710 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84db68679b-lrp9w_a9ba08fa-d3ea-4ef0-949c-c2f210aa3174/main/0.log" May 28 16:42:15.685252 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:15.685163 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv_65339a31-2330-4b93-86f0-109ca6c34fd5/storage-initializer/0.log" May 28 16:42:15.692751 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:15.692722 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-8f8dc67b7-5s5xv_65339a31-2330-4b93-86f0-109ca6c34fd5/main/0.log" May 28 16:42:22.767256 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:22.767210 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-pb7mn_b113a9e6-a1e9-43bf-a08c-b7e6e466640f/global-pull-secret-syncer/0.log" May 28 16:42:22.915152 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:22.915109 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-jq2b6_c0258ed5-55e5-4a30-a658-3e8d61330e0a/konnectivity-agent/0.log" May 28 16:42:22.978879 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:22.978846 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-102.ec2.internal_3682fbf2c94d063cca39e2c160569366/haproxy/0.log" May 28 16:42:27.423600 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:27.423562 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-pldc9_c5233b77-fbae-4067-8517-b7351a3dbb42/manager/0.log" May 28 16:42:27.490838 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:27.490799 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-528wb_c671d728-dfff-445d-8ad0-df325c0494f7/registry-server/0.log" May 28 16:42:27.580872 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:27.580838 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-99fqv_545a04e9-3395-44c1-9791-d2a681b87bf5/manager/0.log" May 28 16:42:29.444608 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:29.444574 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rwqpn_2c6458fc-1987-48f7-abc9-289365a79370/node-exporter/0.log" May 28 16:42:29.464530 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:29.464502 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rwqpn_2c6458fc-1987-48f7-abc9-289365a79370/kube-rbac-proxy/0.log" May 28 16:42:29.484944 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:29.484910 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rwqpn_2c6458fc-1987-48f7-abc9-289365a79370/init-textfile/0.log" May 28 16:42:30.844928 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:30.844887 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ds4l8/perf-node-gather-daemonset-gdm67"] May 28 16:42:30.847433 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:30.847415 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ds4l8/perf-node-gather-daemonset-gdm67" May 28 16:42:30.849809 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:30.849782 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ds4l8\"/\"kube-root-ca.crt\"" May 28 16:42:30.850923 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:30.850904 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ds4l8\"/\"default-dockercfg-9xvg7\"" May 28 16:42:30.851010 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:30.850908 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ds4l8\"/\"openshift-service-ca.crt\"" May 28 16:42:30.857251 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:30.857227 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ds4l8/perf-node-gather-daemonset-gdm67"] May 28 16:42:30.946942 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:30.946902 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02bd1d30-0903-4775-a4c6-b93303e64e56-sys\") pod \"perf-node-gather-daemonset-gdm67\" (UID: \"02bd1d30-0903-4775-a4c6-b93303e64e56\") " pod="openshift-must-gather-ds4l8/perf-node-gather-daemonset-gdm67" May 28 16:42:30.946942 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:30.946947 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/02bd1d30-0903-4775-a4c6-b93303e64e56-podres\") pod \"perf-node-gather-daemonset-gdm67\" (UID: \"02bd1d30-0903-4775-a4c6-b93303e64e56\") " pod="openshift-must-gather-ds4l8/perf-node-gather-daemonset-gdm67" May 28 16:42:30.947167 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:30.946967 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/02bd1d30-0903-4775-a4c6-b93303e64e56-proc\") pod \"perf-node-gather-daemonset-gdm67\" (UID: \"02bd1d30-0903-4775-a4c6-b93303e64e56\") " pod="openshift-must-gather-ds4l8/perf-node-gather-daemonset-gdm67" May 28 16:42:30.947167 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:30.947000 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv58p\" (UniqueName: \"kubernetes.io/projected/02bd1d30-0903-4775-a4c6-b93303e64e56-kube-api-access-nv58p\") pod \"perf-node-gather-daemonset-gdm67\" (UID: \"02bd1d30-0903-4775-a4c6-b93303e64e56\") " pod="openshift-must-gather-ds4l8/perf-node-gather-daemonset-gdm67" May 28 16:42:30.947167 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:30.947053 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/02bd1d30-0903-4775-a4c6-b93303e64e56-lib-modules\") pod \"perf-node-gather-daemonset-gdm67\" (UID: \"02bd1d30-0903-4775-a4c6-b93303e64e56\") " pod="openshift-must-gather-ds4l8/perf-node-gather-daemonset-gdm67" May 28 16:42:31.048147 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:31.048108 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02bd1d30-0903-4775-a4c6-b93303e64e56-sys\") pod \"perf-node-gather-daemonset-gdm67\" (UID: \"02bd1d30-0903-4775-a4c6-b93303e64e56\") " pod="openshift-must-gather-ds4l8/perf-node-gather-daemonset-gdm67" May 28 16:42:31.048147 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:31.048149 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/02bd1d30-0903-4775-a4c6-b93303e64e56-podres\") pod \"perf-node-gather-daemonset-gdm67\" (UID: \"02bd1d30-0903-4775-a4c6-b93303e64e56\") " pod="openshift-must-gather-ds4l8/perf-node-gather-daemonset-gdm67" May 28 16:42:31.048392 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:31.048170 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/02bd1d30-0903-4775-a4c6-b93303e64e56-proc\") pod \"perf-node-gather-daemonset-gdm67\" (UID: \"02bd1d30-0903-4775-a4c6-b93303e64e56\") " pod="openshift-must-gather-ds4l8/perf-node-gather-daemonset-gdm67" May 28 16:42:31.048392 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:31.048222 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nv58p\" (UniqueName: \"kubernetes.io/projected/02bd1d30-0903-4775-a4c6-b93303e64e56-kube-api-access-nv58p\") pod \"perf-node-gather-daemonset-gdm67\" (UID: \"02bd1d30-0903-4775-a4c6-b93303e64e56\") " pod="openshift-must-gather-ds4l8/perf-node-gather-daemonset-gdm67" May 28 16:42:31.048392 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:31.048325 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02bd1d30-0903-4775-a4c6-b93303e64e56-sys\") pod \"perf-node-gather-daemonset-gdm67\" (UID: \"02bd1d30-0903-4775-a4c6-b93303e64e56\") " pod="openshift-must-gather-ds4l8/perf-node-gather-daemonset-gdm67" May 28 16:42:31.048392 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:31.048337 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/02bd1d30-0903-4775-a4c6-b93303e64e56-proc\") pod \"perf-node-gather-daemonset-gdm67\" (UID: \"02bd1d30-0903-4775-a4c6-b93303e64e56\") " pod="openshift-must-gather-ds4l8/perf-node-gather-daemonset-gdm67" May 28 16:42:31.048392 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:31.048343 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/02bd1d30-0903-4775-a4c6-b93303e64e56-lib-modules\") pod \"perf-node-gather-daemonset-gdm67\" (UID: \"02bd1d30-0903-4775-a4c6-b93303e64e56\") " pod="openshift-must-gather-ds4l8/perf-node-gather-daemonset-gdm67" May 28 16:42:31.048392 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:31.048336 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/02bd1d30-0903-4775-a4c6-b93303e64e56-podres\") pod \"perf-node-gather-daemonset-gdm67\" (UID: \"02bd1d30-0903-4775-a4c6-b93303e64e56\") " pod="openshift-must-gather-ds4l8/perf-node-gather-daemonset-gdm67" May 28 16:42:31.048786 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:31.048431 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/02bd1d30-0903-4775-a4c6-b93303e64e56-lib-modules\") pod \"perf-node-gather-daemonset-gdm67\" (UID: \"02bd1d30-0903-4775-a4c6-b93303e64e56\") " pod="openshift-must-gather-ds4l8/perf-node-gather-daemonset-gdm67" May 28 16:42:31.056192 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:31.056170 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv58p\" (UniqueName: \"kubernetes.io/projected/02bd1d30-0903-4775-a4c6-b93303e64e56-kube-api-access-nv58p\") pod \"perf-node-gather-daemonset-gdm67\" (UID: \"02bd1d30-0903-4775-a4c6-b93303e64e56\") " pod="openshift-must-gather-ds4l8/perf-node-gather-daemonset-gdm67" May 28 16:42:31.158798 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:31.158694 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ds4l8/perf-node-gather-daemonset-gdm67" May 28 16:42:31.295150 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:31.295124 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ds4l8/perf-node-gather-daemonset-gdm67"] May 28 16:42:31.298040 ip-10-0-143-102 kubenswrapper[2566]: W0528 16:42:31.298005 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod02bd1d30_0903_4775_a4c6_b93303e64e56.slice/crio-b3adf0d743dcf1a09cb63d81c91f37b2efcfe1d8651b3eca564e91ca6eb6fd5a WatchSource:0}: Error finding container b3adf0d743dcf1a09cb63d81c91f37b2efcfe1d8651b3eca564e91ca6eb6fd5a: Status 404 returned error can't find the container with id b3adf0d743dcf1a09cb63d81c91f37b2efcfe1d8651b3eca564e91ca6eb6fd5a May 28 16:42:31.299786 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:31.299768 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 28 16:42:31.343632 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:31.343607 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ds4l8/perf-node-gather-daemonset-gdm67" event={"ID":"02bd1d30-0903-4775-a4c6-b93303e64e56","Type":"ContainerStarted","Data":"b3adf0d743dcf1a09cb63d81c91f37b2efcfe1d8651b3eca564e91ca6eb6fd5a"} May 28 16:42:31.923920 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:31.923882 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5cccdf6b74-q7q8r_19e724a8-a757-4dbb-b236-7cdb069ee6e8/console/0.log" May 28 16:42:32.348335 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:32.348267 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ds4l8/perf-node-gather-daemonset-gdm67" event={"ID":"02bd1d30-0903-4775-a4c6-b93303e64e56","Type":"ContainerStarted","Data":"e4a604e685b9317513967eb35433b90c545db434849214ad9afceafb12290d93"} May 28 16:42:32.348335 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:32.348342 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ds4l8/perf-node-gather-daemonset-gdm67" May 28 16:42:32.362761 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:32.362715 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-ds4l8/perf-node-gather-daemonset-gdm67" podStartSLOduration=2.362701524 podStartE2EDuration="2.362701524s" podCreationTimestamp="2026-05-28 16:42:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 16:42:32.36192014 +0000 UTC m=+2091.588950698" watchObservedRunningTime="2026-05-28 16:42:32.362701524 +0000 UTC m=+2091.589732079" May 28 16:42:33.193143 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:33.193118 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cwtzt_8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5/dns/0.log" May 28 16:42:33.213223 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:33.213170 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cwtzt_8d9c00b9-39ab-4064-9cbc-7bc8ce8cfde5/kube-rbac-proxy/0.log" May 28 16:42:33.292604 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:33.292574 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4k22m_dc91efd9-d166-4a72-bcdc-3247cc10fce8/dns-node-resolver/0.log" May 28 16:42:33.761099 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:33.761056 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-658d7dd74-s7f52_277f3545-b472-4ea1-be1a-42144d81f7ed/registry/0.log" May 28 16:42:33.799384 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:33.799341 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-z55zs_5ae4021e-a738-403c-9143-976b0c75fca8/node-ca/0.log" May 28 16:42:34.556395 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:34.556365 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-686dffbf4bztfxj_6f370873-429d-4cfd-9a83-be20b295ffa8/istio-proxy/0.log" May 28 16:42:34.772522 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:34.772487 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d6f447cf5-xrk5v_73f49f56-c9e6-4ce3-9ce3-cbd1b0f07499/istio-proxy/0.log" May 28 16:42:35.349493 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:35.349456 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zg8g7_b104af3f-b526-4a24-956d-4d85f22a39c1/serve-healthcheck-canary/0.log" May 28 16:42:35.839030 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:35.838996 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7x5pb_acb19e01-6353-4282-b75d-095f260c2083/kube-rbac-proxy/0.log" May 28 16:42:35.861677 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:35.861645 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7x5pb_acb19e01-6353-4282-b75d-095f260c2083/exporter/0.log" May 28 16:42:35.881624 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:35.881578 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7x5pb_acb19e01-6353-4282-b75d-095f260c2083/extractor/0.log" May 28 16:42:37.864454 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:37.864417 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-95cfdd646-mpl9q_73ef29d0-0d5c-4419-8b95-b5fd0b0ec7aa/manager/0.log" May 28 16:42:37.921264 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:37.921235 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-mnfdq_415ee9e9-d535-4622-8634-5aed28c23d70/postgres/0.log" May 28 16:42:38.361716 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:38.361689 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ds4l8/perf-node-gather-daemonset-gdm67" May 28 16:42:38.925230 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:38.925197 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-677666f8d9-2rlmg_03e20f78-fb09-4536-95cd-3eddaaf2615f/manager/0.log" May 28 16:42:41.342606 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:41.342575 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-db77q_d07065a1-27ff-4f50-9b8e-88d35f69bc89/ovn-acl-logging/0.log" May 28 16:42:41.343971 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:41.343953 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-db77q_d07065a1-27ff-4f50-9b8e-88d35f69bc89/ovn-acl-logging/0.log" May 28 16:42:44.345080 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:44.345047 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j8ptw_714f97cf-9a9a-4321-a50d-511802063910/kube-multus-additional-cni-plugins/0.log" May 28 16:42:44.365529 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:44.365498 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j8ptw_714f97cf-9a9a-4321-a50d-511802063910/egress-router-binary-copy/0.log" May 28 16:42:44.382923 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:44.382891 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j8ptw_714f97cf-9a9a-4321-a50d-511802063910/cni-plugins/0.log" May 28 16:42:44.401454 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:44.401415 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j8ptw_714f97cf-9a9a-4321-a50d-511802063910/bond-cni-plugin/0.log" May 28 16:42:44.421553 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:44.421524 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j8ptw_714f97cf-9a9a-4321-a50d-511802063910/routeoverride-cni/0.log" May 28 16:42:44.439503 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:44.439477 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j8ptw_714f97cf-9a9a-4321-a50d-511802063910/whereabouts-cni-bincopy/0.log" May 28 16:42:44.457342 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:44.457315 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j8ptw_714f97cf-9a9a-4321-a50d-511802063910/whereabouts-cni/0.log" May 28 16:42:44.629347 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:44.629249 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-b56lh_a40ad2f3-e916-4272-9a82-9b0a818900e5/kube-multus/0.log" May 28 16:42:44.698951 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:44.698895 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6dt52_3b54fd06-9913-49b9-ac98-d2916e5f0276/network-metrics-daemon/0.log" May 28 16:42:44.716658 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:44.716632 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6dt52_3b54fd06-9913-49b9-ac98-d2916e5f0276/kube-rbac-proxy/0.log" May 28 16:42:45.775398 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:45.775362 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-db77q_d07065a1-27ff-4f50-9b8e-88d35f69bc89/ovn-controller/0.log" May 28 16:42:45.790128 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:45.790098 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-db77q_d07065a1-27ff-4f50-9b8e-88d35f69bc89/ovn-acl-logging/0.log" May 28 16:42:45.805243 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:45.805204 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-db77q_d07065a1-27ff-4f50-9b8e-88d35f69bc89/ovn-acl-logging/1.log" May 28 16:42:45.824883 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:45.824847 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-db77q_d07065a1-27ff-4f50-9b8e-88d35f69bc89/kube-rbac-proxy-node/0.log" May 28 16:42:45.843257 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:45.843219 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-db77q_d07065a1-27ff-4f50-9b8e-88d35f69bc89/kube-rbac-proxy-ovn-metrics/0.log" May 28 16:42:45.857811 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:45.857777 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-db77q_d07065a1-27ff-4f50-9b8e-88d35f69bc89/northd/0.log" May 28 16:42:45.875443 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:45.875405 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-db77q_d07065a1-27ff-4f50-9b8e-88d35f69bc89/nbdb/0.log" May 28 16:42:45.894790 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:45.894755 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-db77q_d07065a1-27ff-4f50-9b8e-88d35f69bc89/sbdb/0.log" May 28 16:42:46.056439 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:46.056355 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-db77q_d07065a1-27ff-4f50-9b8e-88d35f69bc89/ovnkube-controller/0.log" May 28 16:42:47.454144 ip-10-0-143-102 kubenswrapper[2566]: I0528 16:42:47.454108 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-2nzbn_f15055af-8047-4a04-b79f-64af8836b946/network-check-target-container/0.log"